Learn / The Academy · 2 min read
World models
AI that guesses what happens next
A world model is AI that has learned how things move and change, so it can guess what happens next. Tip a full mop bucket and it predicts the spill, which is why robot teams use these models to let robots practice in a made-up copy of the world before they touch a real one.
What you already know without thinking
You know a wet floor stays slick for a few minutes, and that a trash bag full of paper is light while one full of coffee cups will drip. Nobody taught you that from a book, because you learned it from years on the job.
A world model tries to learn that same kind of sense from video, watching what happens over and over until it can predict the next few seconds.
Practice without breaking anything
A robot that knocks over a real bucket makes a real mess, but in a world model it can knock over 10,000 buckets and nobody has to mop.
NVIDIA offers a set of these called Cosmos, which it calls "world foundation models" and says teams use for robots and self-driving cars. They can also create made-up video to add to the real video used for training.
Why made-up video still needs real video
A world model can only guess well about things it has seen, so if it has never watched a mop on tile, its guess about a mop on tile will be off.
Ropedia describes its 10,000-hour first-person set as built for world models, among other uses. Real footage is the starting point, and the made-up video comes after.
A world model learns how things behave from watching them, and your crew works on carpet, tile, sealed concrete and glass in buildings with very different layouts. Real footage from all of those places teaches it how a mop, a rag or a trash liner behaves in each one.
What other owners ask
Is a world model the same as a video game?
It is a bit like one, except a game follows rules someone wrote and a world model learns its rules from watching real video.
If a world model can make video, why pay for real video?
It can only copy what it has learned, and real video from real buildings is what it learns from in the first place.
Where to read more
These are public examples from other companies. They are not Colby partners or customers.