跳到正文
Ruixing(Rising) Zhang· 来源· @xing_rui12683 · X·· 10 小时前AI 评分49
正文 · 原文

Genie 4 just released. And it emphasized code world model as the future work. I wrote a blog on code world model several days before:

Let Video Generation Models Do What They Do Best: Thoughts on the Rise of Code World Models

Over the past few days, several teams around the world have released Code-based World/Video Models.

A non-exhaustive list includes:
- AgentGarten by MirroS @MirroS_ai ,
- StreamRender-H3 by NVIDIA's SANA team @xieenze_jr ,
- Code Video Model by Microsoft @guocheng_qian ,
- Eden Live by Tripo @huanngzh ,
- Matrix Game 4.0 by Skywork @Skywork_ai

Personally, this approach aligns very closely with my vision of what a World Model should look like.

The general idea is roughly as follows:

Use code to maintain the world's states and rules, generate a rough draft of the scene (e.g., a semantic map), and then let a video generation model turn that draft into photorealistic video.

The advantages are fairly obvious:

1) Video models no longer need to maintain complex world states and memories.

The world's states and history are explicitly stored in code.

Where objects are, what they've experienced, how they've changed, or even things like energy levels and equipment in a game can all be maintained through code.

The video model no longer has to remember and infer everything solely from previously generated frames.

2) Video models no longer need to learn every complex law of physics from scratch.

Many physical laws, such as gravity, can already be modeled in code.

This means that if we want to move our world to another planet, or even create an entirely fictional universe, we can simply modify the corresponding physical parameters and simulation rules to construct a new environment.

3) Changing viewpoints becomes much easier.

If we want to switch camera angles, we simply modify the camera parameters in code, generate a new draft, and pass it to the video model for rendering.

Since the underlying geometry and object states are shared, maintaining consistency across viewpoints also becomes much easier.

4) Expanding the world becomes a matter of letting Agents write code.

For example, if we want to add a new neighborhood next to an existing city, we can simply have an Agent write the corresponding scene descriptions, objects, and interaction logic, then hand everything over to the video model for rendering.

So, to me, Code World Models seem like a much better approach.

But here's another question:

Could we get rid of video generation models entirely and rely solely on code and traditional renderers?

I think the answer is clearly no.

Anyone who has taken an introductory computer graphics course probably knows how difficult it is to render fire.

Fire, smoke, complex lighting, and material effects have historically been some of the most challenging problems in computer graphics.

Even today, rendering realistic fire is far from a completely solved problem.

Ultimately, a traditional renderer is still a renderer.

At its core, traditional rendering relies heavily on models and knowledge explicitly designed by humans.

But *The Bitter Lesson* tells us something important:

Methods that can effectively leverage computation and data for learning tend to have greater long-term potential than approaches that continually depend on complex, hand-engineered rules.

We should approach this problem in a more scalable way.

Instead of manually designing complicated rendering rules for every possible type of fire, why not learn directly from massive amounts of real-world video data what fire should actually look like?

To me, that feels like a much more natural approach.

And that's why I believe the current direction of Code World Models is a promising one.

Asking video generation models to handle the memory, physical laws, and state evolution of an entire world is simply asking too much of most video models today.

Thinking about it this way, it's easier to understand why so many teams have recently started exploring Code World Models.

Perhaps Code Models themselves are what will ultimately lead us to AGI.

But video generation models can still help us see a much better world.

BTW, all the best code world are on reactor @reactorworld . We have incorpted most of them in Reactor-Cookbook.

来源:Ruixing(Rising) Zhang · x.com