An AI-based world simulator called "The Matrix" is released! Created by an all-Chinese team, the simulator can generate unlimited high-fidelity 720p real scene videos and support real-time interaction, just like entering the virtual world of the movie "The Matrix". Users can freely shuttle through various scenes such as deserts, grasslands, and cities through simple button control, and experience smooth dynamic images at 16 frames per second. The highlight of its core technology lies in unprecedented frame-level control and zero-sample generalization capabilities, which can accurately generate scenes even in untrained environments, bringing an immersive virtual experience. This project integrates AAA-level game data and a large number of real-life scene videos, and achieves real-time generation through a clever combination of interactive modules, moving window denoising process models, and flow consistency models.
The simulator provides a 14-minute demo video and can actually generate up to an hour of content. The project team members come from well-known institutions such as Alibaba and the University of Hong Kong. Its technological breakthrough lies in its learning ability and its ability to accurately generate in unseen environments, such as simulating the driving scenarios of the BMW X3 in different environments, and even "car in water" Swimming" and other wonderful pictures. In the future, the team will continue to work hard to bring users a more realistic virtual experience. The project entrance and paper link have been provided, welcome to experience and learn more about it.