Google DeepMind unveiled Genie 4 on Tuesday, an AI model that generates interactive virtual worlds in real time, with a range of potential applications.
Genie 4 is natively multimodal, Google said in a blog post, meaning audio-like music, ambient noises and interactive dialogue can be generated simultaneously within the world.
More specifically, users can use natural language to modify simulations in real time, such as altering the weather, adjusting camera angles and introducing new people and objects. Genie 4 also supports interactive dialogue. The model automatically adjusts the scene to match the prompt and keep it moving cohesively, handling real-world physics scenarios, such as aerodynamics and collisions.
"The most dramatic leap is how Genie 4 simulates people, creating characters that move naturally, respond organically to their environments, and can be interactively steered in real-time," Google DeepMind's Raia Hadsell, Diego Rivas, and Hang Qi said in the post.
The model is initially being released as a research preview for a select group of testers to gather feedback before a broader release to consumers and developers, the firm said, starting with Google AI Ultra subscribers in Project Genie and early access partners of the Gemini API and Google AI Studio.
Leveraging the Gemini family
Genie 4 incorporates capabilities from Google's existing Gemini AI model family, with Gemini Omni anchoring the simulations in real-world appearance and behavior while other models handle complex reasoning to help maintain a simulation's coherence and continuity, the authors said.
Gemini 3.8 Flash serves as a "guiding agent in the loop" to help achieve real-time capabilities, continuously analyzing the scene, breaking down goals into a sequence of steps, and reviewing incoming frames to track and verify progress, according to the firm.
However, the authors cautioned that Genie 4 remains in an early area of research and while it can maintain coherent environments over short sessions, elements can drift over time, especially as simulations run longer or undergo multiple complex prompt edits.
Google DeepMind said the technology could eventually help train AI agents and robots in virtual environments, while also enabling new applications in filmmaking, video games and other interactive experiences.