Google DeepMind ships Genie 4: photorealistic worlds, audio, and real-time steering

Google DeepMind released Genie 4, its newest world model, in research preview on Tuesday, and the model does something no earlier Genie release could: users can type prompts mid-simulation to change the scene while it runs.

The release marks Google's most aggressive step yet in a world model market that is consolidating fast. Fei-Fei Li's World Labs is being acquired by AMD, Amazon is in talks to buy Decart, and Rivian just acquired Third Dimension to bring simulation in-house for self-driving testing. Genie 4 is Google's latest move to stay ahead in that crowded field.

The model generates what Google calls its most photorealistic and interactive worlds yet, and it adds real-time audio that matches the simulated environment — music, ambient noise, and interactive dialogue included. It arrives a couple of months after AI video startup Runway released its second world model, GWM Worlds 2, a playable model that responds dynamically to user inputs and also generates audio.

What can Genie 4 actually do?

Google released Genie 4's predecessor, Genie 3, as a research preview a little over a year ago. The new version improves on it in several measured ways:

• Longer consistency. Genie 4 maintains a consistent scene for up to two minutes, double Genie 3's limit.
• Freeform real-time edits. Users can change the weather or camera angle mid-simulation, or add new people and objects to a running scene.
• Audio. The model generates sound matched to the world, including music, ambient noises, and interactive dialogue.
• Better physics and characters. Google says Genie 4 delivers improved physics and more human-like characters than its predecessor.

The technical stakes matter beyond demos. World models are pitched for gaming, interactive learning, and robotics — three markets where simulation quality and runtime consistency determine whether a model is a toy or a tool. Google offers one speculative learning example: entering a painting of a historical scene and chatting with the painting's subjects about the event portrayed.

How does Gemini fit in?

Gemini, Google's AI model, is more closely integrated with Genie 4 than with earlier versions. Gemini acts as the director: it plans steps and checks frames while Genie creates the visuals.

In a research experiment tied to Gemini Robotics, a Gemini 4 Argon model broke down tasks into steps that Genie 4 then simulated in real time. The experiment points to a concrete engineering use: testing robot scenarios in simulation before they happen in the physical world.

Tejas Kulkarni, research director at Google, framed the integration as the model of the future. "What we'll increasingly see in the future are these hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction," Kulkarni said during a briefing.

What are filmmakers doing with it?

Google's "trusted testers" — the people who get first access to the company's cutting-edge products — are pointing to a use case the company did not originally pitch: prototyping films.

Diego Rivas, a Google DeepMind engineer, explained the value during a press briefing. "The real time aspect becomes most valuable in the areas where keeping a state of flow in the creativity space becomes incredibly useful," Rivas said. "For example, in the ideation phase, in the storyboarding phase — anytime you're kind of in need of rapid ideation."

That usage would extend an existing trend inside Google's tooling. Google also offers Veo, its AI video generation model, which makes high-definition clips with built-in synchronized audio from text, image, or video prompts. Veo has been growing in popularity among filmmakers.

Rivas added that users could, eventually, rely on Gemini Omni — a multimodal large language model that can generate video from images, audio, and text — to turn Genie 4 prototypes into finished creative assets. Those capabilities, he said, are still a ways off.

What are the known weaknesses?

The DeepMind team acknowledged gaps during the briefing. Details and objects can drift the longer a simulation runs. The team has not yet figured out how to guarantee that audio will be accurate or that any in-world text will be legible.

Rivas stressed throughout the briefing that it is still early days for Genie 4 and that the end-to-end product experience remains unclear. The research preview label carries weight here: two-minute scene consistency and mid-simulation steering are research milestones, not shipping product guarantees.

Who gets access, and when?

Genie 4 is available now as a research preview for trusted testers. In the coming weeks, the model will roll out to three groups:

• Google AI Ultra subscribers, through Project Genie, Google's AI tool for generating interactive worlds from text or image prompts
• Early access partners of the Gemini API
• Google AI Studio users

Why the market context matters?

The world model industry is consolidating at the same moment its outputs are becoming photorealistic and interactive. AMD is acquiring Fei-Fei Li's World Labs. Amazon is in talks to buy Decart. Rivian has acquired Third Dimension to run self-driving testing simulations in-house.

That consolidation signals that incumbent chipmakers, cloud providers, and automakers now treat world simulation as core infrastructure rather than a research curiosity. Runway's GWM Worlds 2, released a couple of months before Genie 4, already offered playable, input-responsive worlds with audio generation. Genie 4's answer is longer consistency, freeform mid-simulation editing, and a tighter coupling to Gemini.

The robotics experiment may prove the most consequential piece. If a Gemini 4 Argon model can decompose tasks and let Genie 4 simulate them in real time, robot developers gain a way to rehearse scenarios in simulation before hardware touches the world — a workflow Rivian just paid to acquire for itself.