A Startup Wants to Train AI World Models on Your Gameplay

A British startup called Worldmodeldata has licensed almost 1 million hours of video game data from studios behind popular titles and is packaging it as training material for world models, the class of AI that researchers such as Fei-Fei Li and Yann LeCun believe large language models cannot replace.

The bet is straightforward. LLMs are trained on text and, by extension, struggle with tasks that demand physical precision: piloting autonomous vehicles, steering robotic arms, or gripping an object without dropping it. World models are supposed to fix that by learning cause and consequence from a combination of visual and action data. The problem is supply. The internet offered oceans of text for LLMs. It offers almost nothing for world models.

"For world models, you need cause and consequence," says Xiatian Zhu, an associate professor specializing in AI at the University of Surrey. "On the internet, we have very little of this type of data."

Video games, Worldmodeldata argues, are the exception. Every session a player runs through a 3D environment generates a stream of paired data: a visual representation of the space and the precise controller inputs that produced each action. With a twirl of a thumbstick, a squeeze of a trigger, and a press of a few buttons, even an unskilled player produces sequences that map action to outcome in a simulated physical world.

The data bottleneck

The stakes here go beyond one startup's business model. Researchers across the field are working under the assumption that world models will improve in line with the size of their training datasets, much as LLMs did. If that assumption holds, the shortage of suitable training data is one of the largest bottlenecks to progress in the entire area.

Some labs have tried to manufacture their own data by attaching sensors to humans and robots in testing environments. The approach has clear limits. It yields small volumes, and it cannot account for the strange, rare situations a deployed model will eventually face.

"You can pay people to demonstrate pick-and-place tasks. But repetition alone won't capture the disorder of the world you're asking a machine to operate in," says Nicole Fraenkel, a partner at VC firm Khosla Ventures, which has invested in General Intuition, one of the companies collecting video game data from its own platform.

Those edge cases matter because the failure modes are expensive. "The corner cases are the ones to actually get right," Fraenkel says. "The cost of error with a car, plane, drone, factory forklift, or autonomous quadruped is very high."

Worldmodeldata's theory is that data harvested from game environments—3D visuals paired with player actions—is available at the necessary scale and varied enough to capture those corner cases. Games are also an exhaust product: studios already collect this information in massive quantities, and nobody has systematically turned it into AI training data.

A broker for game data

The company is not alone in spotting the opportunity. General Intuition and Niantic are already collecting gameplay data from their own platforms to build models. Worldmodeldata positions itself differently, as a broker that curates and organizes data from many studios so that AI labs do not have to strike individual agreements with each one.

"There are millions of great games, and they are more and more similar to the real world," Rhea Loucas, CEO of Worldmodeldata, told WIRED. "Why don't we take the vast, abundant, diverse experiences from video games, and teach AI?"

Loucas declined to name the studios behind the nearly 1 million hours of licensed data. The startup plans to eventually create avenues for individual players to be compensated for their contributions as well.

Her view of the endgame is specific: video game data will make up the majority of training material for world models, with fine-tuning done later on data specific to a given real-world environment or task. "This could well lead to the GPT moment for world models—making them really useful," Loucas claims.

Worldmodeldata has an advisory connection to one of the field's most prominent figures. Yann LeCun, the celebrated researcher now focused on world models as the path beyond LLMs, advises the startup.

The physics problem

Not everyone in the field shares the optimism, and the skeptics include one of the most powerful companies in AI.

Nvidia, which publishes a family of world models optimized to run on its chips, prefers a different foundation entirely. Rather than training on gameplay data, the company built a custom engine specifically designed to replicate real-world physics, and uses that as the backbone for its AI.

Ming-Yu Liu, who leads world model development at Nvidia, says models trained on video game inputs are unlikely to handle tasks requiring fine-grained motor control, such as carefully manipulating objects. His reasoning is concrete. Video game physics is often eccentric, and developers take shortcuts to create the illusion of realism. A character might dip a hand to collect an apple from a table without any code specifying the pressure each finger applies to keep the apple from slipping.

"I would be more conservative on using video game data for manipulation," Liu says. "The physics for manipulation is much more involved."

In his view, training on video game data is better reserved for world models designed to generate hyperrealistic video or 3D environments, applications where approximate physics is tolerable.

Zhu, the University of Surrey academic, raises a similar objection from the research side. "Video games are, in essence, simulators. They do have some degree of physical grounding," he says. "But they are very coarse, approximate."

The disagreement is not academic hair-splitting. It cuts to the question of what world models are actually for. If their future lies in generating synthetic video and 3D worlds, game data may be abundant and sufficient. If they are meant to control robots, forklifts, and drones in physical space, then data that approximates physics with programmer shortcuts may embed errors the models cannot easily unlearn.

An unsettled field

The hypothesis that gameplay data can carry world models to their breakout moment remains untested. No one has yet demonstrated at scale that controller inputs and rendered 3D scenes can produce a model capable of reliable real-world action. The field's own framing reflects that uncertainty.

"There are many paths to the promised land," says Fraenkel of Khosla Ventures. "The truth is, the jury is still out on which one is going to work best."

For now, the money and the research effort are spreading across those paths simultaneously. Nvidia is building physics engines from scratch. General Intuition and Niantic are mining their own platforms. Worldmodeldata is betting that scale and variety, harvested from millions of players and millions of hours of gameplay, will matter more than physical fidelity—and that the market will need a neutral broker to supply it.

The startup's next test is whether AI labs will actually pay for nearly a million hours of licensed gameplay data, and whether models trained on it can do more than render convincing video. Until world models reach their ChatGPT moment, every proposed route to that milestone—including this one—remains on the table.