Some artificial intelligence industry experts believe large language models face limits because they cannot navigate the physical world. LLMs rely entirely on text training. Researchers like Fei-Fei Li and Yann LeCun focus on world models to teach machines real world physics. These world models require both visual data and action data to master tasks.
Finding enough training material remains a major hurdle for developers. Internet data lacks the cause and consequence required for world models. Labs previously tried attaching sensors to humans and robots to manually generate data. This manual method produces limited data and misses unexpected fringe scenarios.
A startup packages video game data for AI
Worldmodeldata is a British startup advised by Yann LeCun that addresses this data shortage. The company packages controller inputs and data from video game studios into training datasets. Rhea Loucas, CEO of Worldmodeldata, states that video games offer vast and diverse experiences to teach artificial intelligence.
Worldmodeldata says it has licensed nearly 1 million hours of data from popular video game studios. The startup eventually plans to create ways to compensate individual players for their data. Companies like General Intuition and Niantic also collect video game data from their own platforms.
Some experts question the value of game data
Not everyone agrees that video game data is useful for training physical AI. Nvidia world model development lead Ming-Yu Liu warns that game physics often relies on shortcuts. Liu argues that video games lack the fine-grained motor control details needed for tasks like object manipulation. He suggests video game data works better for generating hyperrealistic video or 3D environments.
Xiatian Zhu, an AI associate professor at the University of Surrey, shares similar concerns about game physics. Zhu notes that video games act as coarse and approximate simulators. Nicole Fraenkel, a partner at Khosla Ventures, acknowledges that the jury remains out on the best path forward.
Worldmodeldata plans to create avenues for individual players to be compensated for their gaming data in the future.
