World models hallucinate successful robot grasps

When a robot misses a grasp, a world model simulating the run needs to show an empty gripper pulling away. In one test case documented by the WorldOracle benchmark, a physical arm reached for a piece of fruit and failed to make contact, but the model generated a video showing the gripper lifting the fruit into the air anyway. Evaluating models by feeding them real initial states and action sequences, a joint team from Princeton, UT Austin, Toronto, NVIDIA, and AgiBot found that seven leading world models routinely make this category of error. They track the kinematic movement of the arm while losing the state of whatever the arm touches. On the AgiBot G2 teleoperation dataset, NVIDIA Cosmos 3 scored 89.4 on arm spatial alignment, but dropped to 45.4 on the manipulated object.

Because each physical interaction sets the initial conditions for the next, small contact errors turn into severe drift during multi-step tasks like hanging towels or pouring water. Across the first thirty seconds of an episode, Cosmos 3 scored 53.99 on single-view fidelity, but its score fell to 26.60 over complete sequences. DreamDojo showed a similar drop, falling from 50.56 to 27.08. Only GE-Sim 2.0 limited the decay, declining from 73.86 to 63.33.

Generative models trained on demonstration datasets carry a strong statistical prior toward successful task completion. When a gripper closes near an object, bias in the training data suggests the object should move with it, regardless of whether or not contact was properly made with the object. If world models are going to function as reliable simulators for policy evaluation or robot trajectory planning, a missed pick has to register as a drop, not a successful grab.