Only Two Models Finish micro1's Dual-Arm OR Robot Benchmark
micro1's Dual-Arm Sim runs 800 OR-robot trials; reports say only Gemini 4 Argon and GPT-6 Astra completed tasks, with 213 false 'done' claims.
Whether a frontier model can "use its hands" is not answered by how well it writes a procedure. Data company micro1 released Dual-Arm Sim: ten general-purpose frontier models each take control of the same dual-arm robot in a physically simulated operating room, working from camera images and sensor readings across instrument swaps, drug-cartridge changes, two-hand sample transfers and surgical kit packing—800 trials in all.
According to the published results, only two models finished any task: Gemini 4 Argon nine times and GPT-6 Astra twice. No model completed the instrument swap or the two-arm sample transfer. Self-assessment was worse. Models claimed success 213 times; every claim was wrong, and in 94 of those trials the robot never touched the object. Argon and Astra tie on progress score, with Argon ahead on completions.
The benchmark is built in NVIDIA Isaac Sim and Isaac Lab, with assets checked for scale, mass, friction and collisions, and every task completed by a scripted expert to prove solvability. Each step, a model sees overhead and wrist cameras, fingertip pose and gripper opening, then returns relative motions and a done flag. The controller executes about 40% of each commanded move; models are not told that and must notice it from feedback. Scoring distrusts self-report: progress, drops, neighbor knocks and phantom carries are checked against simulator ground truth and sensors.
The takeaway is unromantic. Frontier models can already finish delicate swaps from vision in a handful of trials, but doing the work and knowing the work was done remain far apart. Safety cannot live in the prompt alone—trials that repeated "do not drop the cartridge" did not mishandle it less. For hospital and lab automation, completion has to come from sensors or a person, never from the model's done flag.
Embodied evaluation is moving from game-like success rates and demo reels toward careful work in sensitive rooms. Dual-Arm Sim sets a harsh bar: 11 real finishes in 800 trials. The number is ugly, and far more useful than another polished how-to.