Scale AI and Hugging Face Open FineART: 533K Robot Subtask Labels

Scale AI has spent years selling data for language models. Now it is open-sourcing robot data for the first time. Together with Hugging Face's LeRobot team, Scale has released FineART, billed as the largest open robot dataset with subtask labels, along with an open vision-language-action model, FineART-VLA.

The dataset is substantial: 40,543 teleoperated episodes totaling 1,718 hours across 151 tasks, all recorded on a Trossen ALOHA-style bimanual robot. The real draw is the labeling. The data is broken into 533,913 timestamped subtasks with natural-language captions, about 13 steps per episode.

One instruction isn't enough; robots need steps

Most robot datasets give an entire episode a single instruction, such as "fold the shirt." Real tasks are long chains: grab a corner, fold it over, smooth it out. Without supervision on the steps in between, models struggle with long-horizon, multi-step manipulation. Existing bimanual datasets with subtask labels usually cover only part of their hours. FineART covers 100%.

The mix leans toward hard work: 77% is pick-and-place and sorting, and the remaining 23% is contact-rich long-tail work such as textiles, cables, tool use and precision insertion. Labels were written by people without vision-language model help, and separate teams handled collection, annotation and quality review. That is exactly the kind of operation Scale is built for.

FineART-VLA: plan, then act

FineART-VLA builds on Physical Intelligence's π0.5, with a 2-billion-parameter Gemma backbone, a SigLIP vision encoder and a 300-million-parameter action expert. It puts System 2 language planning and System 1 continuous control in one policy: first it says in words what the next step should be, then it produces actions for that step. It is the first open System 1–System 2 VLA in the LeRobot ecosystem.

The paper's numbers make the case for subtask labels. On a spatial-instruction task, mid-training on FineART lifted success from 32% to 100%. On unseen long-horizon tasks with step-by-step human guidance, success rose from 16% to 76%. On a new robot, the YAM, it matched baseline performance with one-tenth of the fine-tuning data.

Scale's new story

For Scale, this release works as a showcase. As labeling demand for language models plateaus, embodied AI is widely seen as the next data gold mine, and robot data is far more expensive and harder to scale than text. Giving away a high-quality dataset shows every robotics company that Scale can do this grind well and at scale. The blog's line inviting teams to get in touch for larger, higher-quality annotation says as much.

For the open community, a dataset released with model weights, training code and annotation tooling is a real public good. In robotics, the scarce resource has never been model architecture. It has been data.