ExplorRobo
robot-learningphysical-aitactile-sensingvlarobot-data

The Robot Data Bottleneck Has Two Halves, and Only One Scales

By ExplorRobo Editorial · · 11 min read

Figures attributed to companies and papers are their published claims. The synthesis, comparisons and judgements below are this author’s analysis.

On 17 September, Figure published results for Helix 2.5, a humanoid policy pretrained on a dataset of human behaviour and evaluated in 30 Bay Area homes the robot had never entered. Figure reports 56 percent zero-shot success against 9 percent without that pretraining. The same day, four arXiv submissions argued the opposite direction: the properties that decide whether a task succeeds are often not in the image at all, but in contact, hidden mass and ground compliance (ForeTac-VLA, Shake to Learn, WM-VS, PIVOT). A day earlier, NVIDIA’s global head of Physical AI told TechCrunch that the field’s core limitation is that nobody has an internet-scale dataset for physical AI the way language models had one for text (TechCrunch). My reading: robot data is no longer one problem. It is two, with different collection economics, and only one scales the way language data did.

What Figure actually put a number on

The six-fold gap is the headline, but the experiment behind it is the more useful artifact. Figure trained two policies on identical task-specification data, one from random weights and one initialised from the Index-pretrained Helix 2.5 model, holding architecture, optimisation, hyperparameters, downstream data and evaluation fixed. Pretraining was the only variable. In blind evaluations, the from-scratch policy completed 9 percent of zero-shot trials and the Index-pretrained policy 56 percent, with no partial credit: a trial counted only if all 13 to 15 scattered toys were in the basket, all towels folded, or the whole bed made.

Figure’s other claims, attributed as such: one base model was adapted to three whole-body behaviours spanning locomotion, rigid and deformable manipulation and active perception; Helix 2.5 matched a Helix 02 policy trained in the evaluation environment while using half the adaptation data; and pretraining improved predictably, with four models trained on nested subsets spanning an 8x range and the largest run’s test loss forecast to four decimal places before training, an error Figure puts at 0.54 percent of the variation across the range. Index, Figure says, now generates roughly 35 minutes of new human experience per second.

Three caveats belong beside those numbers. The evaluation was run by Figure, so “blind” describes the graders, not the design, and there is no independent replication. The scaling law is measured on held-out next-action prediction loss with model size fixed, which makes it a data-scaling result rather than a measure of task success. And 56 percent means the best system in that post still failed 44 percent of graded trials.

What vision does not carry

The four submissions address different problems and do not cite one another in their abstracts, as far as those pages show. What they share is the claim that cameras are an incomplete state estimate.

ForeTac-VLA says it plainly: vision-language-action models rely on visual perception, which limits robustness in contact-rich environments where critical interaction states may not be visually observable, and existing tactile-enhanced VLA methods stay reactive rather than modelling how contact will evolve. Its fix is a forecasting module: tactile observations are fused with vision-language features through bidirectional cross-attention, and a transformer predicts multi-step future tactile states to condition action generation. Across four real contact-rich tasks the authors report 95 percent average success, 36.25 percentage points above a fine-tuned VLA.

Shake to Learn uses mechanics instead of learned touch. A sealed object reveals little about its centre of mass until it is lifted or shaken, so the authors use an origami-inspired soft arm as a physical reservoir computer: after grasping, a fixed shaking input is applied at the base and the ringdown response is recorded by camera or embedded sensors. Because the input is constant, hidden properties show up in the coupled robot-object dynamics, and a lightweight linear readout decodes the centre of mass before guiding a regrasp.

WM-VS attacks what its authors call the prediction-control mismatch: a model can predict a plausible next frame that still does not reduce task error. The world model is aligned to a four-dimensional servo coordinate derived from offline DINOv2 correspondences, after which a reactive joint-velocity policy is trained with consequence supervision and short imagined rollouts. On a real 7-DoF eye-to-hand system, corner RMSE fell to no more than 10 percent of its initial value in 30 of 30 trials and stayed there in 25 of 30; removing the future-error alignment cut retention to 26.67 percent. The authors say code and data will be released; I have not found the repository, so that is an intent, not an artifact.

PIVOT goes outdoors. Geometry-based traversability is fast but conservative, so the authors add vision-language semantic reasoning, grounded by measuring how well predicted energy cost, vibration and wheel slip correlate with real measurements and weighting each modality by that correlation. Geometry planning stays the nominal mode and semantic replanning runs only when geometry finds no path. Over five closed-loop trials on a route totalling about 6.4 km, the authors report autonomy rising from 59.6 to 97.0 percent and interventions falling from 11 to 3.

All four are lab or single-site results on specific tasks. Together they describe a ceiling that Figure is not buying down.

Why the two halves do not feed each other

Human video is passive, cheap at enormous scale, and rich in appearance, semantics and coarse motion. It carries no fingertip force, no slip event, no moment of inertia, no ground stiffness. Index is described by Figure as a dataset of human behaviour, and the behaviours it demonstrably transferred are those whose difficulty is picking the right object and reaching it in a new layout. Towel folding and bed making also involve deformable contact, and Figure’s numbers put zero-shot success there near half of trials. My analysis: that is the boundary. The semantic and spatial half of generalisation is being solved with passive data available at human scale; the physical half needs instrumented, active, task-specific data whose collection rate is measured in robot hours.

So the failure profile of a deployed generalist should shift from “it did not recognise the object” to “it could not feel what it was doing”. Two consequences follow. More video of environments has a lower marginal return than instrumenting a channel you do not currently sense. And contact data is expensive in a way camera data is not, because collecting it requires touching the object you are trying to manipulate.

NVIDIA’s framing is that the field is waiting for a ChatGPT moment and that startups are manufacturing the missing scale through simulation, synthetic data and multi-embodiment foundation models. I think that is half right. No single dataset unlocks everything, because the two halves have different physics: rigid-body contact and rendering are the mature parts of a simulator, while soft-body and cloth contact, which is what towel folding and bed making are, is where fidelity is hardest to establish.

Meanwhile, deployment is being bought with constraints

A third answer appeared the same week and does not depend on a foundation model. Agility announced Digit 5 on 15 September, and the announcement is mostly about the constraints that let a humanoid work beside people: human detection, visual and auditory intent cues, and an independent safety controller that oversees the response when someone is inside an unsafe distance (Agility). Agility is the first launch partner for NVIDIA’s Halos for Robotics safety stack and is contributing to the standards for the class, ANSI/A3 TR R15.108 and ISO 25785-1.

The hardware figures are Agility’s claims: repeated lifts up to 50 lb (22.7 kg), a 90-minute battery that charges in 9 minutes for a 10:1 run-to-charge ratio and, in the company’s words, more than 20 hours of productive work in a 24-hour day. The operational record cited is Digit 4: more than 65,000 hours of operation and, at GXO’s Flowery Branch facility, a 100,000-tote milestone at approximately 98 percent accuracy while on-task. None of that required solving general-purpose manipulation. It required a defined task family, a safety case, an independent controller, fleet telemetry and years of running the machine.

What this means if you are building robots

  • Instrument the state your task depends on. If a policy keeps failing on deformables, transparency, occlusion or compliance, more images of environments are unlikely to fix it. Add force or tactile sensing, and forecast contact rather than reacting to it.
  • Train on consequences, not only actions. WM-VS reports retaining its converged error level in 83.33 percent of trials, falling to 26.67 percent when future-error alignment is removed. Imitation alone does not supply that signal.
  • Keep a deterministic nominal mode. PIVOT runs geometry-based planning by default and calls the learned semantic path only when geometry fails, which keeps the expensive model off the critical path.
  • Write the evaluation rubric before you evaluate. Figure’s appendix is a usable template: a unique initial configuration per trial, one fixed checkpoint across all environments, no partial credit, per-object timeouts, and an automatic failure for any rollout needing a human safety intervention. Blinding the graders is the minimum bar, and it is not an independent evaluation. For the architecture background behind these policies, see Why Vision-Language-Action Models Matter for Robotics. For capturing the data those pipelines train on, recording and replaying ROS 2 bags with rosbag2 covers the tooling.

What to watch next

Independent replication of Helix 2.5 outside Figure, on environments and objects Figure did not choose, is the most informative thing that could happen next quarter. After that: whether anyone pools tactile data across labs the way Open X-Embodiment pooled teleoperation, since contact data is the half that does not scale on its own; whether the promised WM-VS code and data ship; whether touch sensing becomes a commodity gripper feature rather than a research instrument; and whether the new humanoid safety standards require data logging and versioning for learned policies, which would quietly make every deployment a data collection programme.

Sources

Frequently asked questions

What does zero-shot mean in Figure's Helix 2.5 results?
Figure states that zero-shot refers to the evaluation environments and manipulated objects: no data was collected in the 30 homes, and no evaluation toy, towel or bedding appeared in task-specification data. The three tasks were still specified through fine-tuning data collected elsewhere.
Why do vision-language-action models need tactile sensing?
Because contact states are not always visible. The ForeTac-VLA authors state that critical physical interaction states may not be visually observable, and report that their forecasting model reaches 95 percent average success across four real contact-rich tasks, 36.25 percentage points above a fine-tuned VLA baseline.

Related reading

The Robot Data Refinery Is the Real Product

Google, Physical Intelligence and Humxn all launched on 13 October. What they actually sell is the machinery that turns experience into training data.

Robot data, cobot software, local AI

Mecka AI raises $60M for robot training data, Universal Robots ships Gen 7 as a cobot lawsuit ends, and NVIDIA puts a petaflop of AI on Windows desks.

World-Action Models Agree on the Bottleneck, Not the Fix

A vendor launch and four arXiv papers attacked the robot data bottleneck in one week, and disagree about where a policy's generality comes from.