ExplorRobo
robot-dataphysical-airobot-learningdata-labelingsimulationfine-tuningrobot-foundation-modelsbenchmark-evaluation

The Robot Data Refinery Is the Real Product

By ExplorRobo Editorial · · 11 min read

Figures attributed to companies and papers are their published claims. The comparisons and judgements below are this author’s analysis. I read the announcements and the benchmark methodology; I have not run any of the systems.

On 13 October, three announcements described what look like three different businesses. Google published Gemini 4 Argon for Robotics, a general model it says scores 87 percent on an embodied reasoning benchmark. Physical Intelligence opened a beta API that fine-tunes its robot foundation models on customer data and serves inference from its own infrastructure. HUMXN Research Labs said it would spend 50 million dollars buying service records and wearable job footage from skilled trades and manufacturing.

The same day, Vals AI updated Robo-bench, which asks a different question: can a general model drive a simulated robot arm and humanoid from motor commands and camera images alone, with the physics running in real time? Gemini 4 Argon scored 0.83 percent. The best model, GPT-6.1 Sol, scored 28.33 percent, and no model averaged more than 2 points out of 100 on the humanoid suite.

Both numbers can be true. The 87 percent measures a model reasoning about a physical scene; the 0.83 percent measures one closing a control loop under a clock. Between them sits the layer this week’s launches actually sell: machinery that turns recorded experience into trainable data, then checks whether the result is right. Call it the data refinery, and it is where the technical difficulty and the business value have moved.

What the three launches actually ship

Google’s post is not a policy release. It is a tour of prompts. Argon rebuilds a scene from a teleoperation episode and tunes it until rendered cameras match the real views and simulated fingertip contact forces match the logged tactile readings. It converts an egocentric human video into a 500 Hz MuJoCo trajectory for a 76 degree of freedom humanoid with two 22 degree of freedom hands. It labels a 136 second bimanual folding video into an episode goal, six subtasks, 17 contact-level primitives, and a quality control pass that downweights fumbles by 0.15 to 0.25 for behaviour cloning. And it generates more than 4,500 domain-randomized episodes from a single overhead photo, which Google says lifted real robot success from 56 to 75 percent and cut completion time from 35 to 24 seconds.

Every one of those is a data operation. The model writes the simulator, the retargeting code, the labels and the acceptance checks. In the one demo that looks like direct control, Argon works through a visual coordinate harness and primitives such as grasp, goto and place_until_contact, checking a wrist camera after every turn. It writes the loop. It does not become the loop.

Physical Intelligence’s API is the same layer with a different commercial shape. You upload data, launch fine-tuning on Pi’s compute, and call a hosted model for inference. Pi describes the loop in its own words: collect demonstrations, train a model, test it on the robot, and use the failures to decide what data to collect next. Almond, an early partner, deburrs spacers for its own dual-arm robot with a model it obtained in four days. The product is not a fixed policy. It is a shorter path around that loop.

HUMXN is the feedstock end. It buys service records, repair histories, job-site photos and training videos, and pays about 20 dollars per hour for approved wearable footage. Its public site describes a marketplace for expert human data with consent records, fingerprints and provenance manifests. Tom’s Hardware reports that the company will subsidize plumbing, electrical, HVAC, duct and carpet cleaning in Minneapolis, Chicago and Miami, so that homeowners get the work free and the company records it.

The supply chain is visible. HUMXN buys experience. Argon and Pi convert it. Robo-bench checks whether the conversion produced something that can act.

The benchmark is an acceptance test, not a contradiction

Robo-bench runs 24 tasks, 13 on a 7 degree of freedom Franka Panda arm and 11 on a 29 degree of freedom Unitree G1 humanoid with Dex3 hands. There is no motion planner and no walking controller. The agent gets the datasheet, joint limits, sensor readings and camera images, never the simulator. Episodes last three minutes of wall clock time, and the physics keeps running while the model thinks. Balance needs a correction about once a second, faster than a model turn, so a model that succeeds has to write code that closes the loop itself. It runs one seed per task, with repeated runs moving a total by up to ten points, so only large gaps count, and 27 points is large.

The test is narrow: not whether a model can plan a week of work or label a dataset, but whether it can produce a controller that survives contact with a clock. ERQA measures reasoning about a scene, not control, so the two numbers describe different layers.

In practice, a frontier model for robotics currently means a model that writes and supervises code above a control stack. That is a job description, not a criticism, and explains why the launches look like data tools. If the output is a simulator, a label set or a retargeting script, its quality depends on the acceptance test attached to it. Argon’s Real2Sim prompt is explicit: rendered views must line up with real cameras, and fingertip contact forces must match logged tactile data. That is what makes a synthetic episode worth training on, or not.

What an hour of experience costs

Brian Potter’s survey of robot training data, published on 15 October, maps the feedstock. Open X-Embodiment assembled about 4,000 hours of robot trajectories. Generalist AI’s GEN 1 used around 500,000 hours of UMI data.

The rates are just as useful. Eidon AI quoted 60 to 100 dollars per hour for robot data. Pantheon quoted 60 to 150 dollars per hour for UMI data, and said a pipeline it built could produce it for 10 dollars per hour. Shift pays up to 20 dollars per hour for egocentric video. Potter estimates the field might need about a billion hours, which at Eidon’s rates alone is 60 to 100 billion dollars, though his own range is 10 to 100 billion because cheaper pipelines exist.

Those figures are not a clean margin. They compare different products: instrumented robot trajectories with action labels and proprioception, UMI data with a gripper and cameras, and passive human video with neither. The spread is the price of instrumentation and accepted quality, not just processing. A recorded hour is cheap. An hour that a policy can learn from is not, and the difference is the work of turning one into the other.

Where a refinery fails quietly

The weak point in every one of these pipelines is the acceptance test, because a bad one produces data that looks clean and teaches the wrong thing.

Argon’s quality control step downweights fumbles by 0.15 to 0.25, which is sensible for behaviour cloning: you do not want a policy to imitate a slip. But a fumble is also where recovery behaviour lives, so a policy trained mostly on clean progress learns the nominal path and gets little signal for getting back. A cell that stops and calls a human wants clean data, while a machine that must finish the job needs recovery segments kept. The pipeline makes one choice and exposes it as a weight, and developers should know which choice they inherit.

The second failure is a simulator that matches pixels but not physics. Argon’s prompt checks both, which is right, and contact force is the part that matters for manipulation. But the check runs against a logged episode, and the policy is still evaluated on the vendor’s own benchmark. Google’s 56 to 75 percent is a company result on its own evaluation, without independent replication. It shows the pipeline can work, not how often it does.

Consent is the third unverified link. HUMXN says ownership is checked and data de-identified before purchase, but the announcement does not describe how consent is recorded at capture, through the technician, the capture vendor and the broker.

If you are buying or building this

  • Write the acceptance test before you buy the data. Define the task, the success condition and the held-out set first. Robo-bench is a good template: wall clock, no simulator access, no planner, one seed and a reported noise band.
  • Price per successful policy, not per hour. A cheap hour that produces no usable episode costs more than an instrumented one. Ask for the acceptance rate and the failure taxonomy, not the total hours.
  • Ask what happens to your demonstrations. Pi’s beta post describes uploading data, fine-tuning and hosted inference, but it does not state retention terms or whether customer data improves the base models. If your demonstrations are your advantage, that is a commercial decision before it is a technical one.
  • Own the control layer and the sensing. A model that writes code above a controller fits the proposer and gate split in The Model Proposes, the Gate Disposes. A model that is the controller is a different risk class. And if the job needs contact, video alone will not carry it, as argued in the two halves of the robot data bottleneck.

What to watch

Whether Pi publishes outcomes from customers beyond a four-day iteration, and whether the API grows an on-device runtime. Pi says most of the applications shown ran on cloud-hosted inference and that network connectivity was one of the problems early deployments highlighted, which puts a wide area network in the policy path.

Whether Google’s simulation gains replicate outside its own evaluation, and whether its labels survive an outside audit. Whether HUMXN’s records become action-labelled data or remain semantic priors for planning and diagnostics. And whether any vendor starts publishing cost per successful policy. Until then, the honest summary is the one the benchmark gives: the models are getting better at describing and preparing work, and the robots are still waiting for a general controller that can close the loop on its own.

Sources

Related reading

World-Action Models Agree on the Bottleneck, Not the Fix

A vendor launch and four arXiv papers attacked the robot data bottleneck in one week, and disagree about where a policy's generality comes from.

Dream-1, Alpamayo and robot data engines

Dream Labs opens a zero-shot robot API, Bolt picks NVIDIA Alpamayo for 25,000 Lucids, NVIDIA opens the GR00T data engine, and RoboStack merges four ROS repos.

Intrinsic Core, AWS stack, Atlas hands

Intrinsic open sources its industrial robot stack, AWS packages an open Physical AI pipeline, and a new benchmark shows frontier models failing at control.