RAX agents, ABB vision, sonar SLAM

Yesterday’s roundup covered Intrinsic Core, the AWS toolchain and Robo-bench. Today: a framework that learns manipulation by writing code on the robot, ABB’s attempt to make vision commissioning a guided workflow, and 6-DoF SLAM on a sensor that works where cameras do not.

RAX learns tasks without demonstrations

A group across Harvard, NVIDIA, UC Berkeley, MIT and Stanford released RAX (Robot Agent eXploration), an open source framework in which a frozen coding agent drives a robot through its perception and control interfaces, explores until a task succeeds, and compiles what worked into a reusable program (arXiv, GitHub, project page).

There is no teleoperation corpus, no action primitives and no value function. The agent gets a language instruction, a few goal images at most, and a budget of episodes. It writes Python that closes the perception, planning and control loop, keeps notes and verified skills in a workspace that persists across episodes, and reuses them on longer tasks. The model weights never change, so what improves is the code, not a policy.

In simulation, across single arm, bimanual, dexterous, mobile and humanoid embodiments, RAX beats the strongest baselines by 37.4 percent on average on unseen environments. On real hardware, the project lists a 72 minute session on an air fryer that compiled into an 18 second program, and a 17 minute session on a stacked tower task that compiled into 10 seconds. The skill library starts with the Rainbow RB-Y1 and the Franka FR3.

The failure handling is the interesting part. When depth was inaccurate the agent programmed contact probing with the end effector; after hitting a cabinet it added a clearance check before every base rotation, and it asked the operator to move objects so it could test the program in new layouts. That is closer to how a systems engineer debugs a cell than to how a policy is trained.

Two caveats: the headline is relative gain, not absolute success, with tasks chosen by the authors, and the robot must expose depth, force and a usable API. That is the modular stack argument again, since agents can only write control code against interfaces that exist.

ABB puts a vision trainer on the controller

ABB Robotics extended its OmniCore EyeMotion vision product with AI Robot Trainer, which connects image capture, model training, deployment and later retuning in one guided workflow (Assembly Magazine). ABB’s figures are commissioning time cut by up to 90 percent and cycle times cut by up to 50 percent. Those are vendor claims, published without an independent test or a description of the baseline system.

The tool targets people who are not machine learning engineers: capture images, train, deploy, retune when lighting or the product changes, with a chatbot guiding the steps. It mixes synthetic and real images, accepts third party cameras, and hands path generation to EyeMotion’s collision-free planning. ABB lists 2D pick and place through 3D bin picking, inspection, sorting and assembly.

Vision is the bridge between perception and action, as ABB Robotics president Marc Segura puts it, but the integration cost has always sat in lighting, lens choice, labelled defects, and the retraining that follows every product change.

Two questions worth asking before believing a 90 percent figure: does training run on the controller, on an edge server or in ABB’s cloud, and can the model be exported and rolled back? How many images per class does a bin picking setup need? Until those answers exist, this is a packaging step for industrial vision, not a new capability.

Sonar SLAM drops the flat floor assumption

RTISSLAM-3D, from Jan Steckel, Vincent Katter and Wouter Jansen, is a full 6-DoF SLAM system for a wheeled robot carrying an in-air 3D imaging sonar, an IMU and wheel encoders (arXiv). Previous in-air sonar SLAM assumed a plane: it estimated a 2D trajectory and discarded elevation. That assumption dies on a forest floor.

Each pulse from an eRTIS sensor, 32 microphones beamformed into 2,293 directions over the frontal hemisphere, becomes a sparse set of 3D detections after a CFAR detector on the sphere, with ground multipath removed. Those detections feed acoustic odometry with beam-shaped uncertainty, keyframes every metre enter a 6-DoF pose graph with IMU attitude factors, and loop closures come from a polar submap descriptor verified by submap registration. No learned components anywhere.

On a 545 metre forest drive in Trondheim, with 3D lidar SLAM as reference, loop closure cut horizontal error from 2.40 metres for wheel and gyro odometry, and 1.74 metres for acoustic odometry, to 0.28 metres. Height error was 0.12 metres, 233 loops closed with none off by more than 0.5 metres, and 94.6 percent of occupied voxels landed within 0.3 metres of the lidar map.

A sensor with a few dozen detections per capture and a few metres of range sounds weak, but ultrasound keeps working in dust, smoke, fog, darkness and next to glass, where cameras and lidar degrade, and the hardware is cheap. Inspection and agricultural platforms have a real option here. The authors say the pipeline will be open sourced.

Tesla faces a school zone petition

NHTSA has begun reviewing a defect petition alleging that roughly 1,000 Tesla Model Y Robotaxis and Cybercabs exceed posted speed limits in active school zones (ExtremeTech). The petition was filed in late September, contains three video links, and is tied to four incidents: one complaint to the agency and three social media reports. It does not allege a crash, injury or death.

A petition is not a finding. The agency has 120 days to decide whether to open a formal investigation, and only an investigation can lead to a recall. NHTSA has examined Tesla’s Full Self-Driving system over traffic violations before, and is separately reviewing certification data for the Cybercab, which has no steering wheel or pedals. Waymo has faced inquiries over stopped school buses.

For people building autonomy stacks, the technical question underneath is temporal map validity. A school zone limit is not a static attribute: it depends on time of day, the school calendar and sometimes a flashing beacon. If that layer is baked into an HD map and refreshed monthly, the planner is wrong every afternoon at pick-up time. Watch whether the petition forces disclosure of how these fleets ingest time-dependent speed limits.

42 Gazebo worlds, generated from scripts

A community project published 42 Apache 2.0 simulation worlds for navigation, exploration, LiDAR and SLAM testing, covering mazes, offices, warehouses, forests, real world floor plans, moving obstacle scenes and challenge courses (ROS Discourse, GitHub).

The method matters more than the count. Gazebo world authoring is tedious, so the author generates layouts in Python, builds geometry in Blender and converts to SDF. Worlds become reproducible and diffable. The scenes are 2D plan based with simple objects, useful for Nav2 and SLAM regression tests rather than perception training.

Quick Bites

• ROS News for the week of October 12 mentions a stripped down RViz that runs in a terminal over SSH, DendROS for colored ROS 2 output, and robodev.tools, a searchable index of a few thousand ROS tools on GitHub (ROS Discourse).
• The MoveIt community meeting on Tuesday, October 20 at 3 PM covers MoveIt 2 Lyrical release preparation, whether the Jazzy release ships now or after backports, an automated backporting pipeline, and official Robotiq drivers (ROS Discourse).
• A BLDC motor selection guide starts from the mechanical load: continuous and peak torque, reflected inertia, regenerative energy, thermal duty cycle and encoder placement, with the reminder that BLDC and PMSM are often used interchangeably for the same three phase machine (The Robot Report).
• IEEE Spectrum’s Video Friday includes a DeepMind demonstration in which Gemini 4 Argon watched a human play cornhole, built a MuJoCo simulation from the video, and transferred the result to an Apptronik Apollo 2 (IEEE Spectrum).