What is NVIDIA Isaac Sim?

NVIDIA Isaac Sim sits between robot software and the real world: an open source application where you import a robot, give it physics and sensors, and run it thousands of times in software before it moves anywhere physical. The product page calls it “an open source reference framework built on NVIDIA Omniverse libraries for robotics simulation, testing, and synthetic data generation” (NVIDIA Developer).

What Isaac Sim actually is

The GitHub repository is blunter: “NVIDIA Isaac Sim is a simulation platform built on NVIDIA Omniverse, designed to develop, test, train, and deploy AI-powered robots in realistic virtual environments” (GitHub: isaac-sim/IsaacSim). Three structural facts matter most.

It is USD-native. Omniverse and Isaac Sim both use Universal Scene Description (OpenUSD) as the scene format, assets must be converted to it before use, and the default unit is meters (Reference Architecture).

It is assembled from extensions, which the reference architecture calls “core building blocks that interact with and add or extend the functionality of Isaac Sim.” That is why the same platform installs as an app, a container, or a set of pip packages.

The licence covers the code, not everything you ship. The repository is Apache 2.0, but its LICENSE file states that building or using it “requires additional components licenced under other terms,” including the Omniverse Kit SDK and 3D assets.

Omniverse, PhysX and Newton

Isaac Sim sits on the Omniverse Kit SDK, version 110.3.0 in the current 6.1.0 GA release (Release Notes). Rendering comes from the Omniverse RTX Renderer, which also underpins sensor realism: cameras in Isaac Sim “are USD Camera prims rendered by the RTX renderer” (Camera Sensors).

Physics supports two backends, and the documentation is explicit about their status: “In addition to the default PhysX SDK backend, you can now use Newton as the simulation backend” (Newton Physics). PhysX remains the default, and Isaac Lab credits it for its “fast and accurate physics simulation” (Isaac Lab).

The mechanics are straightforward. Omniverse Physics parses the USD Physics schema of your robot and environment, creates matching objects in the selected backend, advances them each timestep given inputs such as control-policy torques, and writes the state back to USD, where a policy or the renderer can consume it (Physics). That round trip lets one asset move between the GUI and a training loop.

Newton is the interesting part, and the one to treat cautiously. NVIDIA’s docs describe it as “a GPU-accelerated, extensible, and differentiable physics simulation engine designed for robotics and research,” built on NVIDIA Warp and integrating MuJoCo Warp, with solvers including XPBD, MuJoCo, Featherstone and SemiImplicit. The same page calls it “an open-source project maintained by Disney Research, Google DeepMind, and NVIDIA,” while the product page says it is co-developed by Google DeepMind and Disney Research and managed by the Linux Foundation. The two NVIDIA sources disagree on governance, so treat those details as moving. The status, though, is unambiguous: “Newton integration in Isaac Sim is experimental,” and robots with closed kinematic chains, such as parallel linkages, do not import.

What GPU acceleration actually buys you

Parallel scenes for learning. Isaac Lab’s published benchmark trains an Isaac-Humanoid-v0 policy for 65.5 million steps (4096 environments x 32 rollout steps x 500 iterations) on a single RTX 4090, reporting 198 to 201 seconds for RL-Games, SKRL and RSL RL and 287 seconds for Stable-Baselines3 (RL Library Comparison). That is what GPU acceleration means in practice: thousands of copies of one scene stepping in parallel.

Synthetic perception data. Replicator randomizes attributes such as lighting, reflection, colour and position, and its annotators cover RGB, bounding boxes, instance segmentation and semantic segmentation, with export to COCO and KITTI (Synthetic Data Generation).

Ray-traced sensors. RTX sensors “use the Omniverse RTX Renderer’s RTX Sensor SDK to sense the environment, enabling interaction with materials in visual and non-visual spectra,” so an RTX Lidar can model returns from transparent or reflective surfaces (RTX Sensors). RTX Lidar ships with vendor configurations including Ouster, SICK and HESAI, and can publish sensor_msgs/PointCloud2 and sensor_msgs/LaserScan over ROS 2 (RTX Lidar).

The Python API and the ROS 2 bridge

Isaac Sim is scripted from Python in two modes: “Standalone Python scripts are executed from the command line and are used to automate tasks or run simulations. Interactive Python scripts are executed in the Python console and are used to explore the NVIDIA Isaac Sim API and test code snippets” (Standalone vs Interactive Python). Binary and container installs expose python.sh; the pip route installs packages such as isaacsim and isaacsim-core into a Python 3.12 environment.

The API surface is mid-migration, worth knowing before you write code against it. The Core API is described as wrappers over raw USD and physics APIs, and the docs state that “Isaac Sim 5.0.0 has introduced the Core Experimental API,” which “will become the base API used in all Isaac Sim source code,” while the current Core API “will be deprecated and removed in future releases” (Core API Overview).

For ROS 2, the bridge is a normal extension that you enable at launch by setting the startup argument ros_bridge_extension to isaacsim.ros2.bridge. Isaac Sim is “compatible with ROS 2 Humble and ROS 2 Jazzy,” with Jazzy on Ubuntu 24.04 recommended. Two details trip people up: Humble on Ubuntu 24.04 is not officially supported, and custom ROS interfaces on Ubuntu 22.04 must be built with a Python 3.12 workspace, because Isaac Sim loads them in its own 3.12 runtime (ROS 2 Installation).

Where Isaac Lab fits

Isaac Lab is a separate open source framework built on Isaac Sim, described in its docs as “a unified and modular framework for robot learning that aims to simplify common workflows in robotics research (such as reinforcement learning, learning from demonstrations, and motion planning).” It lists PhysX physics, tiled rendering APIs for vectorized rendering, domain randomization and cloud support as key features, ships environments across classic control, manipulation, locomotion and navigation, and is BSD-3-Clause with some parts under Apache-2.0 (Isaac Lab).

For reinforcement learning it documents training scripts for four libraries, RL-Games, SKRL, RSL RL and Stable-Baselines3, plus multi-GPU and multi-node training (Reinforcement Learning). For imitation learning it covers teleoperation with Isaac Lab Mimic, augmented imitation learning, and SkillGen for automated demonstration generation (Imitation Learning). Isaac Sim itself scales across multiple GPUs, and NVIDIA offers container images and a hosted cloud option.

Where it fits next to Gazebo and MuJoCo

Gazebo’s documentation introduces Gazebo Sim as “a 3D robotics simulator” launched with gz sim, and the current Jetty release is an LTS supported from September 2025 to May 2031 (Gazebo). MuJoCo describes itself as “a general purpose physics engine” for robotics and machine learning research, built as a C/C++ library with MJCF as its native model format and open sourced by Google DeepMind in May 2022 (MuJoCo). Isaac Sim brings USD as the scene format, RTX ray tracing for sensors and synthetic data, and a hard requirement for an RT-core GPU.

My reading, not NVIDIA’s: pick Gazebo when your worlds are SDF-first and ROS-centric, pick MuJoCo when the research question is dynamics or control and you want a small fast engine, and pick Isaac Sim when the bottleneck is perception data, sensor realism, or how many environments you can train on at once. The boundary is softening, because Newton embeds MuJoCo Warp based solvers inside Isaac Sim, although that path is experimental today.

What you need to run it

The published requirements decide whether Isaac Sim fits your machine. For x86_64, the minimum is Ubuntu 22.04/24.04 or Windows 11, an Intel Core i7 (7th generation) or AMD Ryzen 5 with 4 cores, 32 GB of RAM, 50 GB of SSD storage, and a GeForce RTX 4080 with 16 GB of VRAM. NVIDIA notes that “Isaac Lab usage will require additional RAM and VRAM for training,” and that GPUs with less than 16 GB of VRAM may be insufficient for scenes rendering more than 16 megapixels per frame (Isaac Sim Requirements).

The same page also states that GPUs without RT cores, specifically the A100 and H100, are unsupported, that the container is Linux only, that aarch64 builds currently target only the NVIDIA DGX Spark, and that online assets need an internet connection unless you download the asset packs.

What it means for robotics developers

1. Your asset pipeline becomes USD-first. URDF and MJCF importers and a CAD converter target conversion into OpenUSD, so your robot description starts to look less like a URDF and more like a composed USD stage.
2. Simulation becomes a Python program. Standalone scripts let regression tests, CI jobs and batch data generation call the simulator directly instead of driving a GUI.
3. ROS 2 stays the seam, but the supported matrix is narrow: Jazzy on Ubuntu 24.04 pairs with Isaac Sim, and Humble lives on Ubuntu 22.04.
4. Version churn is a real cost. Extensions are deprecated quickly, for example the older camera and RTX sensor extensions since 6.0 in favour of isaacsim.sensors.experimental.rtx, and the Core API is on a path to removal. Pin your version and read the migration guides.

What comes next

Isaac Sim 6.1.0 GA updated the experimental Newton integration to Newton 1.5.0 and added support for the Vertex Block Descent and XPBD solvers, hydroelastic contacts, and UI workflows for authoring robots with MuJoCo and Newton schemas (Release Notes). Alongside that, NVIDIA positions generative world foundation models as a data augmentation layer, neural reconstruction with NuRec to turn real sensor captures into interactive scenes, and teleoperation tooling for collecting demonstrations (NVIDIA Developer). The thing to watch is not a version number but which of those pieces stop being labelled experimental.

Sources

• NVIDIA Developer: Isaac Sim
• GitHub: isaac-sim/IsaacSim
• Isaac Sim Documentation: What Is Isaac Sim?
• Isaac Sim Documentation: Release Notes
• Isaac Sim Documentation: Reference Architecture and Task Groupings
• Isaac Sim Documentation: Physics
• Isaac Sim Documentation: Newton Physics Backend
• Isaac Sim Documentation: RTX Sensors
• Isaac Sim Documentation: Camera Sensors
• Isaac Sim Documentation: RTX Lidar Sensor
• Isaac Sim Documentation: Synthetic Data Generation
• Isaac Sim Documentation: Perception Data Generation (Replicator)
• Isaac Sim Documentation: Standalone vs Interactive Python
• Isaac Sim Documentation: Core API Overview
• Isaac Sim Documentation: Python Environment Installation
• Isaac Sim Documentation: ROS 2 Installation
• Isaac Sim Documentation: Isaac Sim Requirements
• Isaac Lab Documentation: Welcome to Isaac Lab
• Isaac Lab Documentation: Reinforcement Learning
• Isaac Lab Documentation: Reinforcement Learning Library Comparison
• Isaac Lab Documentation: Imitation Learning
• Gazebo Documentation: Getting Started
• MuJoCo Documentation: Overview