Interop on isaac_sim_5p1 by cvolkcvolk · Pull Request #465 · isaac-sim/IsaacLab-Arena

cvolkcvolk · 2026-03-04T14:52:34Z

Summary

Enables Isaac Lab's train.py and play.py scripts to run Arena environments directly via an external callback, without requiring Arena-specific training scripts.

Original PR: Isaac Lab Interop. #413

Changes

Adds environment_registration_callback(), an Isaac Lab external callback that parses CLI args, builds, and registers an Arena environment before Isaac Lab's train/play script runs.
Adds RLPolicyCfg, a @configclass subclassing RslRlOnPolicyRunnerCfg with Arena defaults (PPO, 4000 iterations, obs groups). Registered as rsl_rl_cfg_entry_point in the gym registry so Isaac Lab's @hydra_task_config can load it.
Drop the need for agent_cfg_path / JSON agent configs.
Agent config is now auto-loaded from params/agent.yaml saved alongside the checkpoint by Isaac Lab's train.py — no separate config file required.
Coerces viewer lookat/eye tuples to plain Python floats (not np.float64) to avoid Hydra/OmegaConf serialization errors. Follow up planned on that.

Deleted

isaaclab_arena_examples/policy/rl_policy/base_rsl_rl_policy.py — replaced by new location
isaaclab_arena_examples/policy/rl_policy/generic_policy.json — no longer needed
isaaclab_arena/scripts/reinforcement_learning/cli_args.py, play.py, train.py — replaced by Isaac Lab's own scripts + the callback

Docs updated

docs/README.md and RSL-RL tutorial .rst files updated new interop training/eval workflow.

Usage

Train:

/isaac-sim/python.sh submodules/IsaacLab/scripts/reinforcement_learning/rsl_rl/train.py \
  --external_callback isaaclab_arena.environments.isaaclab_interop.environment_registration_callback \
  --task lift_object --num_envs 512 \
  agent.max_iterations=4000 agent.policy.activation=relu

Evaluate (checkpoint path is the only required argument — agent config is loaded automatically from params/agent.yaml in the same directory):

/isaac-sim/python.sh isaaclab_arena/evaluation/policy_runner.py \
  --policy_type rsl_rl \
  --num_episodes 1 \
  --checkpoint_path logs/rsl_rl/generic_experiment/<timestamp>/model_0.pt \
  lift_object --rl_training_mode False

Signed-off-by: Clemens Volk <cvolk@nvidia.com>

Cherry-picks the fix that defers isaaclab.utils imports until after AppLauncher starts, required because the recent 'Remove mesh import *' commit added pxr (OpenUSD) as a transitive import in utils/__init__.py. Signed-off-by: Clemens Volk <cvolk@nvidia.com>

Remove the separate JSON agent config file. RslRlActionPolicy now auto-detects params/agent.yaml saved alongside the checkpoint by IsaacLab's train.py, making the checkpoint the single source of truth. Signed-off-by: Clemens Volk <cvolk@nvidia.com>

- Delete rigid_object_variant.py (prototype scratch file) - Move base_rsl_rl_policy.py to isaaclab_arena_examples/policy/ - Update lift_object_environment.py import for new module path - Replace WIP comment in cameras.py with clean TODO(cvolk) - Add TODO(cvolk) to RL workflow docs for follow-up rewrite Signed-off-by: Clemens Volk <cvolk@nvidia.com>

Replace the outdated Docker-based instructions with the correct host workflow using a Python 3.11 venv. Signed-off-by: Clemens Volk <cvolk@nvidia.com>

…back - Replace Arena's removed train.py with IsaacLab's train.py + --external_callback - Add explanation of how the callback registers the environment before training - Add Hydra override examples for hyperparameter tuning - Update tensorboard command to use /isaac-sim/python.sh -m tensorboard.main - Rewrite evaluation section: drop removed play.py method, update commands to remove --agent_cfg_path (checkpoint now auto-loads params/agent.yaml) - Update step 1 validation command to use IsaacLab train.py Signed-off-by: Clemens Volk <cvolk@nvidia.com>

Signed-off-by: Clemens Volk <cvolk@nvidia.com>

AppLauncher's enable_pinocchio path wraps _start_app() with a patch that calls from pxr import Gf immediately after startup. If Isaac Sim's extension loading is incomplete (e.g. due to a version constraint in the experience file), pxr is never added to sys.path and the patch crashes with ModuleNotFoundError. Setting disable_pinocchio_patch=True tells AppLauncher to skip the patch. Pinocchio is already imported before AppLauncher is constructed, which is sufficient for it to work correctly. Signed-off-by: Clemens Volk <cvolk@nvidia.com>

isaaclab_arena/utils/isaaclab_utils/simulation_app.py

Replace the static lift_object_model.pt fixture with a dynamic train-then-eval round trip in a dedicated test_rsl_rl.py. The previous test_rl_policy_lift_object loaded a pre-committed checkpoint that had no params/agent.yaml alongside it, causing a FileNotFoundError after the PR switched RslRlActionPolicy to load agent config from that file instead of a separate JSON. The new test_rl_train_and_eval_lift_object trains for one iteration via IsaacLab's rsl_rl/train.py (which saves params/agent.yaml), locates the produced checkpoint by mtime, then runs policy_runner.py against it. This exercises the full train-to-eval pipeline and keeps test_policy_runner.py focused on policy-runner-only tests. Signed-off-by: Clemens Volk <cvolk@nvidia.com>

alexmillane

Looks great! I have a couple of nits. Feel free to do those in follow ups.

docs/pages/example_workflows/reinforcement_learning/step_2_policy_training.rst

docs/README.md

isaaclab_arena_examples/policy/base_rsl_rl_policy.py

Address two nits from PR #465 review: - Replace /isaac-sim/python.sh with the python alias in all RL workflow step pages (steps 1-3), consistent with other doc pages in the repo (e.g. static_manipulation workflow). - Remove --headless from the default training commands in step 2; add a tip directing users to pass it for headless server runs. New users benefit from seeing visual feedback by default. Signed-off-by: Clemens Volk <cvolk@nvidia.com>

## Summary Addresses two nits from the PR #465 review that were approved but not resolved before merge: - Use the `python` alias instead of `/isaac-sim/python.sh` in all RL workflow doc pages - Remove `--headless` as a default flag in the training command examples Signed-off-by: Clemens Volk <cvolk@nvidia.com>

qianl-nv and others added 21 commits March 3, 2026 23:54

Upgrade IsaacSim base docker from 5.0 to 5.1

39236b5

bump to IL latest main

e0fa516

Remove deprecated omni.log

d14e72c

IsaacLab fixes for arena unit test

8c3c261

Update IsaacLab submodule to main (ddb044eb)

2b525a0

Signed-off-by: Clemens Volk <cvolk@nvidia.com>

Merge branch 'main' into qianl/isaac_sim_5p1

34ddfbe

Bump to our own IsaacLab branch with fixes.

1e82aba

lint.

452e42c

Works but dirty.

793606e

Get hard coding of the policy out of the compiler.

40cf374

Cleanup

17168ac

Move environment callback somewhere more sensible.

a68b617

Remove RL scripts.

07a1828

Remove bug to get working.

75f758c

Update docs README to reflect host-based build workflow

3d42eec

Replace the outdated Docker-based instructions with the correct host workflow using a Python 3.11 venv. Signed-off-by: Clemens Volk <cvolk@nvidia.com>

Remove stale TODO from step_1 environment setup doc

f2c238b

Signed-off-by: Clemens Volk <cvolk@nvidia.com>

cvolkcvolk mentioned this pull request Mar 4, 2026

Update IsaacLab to latest main #336

Merged

Base automatically changed from qianl/isaac_sim_5p1 to main March 4, 2026 15:35

Merge branch 'main' into cvolk/isaac_lab_interop_on_5p1

68b4941

cvolkcvolk commented Mar 4, 2026

View reviewed changes

isaaclab_arena/utils/isaaclab_utils/simulation_app.py Outdated Show resolved Hide resolved

cvolkcvolk added 2 commits March 4, 2026 17:27

Remove disable_pinocchio_patch

aba48e7

cvolkcvolk enabled auto-merge (squash) March 6, 2026 08:24

cvolkcvolk changed the title ~~Interop on qianl_isaac_sim_5p1~~ Interop on isaac_sim_5p1 Mar 6, 2026

cvolkcvolk mentioned this pull request Mar 6, 2026

Isaac Lab Interop. #413

Draft

Merge branch 'main' into cvolk/isaac_lab_interop_on_5p1

60681a9

alexmillane approved these changes Mar 6, 2026

View reviewed changes

cvolkcvolk merged commit 4fd95ba into main Mar 6, 2026
5 checks passed

cvolkcvolk mentioned this pull request Mar 9, 2026

Follow-up doc nits from PR #465 #476

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Interop on isaac_sim_5p1#465

Interop on isaac_sim_5p1#465
cvolkcvolk merged 25 commits intomainfrom
cvolk/isaac_lab_interop_on_5p1

cvolkcvolk commented Mar 4, 2026 •

edited

Loading

Uh oh!

Uh oh!

alexmillane left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

cvolkcvolk commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Deleted

Docs updated

Usage

Uh oh!

Uh oh!

alexmillane left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cvolkcvolk commented Mar 4, 2026 •

edited

Loading