Skip to content

Commit 6b794ac

Browse files
Mayankm96jsmith-bdaikellyguo11
authored
Adds USD-level randomization mode to event manager (#2040)
# Description Certain scene-level randomizations (such as randomizing the scale) must happen before the simulation starts playing. To this end, the MR adds a new event mode called "prestartup," which gets called right after the scene design is complete and before the simulation is played. Since the scene entities cannot be resolved before the simulation starts playing (as we currently rely on PhysX to provide us with the joint/body ordering), the MR adds a callback to resolve the scene entity configurations separately once the simulation plays. This MR replaces the prior implementation in #1165 ## Type of change - New feature (non-breaking change which adds functionality) - This change requires a documentation update ## Screenshots If you execute: ```bash ./isaaclab.sh -p scripts/tutorials/03_envs/create_cube_base_env.py --num_envs 32 ``` Output: ![image](https://github.com/user-attachments/assets/a4b3a5e7-ff5c-4593-97ce-3b5af19bfde2) ## Checklist - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there --------- Signed-off-by: Mayank Mittal <[email protected]> Co-authored-by: James Smith <[email protected]> Co-authored-by: Kelly Guo <[email protected]>
1 parent 80e3964 commit 6b794ac

File tree

10 files changed

+308
-55
lines changed

10 files changed

+308
-55
lines changed

.vscode/tools/settings.template.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,8 @@
5656
"discretization",
5757
"trimesh",
5858
"uninstanceable",
59-
"coeff"
59+
"coeff",
60+
"prestartup"
6061
],
6162
// This enables python language server. Seems to work slightly better than jedi:
6263
"python.languageServer": "Pylance",

scripts/tutorials/03_envs/create_cube_base_env.py

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,15 @@
99
1010
While going through this tutorial, we recommend you to pay attention to how a custom action term
1111
is defined. The action term is responsible for processing the raw actions and applying them to the
12-
scene entities. The rest of the environment is similar to the previous tutorials.
12+
scene entities.
13+
14+
We also define an event term called 'randomize_scale' that randomizes the scale of
15+
the cube. This event term has the mode 'prestartup', which means that it is applied on the USD stage
16+
before the simulation starts. Additionally, the flag 'replicate_physics' is set to False,
17+
which means that the cube is not replicated across multiple environments but rather each
18+
environment gets its own cube instance.
19+
20+
The rest of the environment is similar to the previous tutorials.
1321
1422
.. code-block:: bash
1523
@@ -223,6 +231,9 @@ def __post_init__(self):
223231
class EventCfg:
224232
"""Configuration for events."""
225233

234+
# This event term resets the base position of the cube.
235+
# The mode is set to 'reset', which means that the base position is reset whenever
236+
# the environment instance is reset (because of terminations defined in 'TerminationCfg').
226237
reset_base = EventTerm(
227238
func=mdp.reset_root_state_uniform,
228239
mode="reset",
@@ -237,6 +248,19 @@ class EventCfg:
237248
},
238249
)
239250

251+
# This event term randomizes the scale of the cube.
252+
# The mode is set to 'prestartup', which means that the scale is randomize on the USD stage before the
253+
# simulation starts.
254+
# Note: USD-level randomizations require the flag 'replicate_physics' to be set to False.
255+
randomize_scale = EventTerm(
256+
func=mdp.randomize_rigid_body_scale,
257+
mode="prestartup",
258+
params={
259+
"scale_range": {"x": (0.5, 1.5), "y": (0.5, 1.5), "z": (0.5, 1.5)},
260+
"asset_cfg": SceneEntityCfg("cube"),
261+
},
262+
)
263+
240264

241265
##
242266
# Environment configuration
@@ -248,7 +272,11 @@ class CubeEnvCfg(ManagerBasedEnvCfg):
248272
"""Configuration for the locomotion velocity-tracking environment."""
249273

250274
# Scene settings
251-
scene: MySceneCfg = MySceneCfg(num_envs=args_cli.num_envs, env_spacing=2.5)
275+
# The flag 'replicate_physics' is set to False, which means that the cube is not replicated
276+
# across multiple environments but rather each environment gets its own cube instance.
277+
# This allows modifying the cube's properties independently for each environment.
278+
scene: MySceneCfg = MySceneCfg(num_envs=args_cli.num_envs, env_spacing=2.5, replicate_physics=False)
279+
252280
# Basic settings
253281
observations: ObservationsCfg = ObservationsCfg()
254282
actions: ActionsCfg = ActionsCfg()
@@ -261,6 +289,7 @@ def __post_init__(self):
261289
# simulation settings
262290
self.sim.dt = 0.01
263291
self.sim.physics_material = self.scene.terrain.physics_material
292+
self.sim.render_interval = 2 # render interval should be a multiple of decimation
264293

265294

266295
def main():

source/isaaclab/config/extension.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[package]
22

33
# Note: Semantic Versioning is used: https://semver.org/
4-
version = "0.36.1"
4+
version = "0.36.2"
55

66
# Description
77
title = "Isaac Lab framework for Robot Learning"

source/isaaclab/docs/CHANGELOG.rst

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,19 @@
11
Changelog
22
---------
33

4+
0.36.2 (2025-03-12)
5+
~~~~~~~~~~~~~~~~~~~
6+
7+
Added
8+
^^^^^
9+
10+
* Added a new event mode called "prestartup", which gets called right after the scene design is complete
11+
and before the simulation is played.
12+
* Added a callback to resolve the scene entity configurations separately once the simulation plays,
13+
since the scene entities cannot be resolved before the simulation starts playing
14+
(as we currently rely on PhysX to provide us with the joint/body ordering)
15+
16+
417
0.36.1 (2025-03-10)
518
~~~~~~~~~~~~~~~~~~~
619

source/isaaclab/isaaclab/envs/direct_marl_env.py

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,10 @@ def __init__(self, cfg: DirectMARLEnvCfg, render_mode: str | None = None, **kwar
9595
else:
9696
raise RuntimeError("Simulation context already exists. Cannot create a new one.")
9797

98+
# make sure torch is running on the correct device
99+
if "cuda" in self.device:
100+
torch.cuda.set_device(self.device)
101+
98102
# print useful information
99103
print("[INFO]: Base environment:")
100104
print(f"\tEnvironment device : {self.device}")
@@ -126,6 +130,17 @@ def __init__(self, cfg: DirectMARLEnvCfg, render_mode: str | None = None, **kwar
126130
else:
127131
self.viewport_camera_controller = None
128132

133+
# create event manager
134+
# note: this is needed here (rather than after simulation play) to allow USD-related randomization events
135+
# that must happen before the simulation starts. Example: randomizing mesh scale
136+
if self.cfg.events:
137+
self.event_manager = EventManager(self.cfg.events, self)
138+
print("[INFO] Event Manager: ", self.event_manager)
139+
140+
# apply USD-related randomization events
141+
if "prestartup" in self.event_manager.available_modes:
142+
self.event_manager.apply(mode="prestartup")
143+
129144
# play the simulator to activate physics handles
130145
# note: this activates the physics simulation view that exposes TensorAPIs
131146
# note: when started in extension mode, first call sim.reset_async() and then initialize the managers
@@ -138,15 +153,6 @@ def __init__(self, cfg: DirectMARLEnvCfg, render_mode: str | None = None, **kwar
138153
# this shouldn't cause an issue since later on, users do a reset over all the environments so the lazy buffers would be reset.
139154
self.scene.update(dt=self.physics_dt)
140155

141-
# -- event manager used for randomization
142-
if self.cfg.events:
143-
self.event_manager = EventManager(self.cfg.events, self)
144-
print("[INFO] Event Manager: ", self.event_manager)
145-
146-
# make sure torch is running on the correct device
147-
if "cuda" in self.device:
148-
torch.cuda.set_device(self.device)
149-
150156
# check if debug visualization is has been implemented by the environment
151157
source_code = inspect.getsource(self._set_debug_vis_impl)
152158
self.has_debug_vis_implementation = "NotImplementedError" not in source_code

source/isaaclab/isaaclab/envs/direct_rl_env.py

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,10 @@ def __init__(self, cfg: DirectRLEnvCfg, render_mode: str | None = None, **kwargs
101101
else:
102102
raise RuntimeError("Simulation context already exists. Cannot create a new one.")
103103

104+
# make sure torch is running on the correct device
105+
if "cuda" in self.device:
106+
torch.cuda.set_device(self.device)
107+
104108
# print useful information
105109
print("[INFO]: Base environment:")
106110
print(f"\tEnvironment device : {self.device}")
@@ -132,6 +136,17 @@ def __init__(self, cfg: DirectRLEnvCfg, render_mode: str | None = None, **kwargs
132136
else:
133137
self.viewport_camera_controller = None
134138

139+
# create event manager
140+
# note: this is needed here (rather than after simulation play) to allow USD-related randomization events
141+
# that must happen before the simulation starts. Example: randomizing mesh scale
142+
if self.cfg.events:
143+
self.event_manager = EventManager(self.cfg.events, self)
144+
print("[INFO] Event Manager: ", self.event_manager)
145+
146+
# apply USD-related randomization events
147+
if "prestartup" in self.event_manager.available_modes:
148+
self.event_manager.apply(mode="prestartup")
149+
135150
# play the simulator to activate physics handles
136151
# note: this activates the physics simulation view that exposes TensorAPIs
137152
# note: when started in extension mode, first call sim.reset_async() and then initialize the managers
@@ -144,15 +159,6 @@ def __init__(self, cfg: DirectRLEnvCfg, render_mode: str | None = None, **kwargs
144159
# this shouldn't cause an issue since later on, users do a reset over all the environments so the lazy buffers would be reset.
145160
self.scene.update(dt=self.physics_dt)
146161

147-
# -- event manager used for randomization
148-
if self.cfg.events:
149-
self.event_manager = EventManager(self.cfg.events, self)
150-
print("[INFO] Event Manager: ", self.event_manager)
151-
152-
# make sure torch is running on the correct device
153-
if "cuda" in self.device:
154-
torch.cuda.set_device(self.device)
155-
156162
# check if debug visualization is has been implemented by the environment
157163
source_code = inspect.getsource(self._set_debug_vis_impl)
158164
self.has_debug_vis_implementation = "NotImplementedError" not in source_code
@@ -199,7 +205,8 @@ def __init__(self, cfg: DirectRLEnvCfg, render_mode: str | None = None, **kwargs
199205
if "startup" in self.event_manager.available_modes:
200206
self.event_manager.apply(mode="startup")
201207

202-
# -- set the framerate of the gym video recorder wrapper so that the playback speed of the produced video matches the simulation
208+
# set the framerate of the gym video recorder wrapper so that the playback speed of the produced
209+
# video matches the simulation
203210
self.metadata["render_fps"] = 1 / self.step_dt
204211

205212
# print the environment information

source/isaaclab/isaaclab/envs/manager_based_env.py

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,10 @@ def __init__(self, cfg: ManagerBasedEnvCfg):
9999
raise RuntimeError("Simulation context already exists. Cannot create a new one.")
100100
self.sim: SimulationContext = SimulationContext.instance()
101101

102+
# make sure torch is running on the correct device
103+
if "cuda" in self.device:
104+
torch.cuda.set_device(self.device)
105+
102106
# print useful information
103107
print("[INFO]: Base environment:")
104108
print(f"\tEnvironment device : {self.device}")
@@ -132,6 +136,16 @@ def __init__(self, cfg: ManagerBasedEnvCfg):
132136
else:
133137
self.viewport_camera_controller = None
134138

139+
# create event manager
140+
# note: this is needed here (rather than after simulation play) to allow USD-related randomization events
141+
# that must happen before the simulation starts. Example: randomizing mesh scale
142+
self.event_manager = EventManager(self.cfg.events, self)
143+
print("[INFO] Event Manager: ", self.event_manager)
144+
145+
# apply USD-related randomization events
146+
if "prestartup" in self.event_manager.available_modes:
147+
self.event_manager.apply(mode="prestartup")
148+
135149
# play the simulator to activate physics handles
136150
# note: this activates the physics simulation view that exposes TensorAPIs
137151
# note: when started in extension mode, first call sim.reset_async() and then initialize the managers
@@ -146,10 +160,6 @@ def __init__(self, cfg: ManagerBasedEnvCfg):
146160
# add timeline event to load managers
147161
self.load_managers()
148162

149-
# make sure torch is running on the correct device
150-
if "cuda" in self.device:
151-
torch.cuda.set_device(self.device)
152-
153163
# extend UI elements
154164
# we need to do this here after all the managers are initialized
155165
# this is because they dictate the sensors and commands right now
@@ -231,9 +241,6 @@ def load_managers(self):
231241
# -- observation manager
232242
self.observation_manager = ObservationManager(self.cfg.observations, self)
233243
print("[INFO] Observation Manager:", self.observation_manager)
234-
# -- event manager
235-
self.event_manager = EventManager(self.cfg.events, self)
236-
print("[INFO] Event Manager: ", self.event_manager)
237244

238245
# perform events at the start of the simulation
239246
# in-case a child implementation creates other managers, the randomization should happen

source/isaaclab/isaaclab/envs/mdp/events.py

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,8 @@
1919

2020
import carb
2121
import omni.physics.tensors.impl.api as physx
22+
import omni.usd
23+
from pxr import Gf, Sdf, UsdGeom, Vt
2224

2325
import isaaclab.sim as sim_utils
2426
import isaaclab.utils.math as math_utils
@@ -31,6 +33,116 @@
3133
from isaaclab.envs import ManagerBasedEnv
3234

3335

36+
def randomize_rigid_body_scale(
37+
env: ManagerBasedEnv,
38+
env_ids: torch.Tensor | None,
39+
scale_range: tuple[float, float] | dict[str, tuple[float, float]],
40+
asset_cfg: SceneEntityCfg,
41+
relative_child_path: str | None = None,
42+
):
43+
"""Randomize the scale of a rigid body asset in the USD stage.
44+
45+
This function modifies the "xformOp:scale" property of all the prims corresponding to the asset.
46+
47+
It takes a tuple or dictionary for the scale ranges. If it is a tuple, then the scaling along
48+
individual axis is performed equally. If it is a dictionary, the scaling is independent across each dimension.
49+
The keys of the dictionary are ``x``, ``y``, and ``z``. The values are tuples of the form ``(min, max)``.
50+
51+
If the dictionary does not contain a key, the range is set to one for that axis.
52+
53+
Relative child path can be used to randomize the scale of a specific child prim of the asset.
54+
For example, if the asset at prim path expression "/World/envs/env_.*/Object" has a child
55+
with the path "/World/envs/env_.*/Object/mesh", then the relative child path should be "mesh" or
56+
"/mesh".
57+
58+
.. attention::
59+
Since this function modifies USD properties that are parsed by the physics engine once the simulation
60+
starts, the term should only be used before the simulation starts playing. This corresponds to the
61+
event mode named "usd". Using it at simulation time, may lead to unpredictable behaviors.
62+
63+
.. note::
64+
When randomizing the scale of individual assets, please make sure to set
65+
:attr:`isaaclab.scene.InteractiveSceneCfg.replicate_physics` to False. This ensures that physics
66+
parser will parse the individual asset properties separately.
67+
"""
68+
# check if sim is running
69+
if env.sim.is_playing():
70+
raise RuntimeError(
71+
"Randomizing scale while simulation is running leads to unpredictable behaviors."
72+
" Please ensure that the event term is called before the simulation starts by using the 'usd' mode."
73+
)
74+
75+
# extract the used quantities (to enable type-hinting)
76+
asset: RigidObject = env.scene[asset_cfg.name]
77+
78+
if isinstance(asset, Articulation):
79+
raise ValueError(
80+
"Scaling an articulation randomly is not supported, as it affects joint attributes and can cause"
81+
" unexpected behavior. To achieve different scales, we recommend generating separate USD files for"
82+
" each version of the articulation and using multi-asset spawning. For more details, refer to:"
83+
" https://isaac-sim.github.io/IsaacLab/main/source/how-to/multi_asset_spawning.html"
84+
)
85+
86+
# resolve environment ids
87+
if env_ids is None:
88+
env_ids = torch.arange(env.scene.num_envs, device="cpu")
89+
else:
90+
env_ids = env_ids.cpu()
91+
92+
# acquire stage
93+
stage = omni.usd.get_context().get_stage()
94+
# resolve prim paths for spawning and cloning
95+
prim_paths = sim_utils.find_matching_prim_paths(asset.cfg.prim_path)
96+
97+
# sample scale values
98+
if isinstance(scale_range, dict):
99+
range_list = [scale_range.get(key, (1.0, 1.0)) for key in ["x", "y", "z"]]
100+
ranges = torch.tensor(range_list, device="cpu")
101+
rand_samples = math_utils.sample_uniform(ranges[:, 0], ranges[:, 1], (len(env_ids), 3), device="cpu")
102+
else:
103+
rand_samples = math_utils.sample_uniform(*scale_range, (len(env_ids), 1), device="cpu")
104+
rand_samples = rand_samples.repeat(1, 3)
105+
# convert to list for the for loop
106+
rand_samples = rand_samples.tolist()
107+
108+
# apply the randomization to the parent if no relative child path is provided
109+
# this might be useful if user wants to randomize a particular mesh in the prim hierarchy
110+
if relative_child_path is None:
111+
relative_child_path = ""
112+
elif not relative_child_path.startswith("/"):
113+
relative_child_path = "/" + relative_child_path
114+
115+
# use sdf changeblock for faster processing of USD properties
116+
with Sdf.ChangeBlock():
117+
for i, env_id in enumerate(env_ids):
118+
# path to prim to randomize
119+
prim_path = prim_paths[env_id] + relative_child_path
120+
# spawn single instance
121+
prim_spec = Sdf.CreatePrimInLayer(stage.GetRootLayer(), prim_path)
122+
123+
# get the attribute to randomize
124+
scale_spec = prim_spec.GetAttributeAtPath(prim_path + ".xformOp:scale")
125+
# if the scale attribute does not exist, create it
126+
has_scale_attr = scale_spec is not None
127+
if not has_scale_attr:
128+
scale_spec = Sdf.AttributeSpec(prim_spec, prim_path + ".xformOp:scale", Sdf.ValueTypeNames.Double3)
129+
130+
# set the new scale
131+
scale_spec.default = Gf.Vec3f(*rand_samples[i])
132+
133+
# ensure the operation is done in the right ordering if we created the scale attribute.
134+
# otherwise, we assume the scale attribute is already in the right order.
135+
# note: by default isaac sim follows this ordering for the transform stack so any asset
136+
# created through it will have the correct ordering
137+
if not has_scale_attr:
138+
op_order_spec = prim_spec.GetAttributeAtPath(prim_path + ".xformOpOrder")
139+
if op_order_spec is None:
140+
op_order_spec = Sdf.AttributeSpec(
141+
prim_spec, UsdGeom.Tokens.xformOpOrder, Sdf.ValueTypeNames.TokenArray
142+
)
143+
op_order_spec.default = Vt.TokenArray(["xformOp:translate", "xformOp:orient", "xformOp:scale"])
144+
145+
34146
class randomize_rigid_body_material(ManagerTermBase):
35147
"""Randomize the physics materials on all geometries of the asset.
36148

0 commit comments

Comments
 (0)