[Question] Accessing per-environment observations

[george-nehma]

Question

I am creating a DirectRLEnv and am looking at creating the observations for my environment. Currently when I run the simulation i set the number of envs to 4. When I call for example self._camera.data.output['rgb'] it gives me an output that is of size torch.Size([4, 100, 100, 3]), which makes sense. But my question is, how should I go about getting the observation for the specific environment, because each environment would be different so if I hard code the observation to be the [0] dimension won't that always return the first environments data? Or is this handled on the back end?

[RandomOakForest]

Thank you for posting this. Each index along the first dimension ([env_idx, :, :, :]) corresponds to data from a specific environment. For example, [0, :, :, :] is the first environment, [1, :, :, :] the second, and so on. IsaacLab automatically manages the batched structures for you during simulation stepping. The environments run in parallel, and sensor outputs (including camera images) are batched together in order. If you use `` (i.e., self._camera.data.output['rgb']), you will only get the observation for the first environment, not all environments. This means you must index using the environment number appropriate for the specific sub-environment you care about.

If you want the observation for environment i, simply index with i:

rgb_obs = self._camera.data.output['rgb'][i]   # shape [100, 100, 3]

This gives you the image for the desired environment. The system does not remap individual data for you—it is up to your logic to index the correct slice.

Let us know if this helped.

[george-nehma]

Thanks for the reply, but I am more concerned with the implementation in the configuration of the environment. The issue im seeing is that all environments behave in the same way before shortly blowing up with NaNs and inf values and I want to make sure I am applying my forces and torques correctly as well as attaining the correct observation for each environment. So if I want to access the lin_vel from my imu in the _get_observations function, calling lin_vel = self._imu.data.lin_vel_b will give each environment their corresponding linear velocity?