Understanding the data

[DavidRSeWell]

Hello, I am trying to verify that I understand the data correctly. Each saved file has 100 saved trajectories from the SAC agent. For hopper v3 this amounts to each file being 100k x 16. So I assume that each trajectory is of length 1k. It appears however that during a single trajectory there might be a reset happening since the done column of the data (the last column) may be 1 before the 1k timestep happens. So, I am wondering if a random reset is happening so there can essentially be more than 100 trajectories getting generated. Essentially in many of the files there is > 100 dones = 1 and I am wondering what these means for how many unique trajectories there are.

[DavidRSeWell]

Adding some additional info that may help to understand my concerns. In this file "sac-hopper-jointdec_50_legjoint.npz" there are 145 occurrences of done > 0.

If you plot the rewards or z pos in 1k timestep chunks it is clear that the episode is being reset.

Also in that file when I am looking between the 2387 and 3381 index which going off the index given by the done conditions should be a single trajectory. I get the following plot. It appears that the episode was reset about 600 timesteps in but the done condition was still 0.

[mpatacchiola]

Hey @DavidRSeWell, I am trying to recall the details of this one. I believe the behaviour you are seeing may be due to the fact that in some environments like Hopper the agent is respawn under particular "unhealthy" conditions (check the docs here). The 1k steps in this case may include multiple restarts.

[DavidRSeWell]

@mpatacchiola Thanks for the response. Yes, that is what is happening.