behavior of on_episode_end() in the ReplayBuffer class when I am only adding a select number of time-steps per interaction.

[WilderLavington]

Hello,

I am currently using the Replay buffer class for storing image data and wanted to know what the effects of calling on_episode_end() for separate buffers of examples. In particular, I am storing examples with reward = 1 in one buffer and reward equal to zero in another. In this environment, reward is equal to 1 only at the end of the episode, and thus I only add one "negative" transition tuple to my replay buffer per episode. In this case, do I still need to call on_episode_end()?

[ymd-h]

@WilderLavington
Thank you for your feedback.

The answer is perhaps "No", but I recommend that you call it.

Generally speaking, we developers assume users call on_episode_end() method in between different episodes,
so that it is safer to call it.

In detail, on_episode_end() do actual works when N-step rewards or memory compression features are enabled.
This situation might be changed in future release.

[WilderLavington]

great thank you!