sbs3

[TwiceMao]

Hi，@zhihanyang2022, Thanks for your awesome work. I want to use your method. I have implemented a specific SAC algorithm on stable baselines3. Can I modify the SAC algorithm on stable baselines to a recurrent version? Is it convenient to do so? Or is there anything to refer~

[zhihanyang2022]

It's not a trivial task to modify an RL algorithm into its recurrent version - this is why this codebase exists. You need to make changes to a lot of things - experience buffer, logging and etc. This is even harder in SB3, because it's such a huge codebase and is highly complex. It might be easier to do everything in this codebase.

In case you don't know, this codebase also supports non-recurrent algorithms. For example, you can run the standard SAC. You can modify this script to run your specific SAC. I think once you figure out how to do that, you can easily modify this script to create a recurrent version of your specific SAC.

I've benchmarked all non-recurrent algorithms in this codebase against SB3 here. We use the exact same hyperparameters. In the early development of this codebase, I borrowed a lot of insights from SB3.

[TwiceMao]

Hi，@zhihanyang2022, in Table 3 of the main paper, what do SAC-obs and SAC-state stand for? I can’t seem to find any explanation in the paper or the appendix.