Saving and Loading trpo model for policynn

[ryanmaxwell96]

Hello,

I'm sorry for asking so many questions. But would you know how to go about saving and loading the subclassed model? I have tried everything I can think of including trying to change the subclassed model to a functional or sequential but I just keep running into issues.

Thanks,

Ryan

[bernardocortez]

I struggled with the same thing! I decided to do it in the hard way: save the weights of the network, all the important variables of the model that I want to keep track of and the replay buffers.
This is not perfect, however, since I have experienced that the agent always gets worse if loaded. I read somewhere that it might has something to do with the state of the optimizer