Skip to content

Conversation

gcroci2
Copy link

@gcroci2 gcroci2 commented Jun 11, 2025

Description

This PR extends the RecurrentPPO implementation to support vanilla RNN layers (nn.RNN) in addition to the existing LSTM support. Users can now choose between LSTM and vanilla RNN through the recurrent_layer_type parameter in policy_kwargs.

Context

  • I have raised an issue to propose this change (closes [Feature Request] Add vanilla RNN support to RecurrentPPO #295)
  • Note 1: I implemented this feature without waiting for the issue discussion as we needed this functionality immediately for our research (we are currently using a fork of the repository for this purpose). I'm fully open to modifying the implementation based on maintainer feedback and community discussion.
  • Note 2: I kept the existing lstm_* variable names (e.g., lstm_hidden_size, n_lstm_layers) to avoid breaking existing code. I'm open to renaming them to more neutral names (e.g., hidden_size, n_layers) if the maintainers prefer, but wanted to prioritize backward compatibility in this initial implementation.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation (update in the documentation)

Checklist:

  • I've read the CONTRIBUTION guide (required)
  • The functionality/performance matches that of the source (required for new training algorithms or training-related features).
  • I have updated the tests accordingly (required for a bug fix or a new feature).
  • I have included an example of using the feature (required for new features).
  • I have included baseline results (required for new training algorithms or training-related features).
  • I have updated the documentation accordingly.
  • I have updated the changelog accordingly (required).
  • I have reformatted the code using make format (required)
  • I have checked the codestyle using make check-codestyle and make lint (required)
  • I have ensured make pytest and make type both pass. (required)

@gcroci2 gcroci2 marked this pull request as draft June 11, 2025 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] Add vanilla RNN support to RecurrentPPO
1 participant