RL-pytorch

PyTorch implementations of Deep Reinforcement Learning algorithms.

Installation

pip install -r requirements.txt

Implemented Algorithms

• Deep Q Networks (DQN) [paper] [official code]
• Deep Double Q Networks (DDQN) [paper]
• Dueling Network Architectures for Deep Reinforcement Learning (DuelDQN) [paper]
• Continuous control with deep reinforcement learning (DDPG) [paper]
• Addressing Function Approximation Error in Actor-Critic Methods (TD3) [paper] [official code]
• Soft Actor-Critic Algorithms and Applications (SAC) [paper] [official code]
• Trust Region Policy Optimization (TRPO) [paper] [official code]
• Proximal Policy Optimization (PPO) [paper] [official code]

Run Experiments

# Train an RL agent.
python train_agent.py agent=ppo env.id=Hopper-v5

# Watch the training results
tensorboard --logdir=./logs

# Collect expert demonstrations
python collect_demo.py agent=ppo env.id=Hopper-v5 expert_model_path=logs/benchmark/Hopper-v5/ppo/20260127_151112/ckpt/best_model.pt

Citation

If you use RL-pytorch in your research, please cite:

@software{rl_pytorch,
  author       = {Yi{-}Chen Li},
  title        = {RL-pytorch: PyTorch Implementations of Deep Reinforcement Learning Algorithms},
  year         = 2026,
  url          = {https://github.com/liyc-ai/RL-pytorch}
}

Acknowledgement

With the progress of this project, I found many open-source materials on the Internet to be excellent references. I am deeply grateful for the efforts of their authors. Below is a detailed list. Additionally, I would like to extend my thanks to my friends from LAMDA-RL for our helpful discussions.

Codebase

• tianshou
• stable-baselines3
• stable-baselines-contrib
• stable-baselines
• spinningup
• RL-Adventure2
• unstable_baselines
• d4rl_evaluations
• TD3
• pytorch-trpo

Blog

• The 37 Implementation Details of Proximal Policy Optimization

Tutorial

• OpenAI SpinningUp