🔰 Value-Evolutionary-Based Reinforcement Learning (VEB-RL)

(ICML 2024) The official code for VEB-RL from Value-Evolutionary-Based Reinforcement Learning) by Pengyi Li.

🚩 Method

VEB-RL is a hybrid framework specifically designed for value-based reinforcement learning methods. VEB-RL integrates genetic algorithms (GA) and cross-entropy method (CEM), using TD error as fitness for more accurate value function approximation. We also propose the Elite Interaction Mechanism to improve sample quality. VEB-RL significantly enhances value-based RL across various tasks.

Tip

🔥 🔥 🔥 If you are interested in ERL for policy search or other hybrid algorithms combining EA and RL, we strongly recommend reading our survey paper: Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms. It provides a comprehensive and accessible overview of research directions and classifications suitable for researchers with various backgrounds.

🙏 Citation

If you do find our paper or the repository helpful (or if you would be so kind as to offer us some encouragement), please consider kindly giving a star, and citing our paper.

@inproceedings{li2023value,
  title={Value-Evolutionary-Based Reinforcement Learning},
  author={Li, Pengyi and Jianye, HAO and Tang, Hongyao and Zheng, Yan and Barez, Fazl},
  booktitle={Forty-first International Conference on Machine Learning},
  year={2023}
}

🛠️ Instructions

You need to create a Weights & Biases account for visualizing results, and you should already have conda installed.

First, we create an environment based on the provided requirements.txt:

conda create --name VEBRL --file requirements.txt

Activate the environment:

conda activate VEBRL

Then enter either the GA_VEB folder or the CEM_VEB folder and directly run run.sh

cd ./GA_VEB or cd ./CEM_VEB
chmod 777 ./run.sh
./run.sh

The specific hyperparameter settings need to be adjusted according to the original paper.

🔰 License & Acknowledgements

VEB-RL is licensed under the MIT license.

✉ Contact

For any questions, please feel free to email [email protected].