Evaluating Frame Stacking Impact in Reinforcement Learning

This repository contains the code to replicate the experiments in Diploma project "Evaluating Frame Stacking Impact in Reinforcement Learning".

Installation

To install the required dependencies, run the following command:

pip install -r requirements.txt

Usage

To train the agent, you can use training scripts provided in the car-racing-train.py and lunar-lander-train.py files.

Each agent can be trained with different configurations. By default, the environments use partial observations. For CarRacing the observations are cropped to remove the bottom indicators, and for LunarLander the linear and angular velocities are removed. To use full observations, you can set the --full flag.

To train the agent with frame stacking, you can set the --frame-stack <number_of_frames> flag followed by the number of frames to stack.

To train the agent with the LSTM layer, you can set the --lstm flag.

Each of these flags should be used separately. You should also specify the --try flag to save the results in the correct directory.

For example, to train the agent on CarRacing with frame stacking of 4, you can run the following:

  python car-racing-train.py --frame-stack 4 --try 1

Evaluation

To evaluate the agent, you can use evaluation scripts provided in the car-racing-eval.py and lunar-lander-eval.py files with the same flags as you used on training. Each evaluation is done on 100 episodes, so it may take a while to complete.

For example, to evaluate the agent on CarRacing with frame stacking of 4, you can run the following:

  python car-racing-eval.py --frame-stack 4 --try 1

Models

The models used in the experiments are saved in the best directory. You can use them to evaluate the agent or train your own instead.