IE801 Project (Team 11)

Team 11 : JeongWoo Park (20243347), Sojeong Rhee (20243606)

Title : Test time adaptation in Offline RL

This repository is forked from Offbench

Main Idea

The project is based on the OPEX, introduced in Park et al., Is Value Learning Really the Main Bottleneck in Offline RL? (NeurIPS 2024 Workshop) Since the paper only reported single step IQL results without implementation details, we applied multi-step OPEX and normalization with gradient norm settings.

Methods

Hyperparameter Search

Setup the Environment

To setup the environment, we recommend to use docker. Simply run

./docker_run.sh

Run Experiments

Inside docker container, simply run

./run.sh

You can modify run.sh file with specific environments and algorithms.

Weights and Biases Online Visualization Integration

This codebase can also log to W&B online visualization platform. To log to W&B, you first need to set your W&B API key environment variable and add --logging.online when launching the script. Alternatively, you could simply run wandb login.

Results

Overall Results on Antmaze dataset

Results on Antmaze-Umaze-Diverse-v2, num_steps = 1