Disentangling Static and Dynamic Information for Reducing Static Bias in Action Recognition

This repository provides the source code for reproducing, training, and evaluating the models proposed in the paper “Disentangling Static and Dynamic Information for Reducing Static Bias in Action Recognition.”

Repository Structure and Experimental Flow

• args/ : Module defining command-line arguments. It consolidates training-related settings such as the number of epochs, batch size, model selection, and dataset name.
• cfg/config.yaml : YAML file describing dataset paths and model-specific hyperparameters. Centralizes configuration management for each dataset and model.
• main_pl.py : Entry point for training and evaluation using Lightning Trainer. It initializes modules based on the contents of args and cfg.
• dataset/ : Implements data loaders, including WebDataset. The TrainValDataModule provides training and validation data as a LightningDataModule.
• model/ : Implements models based on TimeSformer and a set of Adversarial Diff models designed to disentangle static and dynamic information.
• setup/ : Configuration for optimizers and schedulers.

The basic experimental flow is as follows:

1. Edit cfg/config.yaml to specify dataset paths and model hyperparameters.
2. Check the command-line arguments supported in args/arg_parse.py, and specify necessary experimental settings (e.g., epochs, batch size, model name, dataset name) either in a bash script or directly through the CLI.
3. Run main_pl.py to perform training and evaluation using Lightning Trainer.

Dataset Preparation

Before running the code, make sure to place your datasets using symbolic links or similar methods in the following directory structure:

project_root/
├─ data/
│  └─ wds/
│     └─ standard/
│        ├─ train_Temporal32/         # Example: Training data for Temporal32
│        ├─ val_Temporal32/
│        ├─ train_UCF101/
│        └─ val_UCF101/
└─ cfg/config.yaml                    # Configuration file pointing to dataset paths

For example, if using Temporal32 in the datasets section of cfg/config.yaml, make sure train_dir and val_dir match the paths above.

Environment Setup

If you are using Conda, you can reproduce the environment by creating a virtual environment from environment.yml:

conda env create -f environment.yml
conda activate conda_env_kobayashi

Example of Execution Using a Bash Script

The following example demonstrates how to train the TimeSformer model on the Temporal32 dataset using two GPUs (ID 2 and 3):

CUDA_VISIBLE_DEVICES=2,3 \
python main_pl.py \
	--exp_name "test" \
	--add_tag "test" \
	-d Temporal32 \
	-m TimeSformer \
	-bs 8 \
	-w 2 \
	-e 10 \
	--use_scheduler \
	--devices "-1" \