Prithwijit_TIP

Research code for training image classifiers, generating saliency maps, perturbing masks, and evaluating explanation quality on ImageNet-style data.

The repository is organized as a collection of runnable scripts rather than a packaged Python library. The main workflows are:

• training baseline classifiers on MNIST or CUB
• generating RISE and SHAPE explanation masks for pretrained torchvision models
• evaluating masks with insertion/deletion curves and image-quality metrics
• creating noisy variants of saved saliency masks
• visualizing predictions and heatmap overlays

Repository Layout

Path	Purpose
train_folder/	Training code for MNIST and CUB-200-2011 with multiple torchvision backbones.
rise/	RISE and SHAPE saliency generation, causal evaluation helpers, and visualization scripts.
iqa/	Insertion/deletion evaluation plus full-reference image-quality metrics.
noisy_mask_gen/	Utilities for generating perturbed/noisy versions of existing .npy saliency masks.
tools/	Batch JSON generation and path-rewriting helpers for experiment bookkeeping.
utils/	Prediction export and statistics scripts for pretrained ResNet-50.
visuals/	Overlay and figure-generation scripts.
jsons/	Precomputed batch definitions used by some evaluation runs.
shape_analysis/, TRUST/	Analysis artifacts and notebooks used for experiments.

What This Code Assumes

• Most saliency and evaluation scripts assume a CUDA-enabled GPU and call .cuda() directly.
• Training in train_folder/train.py can fall back to CPU, but the saliency-generation and evaluation pipelines generally cannot without code changes.
• The main explanation workflows expect an ImageNet-style validation subset where each class has its own numeric folder.
• Several scripts still contain experiment-specific absolute paths. Search and update them before long runs.

Use this command to find the hardcoded paths that usually need cleanup:

rg -n "/home/prithwijit|/mnt/data-tmp|/data/prithwijit|TIP2025" rise iqa noisy_mask_gen tools visuals

The most important path to change is the shared mask cache used by the RISE/SHAPE scripts:

• rise/riser.py
• rise/shaper.py
• rise/rise_by_path.py
• rise/shape_by_path.py
• rise/riser_selective.py
• rise/shaper_selective.py

Those files currently expect a writable mask file such as /home/prithwijit/TIP2025/masks.npy.

Environment Setup

1. Create a Python environment

python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip

2. Install PyTorch first

Install torch and torchvision using the build that matches your CUDA runtime. If you are working on CPU only, install the CPU build instead.

3. Install the remaining dependencies

pip install numpy tqdm pillow matplotlib pandas scipy scikit-image scikit-learn opencv-python wandb piq lpips torchmetrics

Optional extras:

• jupyter if you want to open the notebooks in rise/, iqa/, TRUST/, or the repository root
• seaborn if you want extra plotting tools beyond the scripts already here

4. Configure Weights & Biases if you plan to train models

train_folder/train.py initializes a W&B run for every model. Either log in:

wandb login

or disable remote logging for local experiments:

export WANDB_MODE=offline

Required Data Layouts

ImageNet-style validation subset

Most scripts under rise/, iqa/, utils/, and visuals/ assume this directory structure:

ILSVRC2012_img_val_subset/
  0/
    ILSVRC2012_val_00000001.JPEG
    ...
  1/
    ...
  999/
    ...

Important details:

• folder names are parsed as class indices
• image extensions can be .jpg, .jpeg, .png, or .JPEG depending on the script
• images are resized to 224 x 224 internally

CUB training layout

train_folder/datasets.py expects CUB data under ./data/CUB relative to the directory from which you run the training command:

data/
  CUB/
    train/
      class_a/
        image1.jpg
      class_b/
        image2.jpg
    val/
      class_a/
      class_b/

Batch JSON layout

Several scripts consume a JSON file that points to saved saliency masks. tools/batching.py creates a file with this general structure:

{
  "tiny_batch": {
    "resnet50": {
      "rise": {
        "129": "/abs/path/to/mask.npy"
      }
    }
  },
  "small_batch": {},
  "large_batch": {}
}

Only the tiny_batch section is used by the current evaluation runners and the selective saliency scripts.

Typical End-to-End Workflow

For the main ImageNet-style explanation pipeline, the usual order is:

1. Prepare an ImageNet-style validation subset.
2. Update hardcoded paths in the scripts you plan to run.
3. Generate saliency masks with RISE or SHAPE.
4. Build a batch JSON that points at those masks.
5. Run insertion/deletion evaluation to produce CSV summaries.
6. Visualize overlays or compute downstream analysis statistics.

The sections below document each stage in more detail.

1. Train Baseline Classifiers

Entry point: train_folder/train.py

This script trains a fixed list of torchvision backbones:

• alexnet
• vgg16
• resnet50
• squeezenet
• densenet
• inception_v3
• googlenet
• shufflenet
• mobilenet
• resnext50
• wide_resnet
• mnasnet
• efficientnet

The script automatically loops through every model in that list and saves the best checkpoint for each one.

Train on MNIST

python train_folder/train.py \
  --dataset MNIST \
  --batch_size 64 \
  --image_size 224 \
  --epochs 30 \
  --lr 0.001 \
  --patience 5

Train on CUB

python train_folder/train.py \
  --dataset CUB \
  --batch_size 32 \
  --image_size 224 \
  --epochs 30 \
  --lr 0.0001 \
  --patience 5

Outputs:

• checkpoints are written to checkpoints/<DATASET>/ relative to your current working directory
• W&B logs one run per model

Notes:

• MNIST is downloaded automatically into ./data
• CUB must already exist under ./data/CUB/train and ./data/CUB/val
• all models start from pretrained torchvision weights

2. Generate Saliency Maps with RISE or SHAPE

The reusable saliency scripts live in rise/.

Full-folder generation

Entry points:

• rise/riser.py
• rise/shaper.py

Supported models in these scripts:

• alexnet
• vgg16
• resnet50
• densenet121
• googlenet
• mobilenet_v2
• mnasnet1_0
• efficientnet_b0

Example commands:

python rise/riser.py \
  --image_folder /abs/path/to/ILSVRC2012_img_val_subset \
  --output_folder /abs/path/to/rise_outputs

python rise/shaper.py \
  --image_folder /abs/path/to/ILSVRC2012_img_val_subset \
  --output_folder /abs/path/to/shape_outputs

Output layout:

<output_folder>/
  <model_name>/
    <class_index>/
      <image_stem>.npy

Behavior to know:

• the parent folder name is parsed as the target class index
• the scripts generate or reuse a shared mask cache file
• output masks are saved as NumPy arrays

JSON-driven generation by image path

Entry points:

• rise/rise_by_path.py
• rise/shape_by_path.py

These scripts expect a JSON mapping set names to explicit image paths:

{
  "high_prob_correct": [
    "/abs/path/to/129/ILSVRC2012_val_00000043.JPEG"
  ],
  "high_prob_incorrect_gt_second": [
    "/abs/path/to/7/ILSVRC2012_val_00001000.JPEG"
  ]
}

Example:

python rise/rise_by_path.py \
  --json_file /abs/path/to/image_sets.json \
  --output_folder /abs/path/to/rise_outputs_by_set \
  --max_samples 100

python rise/shape_by_path.py \
  --json_file /abs/path/to/image_sets.json \
  --output_folder /abs/path/to/shape_outputs_by_set \
  --max_samples 100

Current limitation:

• both *_by_path.py scripts are currently wired to resnet50 only

Selective generation from tiny_batch

Entry points:

• rise/riser_selective.py
• rise/shaper_selective.py

These scripts read the tiny_batch section of a batch JSON file, derive a whitelist of image basenames, and only generate masks for those images.

Example:

python rise/riser_selective.py \
  --image_folder /abs/path/to/ILSVRC2012_img_val_subset \
  --output_folder /abs/path/to/rise_tiny_batch \
  --json_file /abs/path/to/batches.json

python rise/shaper_selective.py \
  --image_folder /abs/path/to/ILSVRC2012_img_val_subset \
  --output_folder /abs/path/to/shape_tiny_batch \
  --json_file /abs/path/to/batches.json

Current limitations:

• rise/riser_selective.py currently runs only alexnet and densenet121
• rise/shaper_selective.py supports the same eight-model set as rise/shaper.py

3. Build Batch Definitions

Entry point: tools/batching.py

This script scans a directory tree of saved mask files and writes batches.json with:

• tiny_batch
• small_batch
• large_batch

Before running it, edit the results_dir constant inside tools/batching.py so it points at your explanation root directory.

Then run:

python tools/batching.py

By default the script:

• looks for a directory structure like <results_dir>/<model>/<explanation>/<class_index>/*.npy
• samples 50 class indices for tiny_batch
• uses the first file per class for small_batch
• places the remaining files into large_batch

Related helper scripts:

• tools/path_replacer.py
• tools/file_path_replacer.py
• tools/tiny_json.py

Those are convenience scripts for rewriting existing JSON files when output locations change.

4. Evaluate Explanations with Insertion/Deletion

There are two main ways to run evaluation.

Batch runner

Entry point: iqa/runner_args.py

This script reads tiny_batch entries from a batch JSON file, reconstructs the corresponding image path, and calls iqa/run_iqa_test.py for each mask.

Example:

python iqa/runner_args.py \
  --json_file /abs/path/to/batches.json \
  --outputs_dir /abs/path/to/csv_outputs \
  --img_dir /abs/path/to/ILSVRC2012_img_val_subset

Generated files:

• <outputs_dir>/<model>_<explanation>_deletion.csv
• <outputs_dir>/<model>_<explanation>_insertion.csv

Single image + mask evaluation

Entry point: iqa/run_iqa_test.py

Example:

python iqa/run_iqa_test.py \
  --model_name resnet50 \
  --csv_file_del /abs/path/to/resnet50_deletion.csv \
  --csv_file_ins /abs/path/to/resnet50_insertion.csv \
  --image_name /abs/path/to/129/ILSVRC2012_val_00000043.JPEG \
  --sal /abs/path/to/rise_outputs/resnet50/129/ILSVRC2012_val_00000043.npy

Supported models in run_iqa_test.py:

• alexnet
• vgg16
• resnet50
• densenet121
• googlenet
• mobilenet_v2
• mnasnet1_0
• efficientnet_b0

Important caveats:

• iqa/runner.py is an older runner with hardcoded paths; prefer iqa/runner_args.py
• iqa/evaluation_iqa.py still contains experiment-specific assumptions, including a hardcoded target class and commented-out IQA metric lines
• as written, the evaluation pipeline is most reliable for the probability trajectory columns in the CSV output; review iqa/evaluation_iqa.py before using the full IQA columns in a paper or report

5. Generate Noisy Variants of Existing Masks

Files:

• noisy_mask_gen/generator.py
• noisy_mask_gen/noises.py

Available transforms:

• Hist_noise
• Perturb_noise
• Entropy_noise
• Sploch
• Noise

What generator.py does:

• walks every .npy file under base_input_dir
• applies each transform
• recreates the original directory structure under a corresponding output root

Before running it, edit base_input_dir in noisy_mask_gen/generator.py.

Then run:

python noisy_mask_gen/generator.py

This is useful when you want stress-test variants of an explanation set without regenerating the original saliency maps.

6. Export Predictions and Analyze Confidence

Save predictions for an ImageNet-style validation set

Entry point: utils/predictions.py

python utils/predictions.py \
  --data_dir /abs/path/to/ILSVRC2012_img_val_subset \
  --output_csv utils/resnet50_val_results.csv

The output CSV includes:

• image path
• ground-truth class
• top-1 prediction and probability
• second-ranked class and probability
• whether the prediction is correct
• ground-truth probability

Plot summary statistics

Entry point: utils/statistics.py

python utils/statistics.py \
  --csv utils/resnet50_val_results.csv \
  --output_prefix utils/resnet_analysis

This generates:

• <output_prefix>_top1_probability.png
• <output_prefix>_top2_probability.png

7. Visualize Heatmaps and Overlays

Generate labeled 2 x 2 plots

Entry point: rise/shape_images.py

This script creates plots containing:

1. the original image
2. the top-prediction overlay
3. the second-prediction overlay
4. the ground-truth overlay

Important format requirement:

• this script expects each .npy mask file to contain a full class stack, typically shaped like [1000, 224, 224]
• it is not directly compatible with the single 2D masks written by rise/riser.py and rise/shaper.py
• the CSV image_path column is expected to be relative to --img-root

Generic example:

python rise/shape_images.py \
  --csv /abs/path/to/predictions_with_relative_paths.csv \
  --img-root /abs/path/to/images \
  --mask-root /abs/path/to/multi_class_masks \
  --synset rise/synset_words.txt \
  --out-root /abs/path/to/overlay_plots

Batch side-by-side overlays

Entry points:

• visuals/save_visuals.py
• visuals/save_visuals2.py

These scripts are convenient for ad hoc figure generation, but both currently use hardcoded configuration variables at the bottom of the file. Update those paths before running them.

Practical Tips

• Run scripts from the repository root unless you intentionally want outputs relative to another working directory.
• Expect torchvision warnings about pretrained=True; the repository uses the older API, which still works but is deprecated in recent torchvision releases.
• Large runs are easiest to manage if you first edit the hardcoded paths, then keep all outputs under one experiment root such as results/, csvs/, and plots/.
• If a script silently skips files, check that class directories are numeric and that mask filenames match image basenames exactly.

Known Limitations

• There is no top-level requirements.txt; dependency installation is currently manual.
• Several scripts are exploratory or one-off experiment runners rather than polished CLI tools.
• The repository mixes reusable scripts with notebooks and saved artifacts, so not every file is part of a single clean pipeline.
• Some scripts assume ImageNet class count 1000 and image size 224 x 224 directly in code.
• The evaluation code should be reviewed before publication-grade use because parts of it are still hardcoded for specific experiments.

Suggested First Run

If you want the shortest path to a reproducible test run:

1. Install the environment and make sure PyTorch can see your GPU.
2. Prepare a small ImageNet-style subset with a few numeric class folders.
3. Edit the mask-cache path in rise/riser.py or rise/shaper.py.
4. Generate masks with one of those scripts.
5. Build a batches.json file with tools/batching.py.
6. Run iqa/runner_args.py to produce deletion and insertion CSVs.
7. Use visuals/save_visuals.py to inspect the saved 2D masks visually.

That gives you the complete generate -> batch -> evaluate -> visualize loop used by the rest of the repository.