Merge branch 'main' of https://github.com/NUS-HPC-AI-Lab/DD-Ranking

Author: lizekai-richard

.gitignore

@@ -0,0 +1,10 @@
+*.egg
+*.eggs
+*.eggs-info
+*.eggs/
+eggs-info/*
+dd_ranking.eggs-info/*
+PKG-INFO
+build/
+*.pyc
+*.txt

README.md

@@ -13,7 +13,7 @@
 Fair and benchmark for dataset distillation.
 </h3> -->
 <p align="center">
-| <a href="https://nus-hpc-ai-lab.github.io/DD-Ranking/"><b>Documentation</b></a> | <a href="https://huggingface.co/spaces/Soptq/DD-Ranking"><b>Leaderboard</b></a> | <b>Paper </b> (Coming Soon) | <a href=""><b>Twitter/X</b></a> | <a href=""><b>Developer Slack</b></a> |
+| <a href="https://nus-hpc-ai-lab.github.io/DD-Ranking/"><b>Documentation</b></a> | <a href="https://huggingface.co/spaces/Soptq/DD-Ranking"><b>Leaderboard</b></a> | <a href=""><b>Paper </b> (Coming Soon)</a> | <a href=""><b>Twitter/X</b> (Coming Soon)</a> | <a href=""><b>Developer Slack</b> (Coming Soon)</a> |
 </p>
 
 
@@ -35,7 +35,7 @@ Fair and benchmark for dataset distillation.
 
 <details>
 <summary>Unfold to see more details.</summary>
-
+<br>
 Dataset Distillation (DD) aims to condense a large dataset into a much smaller one, which allows a model to achieve comparable performance after training on it. DD has gained extensive attention since it was proposed. With some foundational methods such as DC, DM, and MTT, various works have further pushed this area to a new standard with their novel designs.
 
 ![history](./static/history.png)
@@ -58,13 +58,16 @@ Motivated by this, we propose DD-Ranking, a new benchmark for DD evaluation. DD-
 
 ## About
 
+<details>
+<summary>Unfold to see more details.</summary>
+<br>
 DD-Ranking (DD, *i.e.*, Dataset Distillation) is an integrated and easy-to-use benchmark for dataset distillation. It aims to provide a fair evaluation scheme for DD methods that can decouple the impacts from knowledge distillation and data augmentation to reflect the real informativeness of the distilled data.
 
 <!-- Hard label is tested -->
 <!-- Keep the same compression ratio, comparing with random selection -->
 **Performance benchmark**
 
-<span style="color: #ffff00;">[To Verify]：</span>Revisit the original goal of dataset distillation: 
+Revisit the original goal of dataset distillation: 
 > The idea is to synthesize a small number of data points that do not need to come from the correct data distribution, but will, when given to the learning algorithm as training data, approximate the model trained on the original data.
 >
 
@@ -85,21 +88,32 @@ $$\text{IOR}/\text{HLR} = \frac{(\text{Acc.}{\text{syn-any}}-\text{Acc.}{\text{r
 DD-Ranking is integrated with:
 <!-- Uniform Fair Labels: loss on soft label -->
 <!-- Data Aug. -->
-- <span style="color: #ffff00;">[To Verify]：</span>Multiple [strategies](https://github.com/NUS-HPC-AI-Lab/DD-Ranking/tree/main/dd_ranking/loss) of using soft labels;
-- <span style="color: #ffff00;">[To Verify]：</span>Data augmentation, reconsidered as [optional tricks](https://github.com/NUS-HPC-AI-Lab/DD-Ranking/tree/main/dd_ranking/aug) in DD;
-- <span style="color: #ffff00;">[To Verify]：</span>Commonly used [model architectures](https://github.com/NUS-HPC-AI-Lab/DD-Ranking/blob/main/dd_ranking/utils/networks.py) in DD.
-<span style="color: #ffff00;">[To Verify]：</span> A new ranking on representative DD methods.
+- Multiple [strategies](https://github.com/NUS-HPC-AI-Lab/DD-Ranking/tree/main/dd_ranking/loss) of using soft labels;
+- Data augmentation, reconsidered as [optional tricks](https://github.com/NUS-HPC-AI-Lab/DD-Ranking/tree/main/dd_ranking/aug) in DD;
+- Commonly used [model architectures](https://github.com/NUS-HPC-AI-Lab/DD-Ranking/blob/main/dd_ranking/utils/networks.py) in DD.
+ A new ranking on representative DD methods.
 
 DD-Ranking is flexible and easy to use, supported by:
 <!-- Defualt configs: Customized configs -->
 <!-- Integrated classes: 1) Optimizer and etc.; 2) random selection tests (additionally, w/ or w/o hard labels)-->
-- <span style="color: #ffff00;">[To Verify]：</span>Extensive configs provided;
-- <span style="color: #ffff00;">[To Verify]：</span>Cutomized configs;
-- <span style="color: #ffff00;">[To Verify]：</span>Testing and training framework with integrated metrics.
+- Extensive configs provided;
+- Cutomized configs;
+- Testing and training framework with integrated metrics.
+
+</details>
+
+## Overview
+Included datasets and methods (hard/soft label).
+|Dataset|Hard Label|Soft Label|
+|:-|:-|:-|
+|CIFAR-10|DC|DATM|
+|CIFAR-100|DSA|SRe2L|
+|TinyImageNet|DM|RDED|
+||MTT|D4M|
 
 ## Coming Soon
-<span style="color: #ffff00;">[To Verify]：</span>Rank on different data augmentation methods.
-<span style="color: #ffff00;">[To Verify]：</span>Rank on different data augmentation methods.
+Rank on different data augmentation methods.
+Rank on different data augmentation methods.
 ## Tutorial
 
 Install DD-Ranking with `pip` or from [source](https://github.com/NUS-HPC-AI-Lab/DD-Ranking/tree/main):
@@ -208,7 +222,7 @@ The following results will be returned to you:
 - [Supported Models]() -->
 
 ## Contributing
-ß
+
 <!-- Only PR for the 1st version of DD-Ranking -->
 Feel free to submit grades to update the DD-Ranking list. We welcome and value any contributions and collaborations.
 Please check out [CONTRIBUTING.md](./CONTRIBUTING.md) for how to get involved.

configs/__init__.py

@@ -1 +1,5 @@
-
+import dd_ranking.aug as aug
+import dd_ranking.config as config
+import dd_ranking.loss as loss
+import dd_ranking.metrics as metrics
+import dd_ranking.utils as utils

dd_ranking/__init__.py

@@ -12,7 +12,7 @@
 from dd_ranking.utils import build_model, get_pretrained_model_path
 from dd_ranking.utils import TensorDataset, get_random_images, get_dataset, save_results
 from dd_ranking.utils import set_seed, train_one_epoch, validate, get_optimizer, get_lr_scheduler
-from dd_ranking.aug import DSA_Augmentation, ZCA_Whitening_Augmentation, Mixup_Augmentation, Cutmix_Augmentation
+from dd_ranking.aug import DSAugmentation, MixupAugmentation, CutmixAugmentation, ZCAWhiteningAugmentation
 from dd_ranking.config import Config
 
 

dd_ranking/metrics/hard_label.py

@@ -13,7 +13,7 @@
 from dd_ranking.utils import TensorDataset, get_random_images, get_dataset, save_results
 from dd_ranking.utils import set_seed, train_one_epoch, validate, get_optimizer, get_lr_scheduler
 from dd_ranking.loss import SoftCrossEntropyLoss, KLDivergenceLoss
-from dd_ranking.aug import DSA_Augmentation, ZCA_Whitening_Augmentation, Mixup_Augmentation, Cutmix_Augmentation
+from dd_ranking.aug import DSAugmentation, MixupAugmentation, CutmixAugmentation, ZCAWhiteningAugmentation
 from dd_ranking.config import Config
 
 

dd_ranking/metrics/soft_label.py

@@ -1,7 +1,9 @@
 from setuptools import setup, find_packages
 
+packages = find_packages()
+
 setup(
-    name='DD-Ranking',  # Package name
+    name='dd_ranking',  # Package name
     version='0.1.0',  # Version number
     description='DD-Ranking: A unified benchmark for dataset distillation method evaluation',
     long_description=open('README.md').read(),  # Use your README as the long description
@@ -13,8 +15,10 @@
         'repository': 'https://github.com/NUS-HPC-AI-Lab/DD-Ranking',
         'documentation': 'https://github.com/NUS-HPC-AI-Lab/DD-Ranking'
     },
-    license='MIT',  # License type
-    packages=find_packages(),  # Automatically discover submodules
+    # license='MIT',  # License type
+    include_dirs=['dd_ranking', 'configs'],
+    include_package_data=True,
+    packages=packages,  # Automatically discover submodules
     install_requires=[
         'torch',
         'numpy',