Skip to content

Commit 4771278

Browse files
Update image
1 parent c0b94c7 commit 4771278

File tree

5 files changed

+7
-5
lines changed

5 files changed

+7
-5
lines changed

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@
33
<p align="center">
44
<picture>
55
<!-- Dark theme logo -->
6-
<source media="(prefers-color-scheme: dark)" srcset="XX.png">
6+
<source media="(prefers-color-scheme: dark)" srcset="static/logo.png">
77
<!-- Light theme logo -->
8-
<img alt="vLLM" src="XX.png" width=55%>
8+
<img alt="DD-Ranking" src="static/logo.png" width=55%>
99
</picture>
1010
</p>
1111

doc/introduction.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,14 +11,16 @@ Dataset Distillation (DD) aims to condense a large dataset into a much smaller o
1111
Notebaly, more and more methods are transitting from "hard label" to "soft label" in dataset distillation, especially during evaluation. **Hard labels** are categorical, having the same format of the real dataset. **Soft labels** are distributions, typically generated by a pre-trained teacher model.
1212
Recently, Deng et al., pointed out that "a label is worth a thousand images". They showed analytically that soft labels are exetremely useful for accuracy improvement.
1313

14-
However, since the essence of soft labels is **knowledge distillation**, we want to ask a question: **Can the test accuracy of the model trained on distilled data reflect the real informativeness of the distilled data?**
14+
However, since the essence of soft labels is **knowledge distillation**, we find that when applying the same evaluation method to randomly selected data, the test accuracy also improves significantly (see the figure above).
1515

16-
Specifically, we have discoverd unfairness of using only test accuracy to demonstrate one's performance from the following three aspects:
16+
This makes us wonder: **Can the test accuracy of the model trained on distilled data reflect the real informativeness of the distilled data?**
17+
18+
Additionally, we have discoverd unfairness of using only test accuracy to demonstrate one's performance from the following three aspects:
1719
1. Results of using hard and soft labels are not directly comparable since soft labels introduce teacher knowledge.
1820
2. Strategies of using soft labels are diverse. For instance, different objective functions are used during evaluation, such as soft Cross-Entropy and Kullback–Leibler divergence. Also, one image may be mapped to one or multiple soft labels.
1921
3. Different data augmentations are used during evaluation.
2022

21-
Motivated by this, we propose DD-Ranking, a new benchmark for DD evaluation. DD-Ranking provides a fair evaluation scheme for DD methods that can decouple the impacts from knowledge distillation and data augmentation to reflect the real informativeness of the distilled data.
23+
Motivated by this, we propose DD-Ranking, a new benchmark for DD evaluation. DD-Ranking provides a fair evaluation scheme for DD methods, and can decouple the impacts from knowledge distillation and data augmentation to reflect the real informativeness of the distilled data.
2224

2325
## Features
2426

doc/static/history.png

22.6 KB
Loading

static/history.png

22.6 KB
Loading

static/logo.png

35.6 KB
Loading

0 commit comments

Comments
 (0)