Skip to content

Commit a417a0a

Browse files
Update readme and doc
1 parent b13c1eb commit a417a0a

File tree

2 files changed

+8
-6
lines changed

2 files changed

+8
-6
lines changed

README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -84,9 +84,11 @@ $\text{Acc.}$ is the accuracy of models trained on different samples. Samples' m
8484
- $\text{syn-any}$: Synthetic dataset with personalized evaluation methods (hard or soft labels);
8585
- $\text{rdm-any}$: Randomly selected dataset (under the same compression ratio) with the same personalized evaluation methods.
8686

87-
<!-- To rank different methods, we combine the above two metrics as follows:
88-
89-
$$\text{IOR}/\text{HLR} = \frac{(\text{Acc.}{\text{syn-any}}-\text{Acc.}{\text{rdm-any}})}{(\text{Acc.}{\text{full-hard}}-\text{Acc.}{\text{syn-hard}})}$$ -->
87+
DD-Ranking uses a weight sum of $-\text{IOR}$ and $\text{HLR}$ to rank different methods:
88+
$$
89+
\text{Rank\_Score} = w_1 (-\text{IOR}) + w_2 \text{HLR}, \quad w_1 + w_2 = 1
90+
$$
91+
By default, we set $w_1 = w_2 = 0.5$ on the leaderboard, meaning that both $\text{IOR}$ and $\text{HLR}$ are equally important. Users can adjust the weights to emphasize one aspect on the leaderboard.
9092

9193
</details>
9294

doc/introduction.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ The evaluation method for DD-Ranking is grounded in the essence of dataset disti
5656
- \\(\text{syn-any}\\): Synthetic dataset with personalized evaluation methods (hard or soft labels);
5757
- \\(\text{rdm-any}\\): Randomly selected dataset (under the same compression ratio) with the same personalized evaluation methods.
5858

59-
<!-- To rank different methods, we combine the above two metrics as DD-Ranking Score:
60-
61-
\\[\text{DD-Ranking Score} = \frac{\text{IOR}}{\text{HLR}} = \frac{(\text{Acc.} \text{syn-any}-\text{Acc.} \text{rdm-any})}{(\text{Acc.} \text{full-hard}-\text{Acc.} \text{syn-hard})}\\] -->
59+
DD-Ranking uses a weight sum of \\(-\text{IOR}\\) and \\(\text{HLR}\\) to rank different methods:
60+
\\[\text{Rank\_Score} = w_1 (-\text{IOR}) + w_2 \text{HLR}, \quad w_1 + w_2 = 1\\]
61+
By default, we set \\(w_1 = w_2 = 0.5\\) on the leaderboard, meaning that both \\(\text{IOR}\\) and \\(\text{HLR}\\) are equally important. Users can adjust the weights to emphasize one aspect on the leaderboard.
6262

0 commit comments

Comments
 (0)