Skip to content

Commit 11be0d6

Browse files
lhoestqdavanstrien
andauthored
Update datasets-download-stats.md (#1466)
* Update datasets-download-stats.md * Update docs/hub/datasets-download-stats.md Co-authored-by: Daniel van Strien <[email protected]> * Update docs/hub/datasets-download-stats.md Co-authored-by: Daniel van Strien <[email protected]> --------- Co-authored-by: Daniel van Strien <[email protected]>
1 parent 3402ebc commit 11be0d6

File tree

1 file changed

+7
-3
lines changed

1 file changed

+7
-3
lines changed

docs/hub/datasets-download-stats.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,11 @@
22

33
## How are download stats generated for datasets?
44

5-
The Hub provides download stats for all datasets loadable via the `datasets` library. To determine the number of downloads, the Hub counts every time `load_dataset` is called in Python, excluding Hugging Face's CI tooling on GitHub. No information is sent from the user, and no additional calls are made for this. The count is done server-side as we serve files for downloads. This means that:
5+
Counting the number of downloads for datasets is not a trivial task, as a single dataset repository might contain multiple files, from multiple subsets and splits (e.g. train/validation/test) and sometimes with many files in a single split. To solve this issue and avoid counting one person's download multiple times, we treat all files downloaded by a user within a 5-minute window as a single dataset download. This counting happens automatically on our servers when files are downloaded (through GET or HEAD requests), with no need to collect any user information or make additional calls.
66

7-
* The download count is the same regardless of whether the data is directly stored on the Hub repo or if the repository has a [script](/docs/datasets/dataset_script) to load the data from an external source.
8-
* If a user manually downloads the data using tools like `wget` or the Hub's user interface (UI), those downloads will not be included in the download count.
7+
## Before Setpember 2024
8+
9+
The Hub used to provide download stats only for the datasets loadable via the `datasets` library. To determine the number of downloads, the Hub previously counted every time `load_dataset` was called in Python, excluding Hugging Face's CI tooling on GitHub. No information was sent from the user, and no additional calls were made for this. The count was done server-side as we served files for downloads. This means that:
10+
11+
* The download count was the same regardless of whether the data is directly stored on the Hub repo or if the repository has a [script](/docs/datasets/dataset_script) to load the data from an external source.
12+
* If a user manually downloaded the data using tools like `wget` or the Hub's user interface (UI), those downloads were not included in the download count.

0 commit comments

Comments
 (0)