Potential bug with date sorting

[cakiki]

Sorting by date somehow reduces the number of rows in the view. The view without the sorting has 215,046 pages and this becomes an oddly round 30,000 pages when sorting. (The filtered view also starts in 2019 even though the data goes up to 2024)

https://huggingface.co/datasets/Lichess/chess960-chess-games/viewer/default/train?sort%5Bcolumn%5D=UTCDate&sort%5Bdirection%5D=desc

[severo]

The sort relies on the duckdb file (only the first 5 GB of data), as (not) mentioned here: https://huggingface.co/docs/hub/en/datasets-viewer#large-scale-datasets.

I'll add the info about "sorting" to the sentence

the Dataset Viewer only shows the first 5GB, and filtering and search are enabled on these first 5GB.

The info is shown in the UI too:

https://huggingface.co/datasets/Lichess/chess960-chess-games/viewer/default/train?sort%5Bcolumn%5D=UTCDate&sort%5Bdirection%5D=desc

[severo]

I opened huggingface/hub-docs#1577

[cakiki]

Ah! That makes sense. So in the example I used, the first 5GB are the rows from 2013 to 2019 which is why the most recent row is 2019.

Preemptively closing this then 😸 Thank you!