Squash the commits in the refs/convert/parquet and refs/convert/duckdb branches

[severo]

See huggingface/huggingface_hub#1639 (comment)

[severo]

When implemented, revert huggingface/hub-docs#1585

[severo]

For PRO users and Enterprise Hub orgs, the dataset viewer is enabled on private datasets, and the storage limits apply. Discussed here: #2384 (comment)

It's a good idea to reduce the storage automatically, instead of relying on huggingface/hub-docs#1585