Add _generate_shards

[lhoestq]

Useful to list a dataset's shards:

from datasets import load_dataset_builder, StreamingDownloadManager

dlm = StreamingDownloadManager()

def get_shards(dataset_name, *args, **kwargs):
    b = load_dataset_builder(dataset_name, *args, **kwargs)
    splits = b._split_generators(dlm)
    return list(b._generate_shards(**splits[0].gen_kwargs))

print(get_shards("username/dataset_name"))
# ['hf://datasets/...', ...]

I'll use this in combination with #7897 in the Dataset Viewer for the and API endpoint that does {dataset, config, split, offset, limit} -> [{fileUri, offset, limit}]. This will be useful to edit datasets since your can get a row's location inside a dataset cc @cfahlgren1

This will be similar to huggingface/dataset-viewer#3276 but works for any dataset format: csv, json, webdataset, images etc.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.