From 1f0e69d199d2b1bbeb64a6ebf1bb29303161bae9 Mon Sep 17 00:00:00 2001 From: Kevin Wang Date: Wed, 3 Sep 2025 13:55:52 -0700 Subject: [PATCH 1/7] Add Daft to list of integrated libraries for datasets --- docs/hub/datasets-adding.md | 2 +- docs/hub/datasets-daft.md | 79 ++++++++++++++++++++++++++++++++++ docs/hub/datasets-libraries.md | 3 +- 3 files changed, 82 insertions(+), 2 deletions(-) create mode 100644 docs/hub/datasets-daft.md diff --git a/docs/hub/datasets-adding.md b/docs/hub/datasets-adding.md index 7bf34b2a4..19ccef83f 100644 --- a/docs/hub/datasets-adding.md +++ b/docs/hub/datasets-adding.md @@ -67,7 +67,7 @@ The rich features set in the `huggingface_hub` library allows you to manage repo ## Using other libraries -Some libraries like [🤗 Datasets](/docs/datasets/index), [Pandas](https://pandas.pydata.org/), [Polars](https://pola.rs), [Dask](https://www.dask.org/) or [DuckDB](https://duckdb.org/) can upload files to the Hub. +Some libraries like [🤗 Datasets](/docs/datasets/index), [Pandas](https://pandas.pydata.org/), [Polars](https://pola.rs), [Dask](https://www.dask.org/), [DuckDB](https://duckdb.org/), or [Daft](https://daft.ai/) can upload files to the Hub. See the list of [Libraries supported by the Datasets Hub](./datasets-libraries) for more information. ## Using Git diff --git a/docs/hub/datasets-daft.md b/docs/hub/datasets-daft.md new file mode 100644 index 000000000..4c7f0ced2 --- /dev/null +++ b/docs/hub/datasets-daft.md @@ -0,0 +1,79 @@ +# Daft + +[Daft](https://daft.ai/) is a high-performance data engine providing simple and reliable data processing for any modality and scale. Daft has native support for reading from and writing to Hugging Face datasets. + +
+ +
+ + +## Getting Started + +To get started, pip install `daft` with the `huggingface` feature: + +```bash +pip install 'daft[hugggingface]' +``` + +## Read + +Daft is able to read datasets directly from Hugging Face using the [`daft.read_huggingface()`](https://docs.daft.ai/en/stable/api/io/#daft.read_huggingface) function or via the `hf://datasets/` protocol. + +### Reading an Entire Dataset + +Using [`daft.read_huggingface()`](https://docs.daft.ai/en/stable/api/io/#daft.read_huggingface), you can easily read a Hugging Face dataset. + + +```python +import daft + +df = daft.read_huggingface("username/dataset_name") +``` + +This will read the entire dataset into a DataFrame. + +### Reading Specific Files + +Not only can you read entire datasets, but you can also read individual files from a dataset. Using a read function that takes in a path (such as [`daft.read_parquet()`](https://docs.daft.ai/en/stable/api/io/#daft.read_parquet), [`daft.read_csv()`](https://docs.daft.ai/en/stable/api/io/#daft.read_csv), or [`daft.read_json()`](https://docs.daft.ai/en/stable/api/io/#daft.read_json)), specify a Hugging Face dataset path via the `hf://datasets/` prefix: + +```python +import daft + +# read a specific Parquet file +df = daft.read_parquet("hf://datasets/username/dataset_name/file_name.parquet") + +# or a csv file +df = daft.read_csv("hf://datasets/username/dataset_name/file_name.csv") + +# or a set of Parquet files using a glob pattern +df = daft.read_parquet("hf://datasets/username/dataset_name/**/*.parquet") +``` + +## Write + +Daft is able to write Parquet files to Hugging Face datasets using [`daft.DataFrame.write_huggingface`](https://docs.daft.ai/en/stable/api/dataframe/#daft.DataFrame.write_deltalake). Daft supports [Content-Defined Chunking](https://huggingface.co/blog/parquet-cdc) and [Xet](https://huggingface.co/blog/xet-on-the-hub) for faster, deduplicated writes. + +Basic usage: + +```python +import daft + +df: daft.DataFrame = ... + +df.write_huggingface("username/dataset_name") +``` + +See the [`DataFrame.write_huggingface`](https://docs.daft.ai/en/stable/api/dataframe/#daft.DataFrame.write_deltalake) API page for more info. + +## Authentication + +The `token` parameter in [`daft.io.HuggingFaceConfig`](https://docs.daft.ai/en/stable/api/config/#daft.io.HuggingFaceConfig) can be used to specify a Hugging Face access token for requests that require authentication (e.g. reading private datasets or writing to a dataset). + +Example of reading a dataset with a specified token: + +```python +from daft.io import IOConfig, HuggingFaceConfig + +io_config = IOConfig(hf=HuggingFaceConfig(token="your_token")) +df = daft.read_parquet("hf://datasets/username/dataset_name", io_config=io_config) +``` diff --git a/docs/hub/datasets-libraries.md b/docs/hub/datasets-libraries.md index aa39757d1..2d38c5d36 100644 --- a/docs/hub/datasets-libraries.md +++ b/docs/hub/datasets-libraries.md @@ -9,6 +9,7 @@ The table below summarizes the supported libraries and their level of integratio | Library | Description | Download from Hub | Push to Hub | | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ----------------- | ----------- | | [Argilla](./datasets-argilla) | Collaboration tool for AI engineers and domain experts that value high quality data. | ✅ | ✅ | +| [Daft](./datasets-daft) | Data engine for large scale, multimodal data processing with a Python-native interface. | ✅ | ✅ | | [Dask](./datasets-dask) | Parallel and distributed computing library that scales the existing Python and PyData ecosystem. | ✅ | ✅ | | [Datasets](./datasets-usage) | 🤗 Datasets is a library for accessing and sharing datasets for Audio, Computer Vision, and Natural Language Processing (NLP). | ✅ | ✅ | | [Distilabel](./datasets-distilabel) | The framework for synthetic data generation and AI feedback. | ✅ | ✅ | @@ -87,7 +88,7 @@ Examples of this kind of integration: #### Rely on an existing libraries integration with the Hub -Polars, Pandas, Dask, Spark and DuckDB all can write to a Hugging Face Hub repository. See [datasets libraries](https://huggingface.co/docs/hub/datasets-libraries) for more details. +Polars, Pandas, Dask, Spark, DuckDB, and Daft all can write to a Hugging Face Hub repository. See [datasets libraries](https://huggingface.co/docs/hub/datasets-libraries) for more details. If you are already using one of these libraries in your code, adding the ability to push to the Hub is straightforward. For example, if you have a synthetic data generation library that can return a Pandas DataFrame, here is the code you would need to write to the Hub: From a16a995280cde670f775b3c05f64486a657fa7c7 Mon Sep 17 00:00:00 2001 From: HuggingFaceInfra <148469759+HuggingFaceInfra@users.noreply.github.com> Date: Wed, 3 Sep 2025 09:21:16 +0200 Subject: [PATCH 2/7] Update Inference Providers documentation (automated) (#1891) Co-authored-by: Wauplin <11801849+Wauplin@users.noreply.github.com> --- docs/inference-providers/providers/replicate.md | 2 +- docs/inference-providers/tasks/image-to-image.md | 2 +- scripts/inference-providers/package.json | 2 +- scripts/inference-providers/pnpm-lock.yaml | 12 ++++++------ 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/inference-providers/providers/replicate.md b/docs/inference-providers/providers/replicate.md index 6487d1fd7..4e20490ec 100644 --- a/docs/inference-providers/providers/replicate.md +++ b/docs/inference-providers/providers/replicate.md @@ -50,7 +50,7 @@ Find out more about Image To Image [here](../tasks/image_to_image). diff --git a/docs/inference-providers/tasks/image-to-image.md b/docs/inference-providers/tasks/image-to-image.md index 23f84dcac..392614205 100644 --- a/docs/inference-providers/tasks/image-to-image.md +++ b/docs/inference-providers/tasks/image-to-image.md @@ -39,7 +39,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/scripts/inference-providers/package.json b/scripts/inference-providers/package.json index 4bf51ece9..ae65464ca 100644 --- a/scripts/inference-providers/package.json +++ b/scripts/inference-providers/package.json @@ -15,7 +15,7 @@ "license": "ISC", "dependencies": { "@huggingface/inference": "^4.7.1", - "@huggingface/tasks": "^0.19.37", + "@huggingface/tasks": "^0.19.42", "@types/node": "^22.5.0", "handlebars": "^4.7.8", "node": "^20.17.0", diff --git a/scripts/inference-providers/pnpm-lock.yaml b/scripts/inference-providers/pnpm-lock.yaml index 0fe303ebe..00f2c8696 100644 --- a/scripts/inference-providers/pnpm-lock.yaml +++ b/scripts/inference-providers/pnpm-lock.yaml @@ -12,8 +12,8 @@ importers: specifier: ^4.7.1 version: 4.7.1 '@huggingface/tasks': - specifier: ^0.19.37 - version: 0.19.37 + specifier: ^0.19.42 + version: 0.19.42 '@types/node': specifier: ^22.5.0 version: 22.5.0 @@ -197,8 +197,8 @@ packages: resolution: {integrity: sha512-yUZLld4lrM9iFxHCwFQ7D1HW2MWMwSbeB7WzWqFYDWK+rEb+WldkLdAJxUPOmgICMHZLzZGVcVjFh3w/YGubng==} engines: {node: '>=18'} - '@huggingface/tasks@0.19.37': - resolution: {integrity: sha512-Te1VB1tB1HoLfTGluCwy8sLO90YV+uNOAFktQ1h7jKas4TlHT/7SlfwFaDJFTV8lN7qCw2nDB+7PRkKzwIb/hg==} + '@huggingface/tasks@0.19.42': + resolution: {integrity: sha512-+FTsNxQA6U8s+cVRzBONXqaQkqB7BrNCONGVk6HdYHMIugEEBo8m9WYZrtr8z0zQPJqqqi+yv2svupE15FCing==} '@jridgewell/resolve-uri@3.1.2': resolution: {integrity: sha512-bRISgCIjP20/tbWSPWMEi54QVPRZExkuD9lJL+UIxUKtwVJA8wW1Trb1jMs1RFXo1CBTNZ/5hpC9QvmKWdopKw==} @@ -418,11 +418,11 @@ snapshots: '@huggingface/inference@4.7.1': dependencies: '@huggingface/jinja': 0.5.1 - '@huggingface/tasks': 0.19.37 + '@huggingface/tasks': 0.19.42 '@huggingface/jinja@0.5.1': {} - '@huggingface/tasks@0.19.37': {} + '@huggingface/tasks@0.19.42': {} '@jridgewell/resolve-uri@3.1.2': {} From eec2edb0f9416fb175618d0d7cfed8b4dff62568 Mon Sep 17 00:00:00 2001 From: burtenshaw Date: Wed, 3 Sep 2025 21:15:48 +0200 Subject: [PATCH 3/7] [inference providers] reorder menu (#1876) * reorder menu in inference providers * move api reference up * only move tasks up and leave api reference below providers * use Inference Tasks and Hub API --- docs/inference-providers/_toctree.yml | 100 +++++++++++++------------- 1 file changed, 49 insertions(+), 51 deletions(-) diff --git a/docs/inference-providers/_toctree.yml b/docs/inference-providers/_toctree.yml index 3feaac75f..29369a606 100644 --- a/docs/inference-providers/_toctree.yml +++ b/docs/inference-providers/_toctree.yml @@ -6,42 +6,9 @@ title: Pricing and Billing - local: hub-integration title: Hub integration - - local: register-as-a-provider - title: Register as an Inference Provider - local: security title: Security -- title: Providers - sections: - - local: providers/cerebras - title: Cerebras - - local: providers/cohere - title: Cohere - - local: providers/fal-ai - title: Fal AI - - local: providers/featherless-ai - title: Featherless AI - - local: providers/fireworks-ai - title: Fireworks - - local: providers/groq - title: Groq - - local: providers/hyperbolic - title: Hyperbolic - - local: providers/hf-inference - title: HF Inference - - local: providers/nebius - title: Nebius - - local: providers/novita - title: Novita - - local: providers/nscale - title: Nscale - - local: providers/replicate - title: Replicate - - local: providers/sambanova - title: SambaNova - - local: providers/together - title: Together - - title: Guides sections: - local: guides/first-api-call @@ -57,25 +24,19 @@ - local: guides/image-editor title: Build an Image Editor - -- title: API Reference +- local: tasks/index + title: Inference Tasks sections: - - local: tasks/index - title: Index - - local: hub-api - title: Hub API - - title: Popular Tasks - sections: - - local: tasks/chat-completion - title: Chat Completion - - local: tasks/feature-extraction - title: Feature Extraction - - local: tasks/text-to-image - title: Text to Image - - local: tasks/text-to-video - title: Text to Video + - local: tasks/chat-completion + title: Chat Completion + - local: tasks/feature-extraction + title: Feature Extraction + - local: tasks/text-to-image + title: Text to Image + - local: tasks/text-to-video + title: Text to Video - title: Other Tasks - isExpanded: false + isExpanded: False sections: - local: tasks/audio-classification title: Audio Classification @@ -108,4 +69,41 @@ - local: tasks/translation title: Translation - local: tasks/zero-shot-classification - title: Zero Shot Classification \ No newline at end of file + title: Zero Shot Classification + +- title: Providers + sections: + - local: providers/cerebras + title: Cerebras + - local: providers/cohere + title: Cohere + - local: providers/fal-ai + title: Fal AI + - local: providers/featherless-ai + title: Featherless AI + - local: providers/fireworks-ai + title: Fireworks + - local: providers/groq + title: Groq + - local: providers/hyperbolic + title: Hyperbolic + - local: providers/hf-inference + title: HF Inference + - local: providers/nebius + title: Nebius + - local: providers/novita + title: Novita + - local: providers/nscale + title: Nscale + - local: providers/replicate + title: Replicate + - local: providers/sambanova + title: SambaNova + - local: providers/together + title: Together + +- local: hub-api + title: Hub API + +- local: register-as-a-provider + title: Register as an Inference Provider \ No newline at end of file From 9d241ae0340cfedd49bbf212d1ae3f9bb60a1cf1 Mon Sep 17 00:00:00 2001 From: HuggingFaceInfra <148469759+HuggingFaceInfra@users.noreply.github.com> Date: Thu, 4 Sep 2025 09:03:47 +0200 Subject: [PATCH 4/7] Update Inference Providers documentation (automated) (#1893) Co-authored-by: Wauplin <11801849+Wauplin@users.noreply.github.com> --- .../providers/featherless-ai.md | 4 +- .../providers/hf-inference.md | 46 ++++--------------- .../providers/replicate.md | 2 +- .../tasks/chat-completion.md | 2 +- docs/inference-providers/tasks/fill-mask.md | 3 +- .../tasks/image-classification.md | 2 - .../tasks/image-segmentation.md | 2 +- .../tasks/image-to-image.md | 2 +- .../tasks/object-detection.md | 6 +-- .../tasks/question-answering.md | 4 +- .../tasks/summarization.md | 3 +- .../tasks/table-question-answering.md | 3 +- .../tasks/text-classification.md | 4 +- .../tasks/text-generation.md | 2 +- .../tasks/token-classification.md | 2 - docs/inference-providers/tasks/translation.md | 3 +- .../tasks/zero-shot-classification.md | 3 +- 17 files changed, 23 insertions(+), 70 deletions(-) diff --git a/docs/inference-providers/providers/featherless-ai.md b/docs/inference-providers/providers/featherless-ai.md index 35f31227d..2f89a71fb 100644 --- a/docs/inference-providers/providers/featherless-ai.md +++ b/docs/inference-providers/providers/featherless-ai.md @@ -52,7 +52,7 @@ Find out more about Chat Completion (LLM) [here](../tasks/chat-completion). @@ -72,6 +72,6 @@ Find out more about Text Generation [here](../tasks/text_generation). diff --git a/docs/inference-providers/providers/hf-inference.md b/docs/inference-providers/providers/hf-inference.md index 7fb0948e6..b458eac86 100644 --- a/docs/inference-providers/providers/hf-inference.md +++ b/docs/inference-providers/providers/hf-inference.md @@ -57,16 +57,6 @@ Find out more about Automatic Speech Recognition [here](../tasks/automatic_speec /> -### Chat Completion (LLM) - -Find out more about Chat Completion (LLM) [here](../tasks/chat-completion). - - - - ### Feature Extraction Find out more about Feature Extraction [here](../tasks/feature_extraction). @@ -83,7 +73,7 @@ Find out more about Fill Mask [here](../tasks/fill_mask). @@ -103,17 +93,7 @@ Find out more about Image Segmentation [here](../tasks/image_segmentation). - - -### Object Detection - -Find out more about Object Detection [here](../tasks/object_detection). - - @@ -123,7 +103,7 @@ Find out more about Question Answering [here](../tasks/question_answering). @@ -133,7 +113,7 @@ Find out more about Summarization [here](../tasks/summarization). @@ -143,7 +123,7 @@ Find out more about Table Question Answering [here](../tasks/table_question_answ @@ -153,17 +133,7 @@ Find out more about Text Classification [here](../tasks/text_classification). - - -### Text Generation - -Find out more about Text Generation [here](../tasks/text_generation). - - @@ -193,7 +163,7 @@ Find out more about Translation [here](../tasks/translation). @@ -203,6 +173,6 @@ Find out more about Zero Shot Classification [here](../tasks/zero_shot_classific diff --git a/docs/inference-providers/providers/replicate.md b/docs/inference-providers/providers/replicate.md index 4e20490ec..6487d1fd7 100644 --- a/docs/inference-providers/providers/replicate.md +++ b/docs/inference-providers/providers/replicate.md @@ -50,7 +50,7 @@ Find out more about Image To Image [here](../tasks/image_to_image). diff --git a/docs/inference-providers/tasks/chat-completion.md b/docs/inference-providers/tasks/chat-completion.md index f1fbb1654..d7d546e19 100644 --- a/docs/inference-providers/tasks/chat-completion.md +++ b/docs/inference-providers/tasks/chat-completion.md @@ -64,7 +64,7 @@ The API supports: diff --git a/docs/inference-providers/tasks/fill-mask.md b/docs/inference-providers/tasks/fill-mask.md index d527ce0df..bcb432006 100644 --- a/docs/inference-providers/tasks/fill-mask.md +++ b/docs/inference-providers/tasks/fill-mask.md @@ -24,7 +24,6 @@ For more details about the `fill-mask` task, check out its [dedicated page](http ### Recommended models -- [FacebookAI/xlm-roberta-base](https://huggingface.co/FacebookAI/xlm-roberta-base): A multilingual model trained on 100 languages. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=fill-mask&sort=trending). @@ -33,7 +32,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/image-classification.md b/docs/inference-providers/tasks/image-classification.md index 3321890eb..e7115ff26 100644 --- a/docs/inference-providers/tasks/image-classification.md +++ b/docs/inference-providers/tasks/image-classification.md @@ -24,8 +24,6 @@ For more details about the `image-classification` task, check out its [dedicated ### Recommended models -- [facebook/deit-base-distilled-patch16-224](https://huggingface.co/facebook/deit-base-distilled-patch16-224): A robust image classification model. -- [facebook/convnext-large-224](https://huggingface.co/facebook/convnext-large-224): A strong image classification model. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=image-classification&sort=trending). diff --git a/docs/inference-providers/tasks/image-segmentation.md b/docs/inference-providers/tasks/image-segmentation.md index 1ceca0e68..c9d33790c 100644 --- a/docs/inference-providers/tasks/image-segmentation.md +++ b/docs/inference-providers/tasks/image-segmentation.md @@ -33,7 +33,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/image-to-image.md b/docs/inference-providers/tasks/image-to-image.md index 392614205..23f84dcac 100644 --- a/docs/inference-providers/tasks/image-to-image.md +++ b/docs/inference-providers/tasks/image-to-image.md @@ -39,7 +39,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/object-detection.md b/docs/inference-providers/tasks/object-detection.md index 3c36c4081..299a41788 100644 --- a/docs/inference-providers/tasks/object-detection.md +++ b/docs/inference-providers/tasks/object-detection.md @@ -24,17 +24,13 @@ For more details about the `object-detection` task, check out its [dedicated pag ### Recommended models -- [facebook/detr-resnet-50](https://huggingface.co/facebook/detr-resnet-50): Solid object detection model pre-trained on the COCO 2017 dataset. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=object-detection&sort=trending). ### Using the API - +There are currently no snippet examples for the **object-detection** task, as no providers support it yet. diff --git a/docs/inference-providers/tasks/question-answering.md b/docs/inference-providers/tasks/question-answering.md index 2f1330014..3c01ff1ff 100644 --- a/docs/inference-providers/tasks/question-answering.md +++ b/docs/inference-providers/tasks/question-answering.md @@ -24,9 +24,7 @@ For more details about the `question-answering` task, check out its [dedicated p ### Recommended models -- [deepset/roberta-base-squad2](https://huggingface.co/deepset/roberta-base-squad2): A robust baseline model for most question answering domains. - [distilbert/distilbert-base-cased-distilled-squad](https://huggingface.co/distilbert/distilbert-base-cased-distilled-squad): Small yet robust model that can answer questions. -- [google/tapas-base-finetuned-wtq](https://huggingface.co/google/tapas-base-finetuned-wtq): A special model that can answer questions from tables. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=question-answering&sort=trending). @@ -35,7 +33,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/summarization.md b/docs/inference-providers/tasks/summarization.md index 6d3994406..8b948f871 100644 --- a/docs/inference-providers/tasks/summarization.md +++ b/docs/inference-providers/tasks/summarization.md @@ -24,7 +24,6 @@ For more details about the `summarization` task, check out its [dedicated page]( ### Recommended models -- [facebook/bart-large-cnn](https://huggingface.co/facebook/bart-large-cnn): A strong summarization model trained on English news articles. Excels at generating factual summaries. - [Falconsai/medical_summarization](https://huggingface.co/Falconsai/medical_summarization): A summarization model trained on medical articles. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=summarization&sort=trending). @@ -34,7 +33,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/table-question-answering.md b/docs/inference-providers/tasks/table-question-answering.md index 358287c69..820afc14f 100644 --- a/docs/inference-providers/tasks/table-question-answering.md +++ b/docs/inference-providers/tasks/table-question-answering.md @@ -24,7 +24,6 @@ For more details about the `table-question-answering` task, check out its [dedic ### Recommended models -- [google/tapas-base-finetuned-wtq](https://huggingface.co/google/tapas-base-finetuned-wtq): A robust table question answering model. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=table-question-answering&sort=trending). @@ -33,7 +32,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/text-classification.md b/docs/inference-providers/tasks/text-classification.md index 1202d5f48..7b1a73a67 100644 --- a/docs/inference-providers/tasks/text-classification.md +++ b/docs/inference-providers/tasks/text-classification.md @@ -24,8 +24,6 @@ For more details about the `text-classification` task, check out its [dedicated ### Recommended models -- [distilbert/distilbert-base-uncased-finetuned-sst-2-english](https://huggingface.co/distilbert/distilbert-base-uncased-finetuned-sst-2-english): A robust model trained for sentiment analysis. -- [ProsusAI/finbert](https://huggingface.co/ProsusAI/finbert): A sentiment analysis model specialized in financial sentiment. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=text-classification&sort=trending). @@ -34,7 +32,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/text-generation.md b/docs/inference-providers/tasks/text-generation.md index 9f9747d59..46dcc7caa 100644 --- a/docs/inference-providers/tasks/text-generation.md +++ b/docs/inference-providers/tasks/text-generation.md @@ -42,7 +42,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/token-classification.md b/docs/inference-providers/tasks/token-classification.md index dc5df09f2..b41170e6c 100644 --- a/docs/inference-providers/tasks/token-classification.md +++ b/docs/inference-providers/tasks/token-classification.md @@ -24,8 +24,6 @@ For more details about the `token-classification` task, check out its [dedicated ### Recommended models -- [dslim/bert-base-NER](https://huggingface.co/dslim/bert-base-NER): A robust performance model to identify people, locations, organizations and names of miscellaneous entities. -- [FacebookAI/xlm-roberta-large-finetuned-conll03-english](https://huggingface.co/FacebookAI/xlm-roberta-large-finetuned-conll03-english): A strong model to identify people, locations, organizations and names in multiple languages. - [blaze999/Medical-NER](https://huggingface.co/blaze999/Medical-NER): A token classification model specialized on medical entity recognition. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=token-classification&sort=trending). diff --git a/docs/inference-providers/tasks/translation.md b/docs/inference-providers/tasks/translation.md index 86a7ac19e..6366a4807 100644 --- a/docs/inference-providers/tasks/translation.md +++ b/docs/inference-providers/tasks/translation.md @@ -24,7 +24,6 @@ For more details about the `translation` task, check out its [dedicated page](ht ### Recommended models -- [google-t5/t5-base](https://huggingface.co/google-t5/t5-base): A general-purpose Transformer that can be used to translate from English to German, French, or Romanian. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=translation&sort=trending). @@ -33,7 +32,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/zero-shot-classification.md b/docs/inference-providers/tasks/zero-shot-classification.md index 1c57edfb9..c99069be0 100644 --- a/docs/inference-providers/tasks/zero-shot-classification.md +++ b/docs/inference-providers/tasks/zero-shot-classification.md @@ -24,7 +24,6 @@ For more details about the `zero-shot-classification` task, check out its [dedic ### Recommended models -- [facebook/bart-large-mnli](https://huggingface.co/facebook/bart-large-mnli): Powerful zero-shot text classification model. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=zero-shot-classification&sort=trending). @@ -33,7 +32,7 @@ Explore all available models and find the one that suits you best [here](https:/ From 5168d787088436915bc6cf3a3e27f5adc7f3f391 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Thu, 4 Sep 2025 14:40:44 +0200 Subject: [PATCH 5/7] More requirement from #1853 more visible (#1894) --- docs/inference-providers/register-as-a-provider.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/docs/inference-providers/register-as-a-provider.md b/docs/inference-providers/register-as-a-provider.md index eab44d0bc..2f62c4676 100644 --- a/docs/inference-providers/register-as-a-provider.md +++ b/docs/inference-providers/register-as-a-provider.md @@ -2,12 +2,18 @@ --Want to be listed as an Inference Provider on the Hugging Face Hub? Let's get in touch! +Want to be listed as an Inference Provider on the Hugging Face Hub? Let's get in touch! Please reach out to us on social networks or [here on the Hub](https://huggingface.co/spaces/huggingface/HuggingDiscussions/discussions/49). + + +Note that Step 3 will require your organization to upgrade their Hub account to a [Team or Enterprise plan](https://huggingface.co/pricing). + + + This guide details the steps for registering as an inference provider on the Hub and provides implementation guidance. 1. **Implement standard task APIs** - Follow our task API schemas for compatibility (see [Prerequisites](#1-prerequisites)). @@ -131,7 +137,7 @@ First step is to use the Model Mapping API to register which HF models are suppo -To proceed with this step, we have to enable your account server-side. Make sure you have an organization on the Hub for your company, and upgrade it to a Team or Enterprise plan. +To proceed with this step, we have to enable your account server-side. Make sure you have an organization on the Hub for your company, and upgrade it to a [Team or Enterprise plan](https://huggingface.co/pricing). From f62c50ac0d4f9fe25919b9a88098cbfbf98a4e49 Mon Sep 17 00:00:00 2001 From: HuggingFaceInfra <148469759+HuggingFaceInfra@users.noreply.github.com> Date: Fri, 5 Sep 2025 09:38:57 +0200 Subject: [PATCH 6/7] Update Inference Providers documentation (automated) (#1897) Co-authored-by: Wauplin <11801849+Wauplin@users.noreply.github.com> --- .../providers/hf-inference.md | 28 +++++++++++++------ docs/inference-providers/providers/nebius.md | 4 +-- .../providers/replicate.md | 2 +- .../inference-providers/providers/together.md | 4 +-- .../tasks/chat-completion.md | 2 +- docs/inference-providers/tasks/fill-mask.md | 2 +- .../tasks/image-classification.md | 4 ++- .../tasks/image-segmentation.md | 3 +- .../tasks/object-detection.md | 6 +++- .../tasks/question-answering.md | 5 ++-- .../tasks/summarization.md | 4 +-- .../tasks/table-question-answering.md | 3 +- .../tasks/text-classification.md | 3 +- .../tasks/text-generation.md | 2 +- .../tasks/text-to-video.md | 2 +- .../tasks/token-classification.md | 3 +- docs/inference-providers/tasks/translation.md | 3 +- scripts/inference-providers/package.json | 2 +- scripts/inference-providers/pnpm-lock.yaml | 12 ++++---- 19 files changed, 56 insertions(+), 38 deletions(-) diff --git a/docs/inference-providers/providers/hf-inference.md b/docs/inference-providers/providers/hf-inference.md index b458eac86..4a84137b1 100644 --- a/docs/inference-providers/providers/hf-inference.md +++ b/docs/inference-providers/providers/hf-inference.md @@ -73,7 +73,7 @@ Find out more about Fill Mask [here](../tasks/fill_mask). @@ -83,7 +83,7 @@ Find out more about Image Classification [here](../tasks/image_classification). @@ -93,7 +93,17 @@ Find out more about Image Segmentation [here](../tasks/image_segmentation). + + +### Object Detection + +Find out more about Object Detection [here](../tasks/object_detection). + + @@ -103,7 +113,7 @@ Find out more about Question Answering [here](../tasks/question_answering). @@ -113,7 +123,7 @@ Find out more about Summarization [here](../tasks/summarization). @@ -123,7 +133,7 @@ Find out more about Table Question Answering [here](../tasks/table_question_answ @@ -133,7 +143,7 @@ Find out more about Text Classification [here](../tasks/text_classification). @@ -153,7 +163,7 @@ Find out more about Token Classification [here](../tasks/token_classification). @@ -163,7 +173,7 @@ Find out more about Translation [here](../tasks/translation). diff --git a/docs/inference-providers/providers/nebius.md b/docs/inference-providers/providers/nebius.md index 325312a54..7d7ab29cc 100644 --- a/docs/inference-providers/providers/nebius.md +++ b/docs/inference-providers/providers/nebius.md @@ -50,7 +50,7 @@ Find out more about Chat Completion (LLM) [here](../tasks/chat-completion). @@ -80,7 +80,7 @@ Find out more about Text Generation [here](../tasks/text_generation). diff --git a/docs/inference-providers/providers/replicate.md b/docs/inference-providers/providers/replicate.md index 6487d1fd7..a53f94ce9 100644 --- a/docs/inference-providers/providers/replicate.md +++ b/docs/inference-providers/providers/replicate.md @@ -70,6 +70,6 @@ Find out more about Text To Video [here](../tasks/text_to_video). diff --git a/docs/inference-providers/providers/together.md b/docs/inference-providers/providers/together.md index 617992c4f..2f24756d8 100644 --- a/docs/inference-providers/providers/together.md +++ b/docs/inference-providers/providers/together.md @@ -50,7 +50,7 @@ Find out more about Chat Completion (LLM) [here](../tasks/chat-completion). @@ -70,7 +70,7 @@ Find out more about Text Generation [here](../tasks/text_generation). diff --git a/docs/inference-providers/tasks/chat-completion.md b/docs/inference-providers/tasks/chat-completion.md index d7d546e19..154d6c18a 100644 --- a/docs/inference-providers/tasks/chat-completion.md +++ b/docs/inference-providers/tasks/chat-completion.md @@ -64,7 +64,7 @@ The API supports: diff --git a/docs/inference-providers/tasks/fill-mask.md b/docs/inference-providers/tasks/fill-mask.md index bcb432006..33344e16b 100644 --- a/docs/inference-providers/tasks/fill-mask.md +++ b/docs/inference-providers/tasks/fill-mask.md @@ -32,7 +32,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/image-classification.md b/docs/inference-providers/tasks/image-classification.md index e7115ff26..01799686e 100644 --- a/docs/inference-providers/tasks/image-classification.md +++ b/docs/inference-providers/tasks/image-classification.md @@ -24,6 +24,8 @@ For more details about the `image-classification` task, check out its [dedicated ### Recommended models +- [facebook/deit-base-distilled-patch16-224](https://huggingface.co/facebook/deit-base-distilled-patch16-224): A robust image classification model. +- [facebook/convnext-large-224](https://huggingface.co/facebook/convnext-large-224): A strong image classification model. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=image-classification&sort=trending). @@ -32,7 +34,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/image-segmentation.md b/docs/inference-providers/tasks/image-segmentation.md index c9d33790c..9dd683387 100644 --- a/docs/inference-providers/tasks/image-segmentation.md +++ b/docs/inference-providers/tasks/image-segmentation.md @@ -24,7 +24,6 @@ For more details about the `image-segmentation` task, check out its [dedicated p ### Recommended models -- [facebook/mask2former-swin-large-coco-panoptic](https://huggingface.co/facebook/mask2former-swin-large-coco-panoptic): Panoptic segmentation model trained on the COCO (common objects) dataset. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=image-segmentation&sort=trending). @@ -33,7 +32,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/object-detection.md b/docs/inference-providers/tasks/object-detection.md index 299a41788..3c36c4081 100644 --- a/docs/inference-providers/tasks/object-detection.md +++ b/docs/inference-providers/tasks/object-detection.md @@ -24,13 +24,17 @@ For more details about the `object-detection` task, check out its [dedicated pag ### Recommended models +- [facebook/detr-resnet-50](https://huggingface.co/facebook/detr-resnet-50): Solid object detection model pre-trained on the COCO 2017 dataset. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=object-detection&sort=trending). ### Using the API -There are currently no snippet examples for the **object-detection** task, as no providers support it yet. + diff --git a/docs/inference-providers/tasks/question-answering.md b/docs/inference-providers/tasks/question-answering.md index 3c01ff1ff..220b9cfbf 100644 --- a/docs/inference-providers/tasks/question-answering.md +++ b/docs/inference-providers/tasks/question-answering.md @@ -24,7 +24,8 @@ For more details about the `question-answering` task, check out its [dedicated p ### Recommended models -- [distilbert/distilbert-base-cased-distilled-squad](https://huggingface.co/distilbert/distilbert-base-cased-distilled-squad): Small yet robust model that can answer questions. +- [deepset/roberta-base-squad2](https://huggingface.co/deepset/roberta-base-squad2): A robust baseline model for most question answering domains. +- [google/tapas-base-finetuned-wtq](https://huggingface.co/google/tapas-base-finetuned-wtq): A special model that can answer questions from tables. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=question-answering&sort=trending). @@ -33,7 +34,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/summarization.md b/docs/inference-providers/tasks/summarization.md index 8b948f871..6e0ff5ead 100644 --- a/docs/inference-providers/tasks/summarization.md +++ b/docs/inference-providers/tasks/summarization.md @@ -24,7 +24,7 @@ For more details about the `summarization` task, check out its [dedicated page]( ### Recommended models -- [Falconsai/medical_summarization](https://huggingface.co/Falconsai/medical_summarization): A summarization model trained on medical articles. +- [facebook/bart-large-cnn](https://huggingface.co/facebook/bart-large-cnn): A strong summarization model trained on English news articles. Excels at generating factual summaries. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=summarization&sort=trending). @@ -33,7 +33,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/table-question-answering.md b/docs/inference-providers/tasks/table-question-answering.md index 820afc14f..358287c69 100644 --- a/docs/inference-providers/tasks/table-question-answering.md +++ b/docs/inference-providers/tasks/table-question-answering.md @@ -24,6 +24,7 @@ For more details about the `table-question-answering` task, check out its [dedic ### Recommended models +- [google/tapas-base-finetuned-wtq](https://huggingface.co/google/tapas-base-finetuned-wtq): A robust table question answering model. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=table-question-answering&sort=trending). @@ -32,7 +33,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/text-classification.md b/docs/inference-providers/tasks/text-classification.md index 7b1a73a67..6ecaefc66 100644 --- a/docs/inference-providers/tasks/text-classification.md +++ b/docs/inference-providers/tasks/text-classification.md @@ -24,6 +24,7 @@ For more details about the `text-classification` task, check out its [dedicated ### Recommended models +- [distilbert/distilbert-base-uncased-finetuned-sst-2-english](https://huggingface.co/distilbert/distilbert-base-uncased-finetuned-sst-2-english): A robust model trained for sentiment analysis. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=text-classification&sort=trending). @@ -32,7 +33,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/text-generation.md b/docs/inference-providers/tasks/text-generation.md index 46dcc7caa..e45ad84fc 100644 --- a/docs/inference-providers/tasks/text-generation.md +++ b/docs/inference-providers/tasks/text-generation.md @@ -42,7 +42,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/text-to-video.md b/docs/inference-providers/tasks/text-to-video.md index 9d8625958..c80c0324b 100644 --- a/docs/inference-providers/tasks/text-to-video.md +++ b/docs/inference-providers/tasks/text-to-video.md @@ -35,7 +35,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/token-classification.md b/docs/inference-providers/tasks/token-classification.md index b41170e6c..d0b614942 100644 --- a/docs/inference-providers/tasks/token-classification.md +++ b/docs/inference-providers/tasks/token-classification.md @@ -24,7 +24,6 @@ For more details about the `token-classification` task, check out its [dedicated ### Recommended models -- [blaze999/Medical-NER](https://huggingface.co/blaze999/Medical-NER): A token classification model specialized on medical entity recognition. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=token-classification&sort=trending). @@ -33,7 +32,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/docs/inference-providers/tasks/translation.md b/docs/inference-providers/tasks/translation.md index 6366a4807..86a7ac19e 100644 --- a/docs/inference-providers/tasks/translation.md +++ b/docs/inference-providers/tasks/translation.md @@ -24,6 +24,7 @@ For more details about the `translation` task, check out its [dedicated page](ht ### Recommended models +- [google-t5/t5-base](https://huggingface.co/google-t5/t5-base): A general-purpose Transformer that can be used to translate from English to German, French, or Romanian. Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=translation&sort=trending). @@ -32,7 +33,7 @@ Explore all available models and find the one that suits you best [here](https:/ diff --git a/scripts/inference-providers/package.json b/scripts/inference-providers/package.json index ae65464ca..23656ab28 100644 --- a/scripts/inference-providers/package.json +++ b/scripts/inference-providers/package.json @@ -15,7 +15,7 @@ "license": "ISC", "dependencies": { "@huggingface/inference": "^4.7.1", - "@huggingface/tasks": "^0.19.42", + "@huggingface/tasks": "^0.19.43", "@types/node": "^22.5.0", "handlebars": "^4.7.8", "node": "^20.17.0", diff --git a/scripts/inference-providers/pnpm-lock.yaml b/scripts/inference-providers/pnpm-lock.yaml index 00f2c8696..bec99cb29 100644 --- a/scripts/inference-providers/pnpm-lock.yaml +++ b/scripts/inference-providers/pnpm-lock.yaml @@ -12,8 +12,8 @@ importers: specifier: ^4.7.1 version: 4.7.1 '@huggingface/tasks': - specifier: ^0.19.42 - version: 0.19.42 + specifier: ^0.19.43 + version: 0.19.43 '@types/node': specifier: ^22.5.0 version: 22.5.0 @@ -197,8 +197,8 @@ packages: resolution: {integrity: sha512-yUZLld4lrM9iFxHCwFQ7D1HW2MWMwSbeB7WzWqFYDWK+rEb+WldkLdAJxUPOmgICMHZLzZGVcVjFh3w/YGubng==} engines: {node: '>=18'} - '@huggingface/tasks@0.19.42': - resolution: {integrity: sha512-+FTsNxQA6U8s+cVRzBONXqaQkqB7BrNCONGVk6HdYHMIugEEBo8m9WYZrtr8z0zQPJqqqi+yv2svupE15FCing==} + '@huggingface/tasks@0.19.43': + resolution: {integrity: sha512-ANO23K3ugclBl6VLwdt+7MxBkRkKEE17USUSqprHb29UB5ISigH+0AJcEuDA064uzn0hqYrG/nOcv1yARRt8bw==} '@jridgewell/resolve-uri@3.1.2': resolution: {integrity: sha512-bRISgCIjP20/tbWSPWMEi54QVPRZExkuD9lJL+UIxUKtwVJA8wW1Trb1jMs1RFXo1CBTNZ/5hpC9QvmKWdopKw==} @@ -418,11 +418,11 @@ snapshots: '@huggingface/inference@4.7.1': dependencies: '@huggingface/jinja': 0.5.1 - '@huggingface/tasks': 0.19.42 + '@huggingface/tasks': 0.19.43 '@huggingface/jinja@0.5.1': {} - '@huggingface/tasks@0.19.42': {} + '@huggingface/tasks@0.19.43': {} '@jridgewell/resolve-uri@3.1.2': {} From 2d43a51bf44ddfa7f3718c80d6ebe3c31efd3f8c Mon Sep 17 00:00:00 2001 From: Kevin Wang Date: Fri, 5 Sep 2025 02:19:09 -0700 Subject: [PATCH 7/7] update wording --- docs/hub/_toctree.yml | 2 ++ docs/hub/datasets-daft.md | 12 ++++++------ docs/hub/datasets-libraries.md | 2 +- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/docs/hub/_toctree.yml b/docs/hub/_toctree.yml index 378d272fd..8a3b87e89 100644 --- a/docs/hub/_toctree.yml +++ b/docs/hub/_toctree.yml @@ -175,6 +175,8 @@ sections: - local: datasets-argilla title: Argilla + - local: datasets-daft + title: Daft - local: datasets-dask title: Dask - local: datasets-usage diff --git a/docs/hub/datasets-daft.md b/docs/hub/datasets-daft.md index 4c7f0ced2..cc92fc649 100644 --- a/docs/hub/datasets-daft.md +++ b/docs/hub/datasets-daft.md @@ -17,11 +17,11 @@ pip install 'daft[hugggingface]' ## Read -Daft is able to read datasets directly from Hugging Face using the [`daft.read_huggingface()`](https://docs.daft.ai/en/stable/api/io/#daft.read_huggingface) function or via the `hf://datasets/` protocol. +Daft is able to read datasets directly from the Hugging Face Hub using the [`daft.read_huggingface()`](https://docs.daft.ai/en/stable/api/io/#daft.read_huggingface) function or via the `hf://datasets/` protocol. ### Reading an Entire Dataset -Using [`daft.read_huggingface()`](https://docs.daft.ai/en/stable/api/io/#daft.read_huggingface), you can easily read a Hugging Face dataset. +Using [`daft.read_huggingface()`](https://docs.daft.ai/en/stable/api/io/#daft.read_huggingface), you can easily load a dataset. ```python @@ -34,7 +34,7 @@ This will read the entire dataset into a DataFrame. ### Reading Specific Files -Not only can you read entire datasets, but you can also read individual files from a dataset. Using a read function that takes in a path (such as [`daft.read_parquet()`](https://docs.daft.ai/en/stable/api/io/#daft.read_parquet), [`daft.read_csv()`](https://docs.daft.ai/en/stable/api/io/#daft.read_csv), or [`daft.read_json()`](https://docs.daft.ai/en/stable/api/io/#daft.read_json)), specify a Hugging Face dataset path via the `hf://datasets/` prefix: +Not only can you read entire datasets, but you can also read individual files from a dataset repository. Using a read function that takes in a path (such as [`daft.read_parquet()`](https://docs.daft.ai/en/stable/api/io/#daft.read_parquet), [`daft.read_csv()`](https://docs.daft.ai/en/stable/api/io/#daft.read_csv), or [`daft.read_json()`](https://docs.daft.ai/en/stable/api/io/#daft.read_json)), specify a Hugging Face dataset path via the `hf://datasets/` prefix: ```python import daft @@ -51,7 +51,7 @@ df = daft.read_parquet("hf://datasets/username/dataset_name/**/*.parquet") ## Write -Daft is able to write Parquet files to Hugging Face datasets using [`daft.DataFrame.write_huggingface`](https://docs.daft.ai/en/stable/api/dataframe/#daft.DataFrame.write_deltalake). Daft supports [Content-Defined Chunking](https://huggingface.co/blog/parquet-cdc) and [Xet](https://huggingface.co/blog/xet-on-the-hub) for faster, deduplicated writes. +Daft is able to write Parquet files to a Hugging Face dataset repository using [`daft.DataFrame.write_huggingface`](https://docs.daft.ai/en/stable/api/dataframe/#daft.DataFrame.write_deltalake). Daft supports [Content-Defined Chunking](https://huggingface.co/blog/parquet-cdc) and [Xet](https://huggingface.co/blog/xet-on-the-hub) for faster, deduplicated writes. Basic usage: @@ -67,9 +67,9 @@ See the [`DataFrame.write_huggingface`](https://docs.daft.ai/en/stable/api/dataf ## Authentication -The `token` parameter in [`daft.io.HuggingFaceConfig`](https://docs.daft.ai/en/stable/api/config/#daft.io.HuggingFaceConfig) can be used to specify a Hugging Face access token for requests that require authentication (e.g. reading private datasets or writing to a dataset). +The `token` parameter in [`daft.io.HuggingFaceConfig`](https://docs.daft.ai/en/stable/api/config/#daft.io.HuggingFaceConfig) can be used to specify a Hugging Face access token for requests that require authentication (e.g. reading private dataset repositories or writing to a dataset repository). -Example of reading a dataset with a specified token: +Example of loading a dataset with a specified token: ```python from daft.io import IOConfig, HuggingFaceConfig diff --git a/docs/hub/datasets-libraries.md b/docs/hub/datasets-libraries.md index 2d38c5d36..4d05bef93 100644 --- a/docs/hub/datasets-libraries.md +++ b/docs/hub/datasets-libraries.md @@ -88,7 +88,7 @@ Examples of this kind of integration: #### Rely on an existing libraries integration with the Hub -Polars, Pandas, Dask, Spark, DuckDB, and Daft all can write to a Hugging Face Hub repository. See [datasets libraries](https://huggingface.co/docs/hub/datasets-libraries) for more details. +Polars, Pandas, Dask, Spark, DuckDB, and Daft can all write to a Hugging Face Hub repository. See [datasets libraries](https://huggingface.co/docs/hub/datasets-libraries) for more details. If you are already using one of these libraries in your code, adding the ability to push to the Hub is straightforward. For example, if you have a synthetic data generation library that can return a Pandas DataFrame, here is the code you would need to write to the Hub: