Skip to content

Commit c400729

Browse files
Vaibhavs10Wauplin
andauthored
purge HF_HUB_ENABLE_HF_TRANSFER; promote Xet (#2000)
* purge HF_HUB_ENABLE_HF_TRANSFER; promote Xet * Apply suggestion from @Wauplin Co-authored-by: Lucain <[email protected]> * review. * Update docs/hub/models-downloading.md * Update docs/hub/models-downloading.md --------- Co-authored-by: Lucain <[email protected]>
1 parent 57cbf36 commit c400729

File tree

2 files changed

+5
-14
lines changed

2 files changed

+5
-14
lines changed

docs/hub/mlx.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -70,9 +70,9 @@ response = generate(model, tokenizer, prompt="hello", verbose=True)
7070
MLX-LM supports popular LLM architectures including LLaMA, Phi-2, Mistral, and Qwen. Models other than supported ones can easily be downloaded as follows:
7171

7272
```py
73-
pip install huggingface_hub hf_transfer
73+
pip install -U huggingface_hub
7474

75-
export HF_HUB_ENABLE_HF_TRANSFER=1
75+
export HF_XET_HIGH_PERFORMANCE=1
7676
hf download --local-dir <LOCAL FOLDER PATH> <USER_ID>/<MODEL_NAME>
7777
```
7878

docs/hub/models-downloading.md

Lines changed: 3 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -51,18 +51,9 @@ Add your SSH public key to [your user settings](https://huggingface.co/settings/
5151
## Faster downloads
5252

5353
If you are running on a machine with high bandwidth,
54-
you can increase your download speed with [`hf_transfer`](https://github.com/huggingface/hf_transfer),
55-
a Rust-based library developed to speed up file transfers with the Hub.
54+
you can speed up downloads by allowing `hf_xet` to run on all CPU cores. `hf_xet` is a Rust-based package leveraging the new [Xet storage backend](https://huggingface.co/docs/hub/en/xet/index) to optimize file transfers with chunk-based deduplication. `hf_xet` is enabled by default but with lower performances to avoid bloating available CPU and bandwidth, which could degrade UX.
5655

5756
```bash
58-
pip install "huggingface_hub[hf_transfer]"
59-
HF_HUB_ENABLE_HF_TRANSFER=1 hf download ...
57+
pip install -U huggingface_hub
58+
HF_XET_HIGH_PERFORMANCE=1 hf download ...
6059
```
61-
62-
> [!WARNING]
63-
> `hf_transfer` is a power user tool!
64-
> It is tested and production-ready,
65-
> but it lacks user-friendly features like advanced error handling or proxies.
66-
> For more details, please take a look at this [guide](https://huggingface.co/docs/huggingface_hub/hf_transfer).
67-
68-

0 commit comments

Comments
 (0)