You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/hub/repositories-getting-started.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,6 +8,9 @@ This document shows how to handle repositories through the web interface as well
8
8
9
9
If you do not have `git` available as a CLI command yet, you will need to [install Git](https://git-scm.com/downloads) for your platform. You will also need to [install Git LFS](https://git-lfs.github.com/), which will be used to handle large files such as images and model weights.
10
10
11
+
> [!TIP]
12
+
> For improved upload and download speeds when working with large files and Git, install the [Git Xet](xet/using-xet-storage#git) extension.
13
+
11
14
To be able to push your code to the Hub, you'll need to authenticate somehow. The easiest way to do this is by installing the [`huggingface_hub` CLI](https://huggingface.co/docs/huggingface_hub/index) and running the login command:
-[Managing the `hf_xet` cache](https://huggingface.co/docs/huggingface_hub/guides/manage-cache#chunk-based-caching-xet)
26
28
27
-
### Recommendations
29
+
## Git
30
+
31
+
Git users can access the benefits of Xet by downloading and installing the Git Xet extension. Once installed, simply use the [standard workflows for managing Hub repositories with Git](../repositories-getting-started) - no additional changes necessary.
32
+
33
+
### Prerequisites
34
+
35
+
Install [Git](https://git-scm.com/) and [Git LFS](https://git-lfs.com/).
36
+
37
+
### Install on macOS or Linux (amd64 or aarch64)
38
+
39
+
To install using [Homebrew](https://brew.sh/):
40
+
```
41
+
brew tap huggingface/tap
42
+
brew install git-xet
43
+
```
44
+
Or, using an installation script, run the following in your terminal (requires `curl` and `unzip`):
45
+
```
46
+
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/huggingface/xet-core/refs/heads/main/git_xet/install.sh | sh
47
+
```
48
+
To verify the installation, run:
49
+
```
50
+
git-xet --version
51
+
```
52
+
### Windows (amd64)
53
+
54
+
Using an installer:
55
+
- Download `git-xet-windows-installer-x86_64.zip` ([available here](https://github.com/huggingface/xet-core/releases/download/git-xet-v0.1.0/git-xet-windows-installer-x86_64.zip)) and unzip.
56
+
- Run the `msi` installer file and follow the prompts.
57
+
58
+
Manual installation:
59
+
- Download `git-xet-windows-x86_64.zip` ([available here](https://github.com/huggingface/xet-core/releases/download/git-xet-v0.1.0/git-xet-windows-x86_64.zip)) and unzip.
60
+
- Place the extracted `git-xet.exe` under a `PATH` directory.
61
+
- Run `git-xet install` in a terminal.
62
+
63
+
To verity the installation, run:
64
+
```
65
+
git-xet --version
66
+
```
67
+
68
+
### Uninstall on macOS or Linux
69
+
70
+
Using Homebrew:
71
+
```
72
+
git-xet uninstall
73
+
brew uninstall git-xet
74
+
```
75
+
If you used the installation script (for MacOS or Linux), run the following in your terminal:
76
+
```
77
+
git-xet uninstall
78
+
sudo rm $(which git-xet)
79
+
```
80
+
### Uninstall on Windows
81
+
82
+
If you used the installer:
83
+
- Navigate to Settings -> Apps -> Installed apps
84
+
- Find "Git-Xet".
85
+
- Select the "Uninstall" option available in the context menu.
86
+
87
+
If you manually installed:
88
+
- Run `git-xet uninstall` in a terminal.
89
+
- Delete the `git-xet.exe` file from the location where it was originally placed.
90
+
91
+
## Recommendations
92
+
93
+
Xet integrates seamlessly with all of the Hub's workflows. However, there are a few steps you may consider to get the most benefits from Xet storage.
94
+
95
+
When uploading or downloading with Python:
96
+
97
+
-**Make sure `hf_xet` is installed**: While Xet remains backward compatible with legacy clients optimized for Git LFS, the `hf_xet` integration with `huggingface_hub` delivers optimal chunk-based performance and faster iteration on large files.
98
+
-**Utilize `hf_xet` environment variables**: The default installation of `hf_xet` is designed to support the broadest range of hardware. To take advantage of setups with more network bandwidth or processing power read up on `hf_xet`'s [environment variables](https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables#xet) to optimize downloads and uploads.
28
99
29
-
Xet integrates seamlessly with the Hub's current Python-based workflows. However, there are a few steps you may consider to get the most benefits from Xet storage:
100
+
When uploading or downloading in Git or Python:
30
101
31
-
-**Use `hf_xet`**: While Xet remains backward compatible with legacy clients optimized for Git LFS, the `hf_xet` integration with `huggingface_hub` delivers optimal chunk-based performance and faster iteration on large files.
32
-
-**Utilize `hf_xet` environment variables**: The default installation of `hf_xet` is designed to support the broadest range of hardware. To take advantage of setups with more network bandwidth or processing power read up on `hf_xet`'s [environment variables](https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables#xet) to further speed up downloads and uploads.
33
102
-**Leverage frequent, incremental commits**: Xet's chunk-level deduplication means you can safely make incremental updates to models or datasets. Only changed chunks are uploaded, so frequent commits are both fast and storage-efficient.
34
103
-**Be Specific in .gitattributes**: When defining patterns for Xet or LFS, use precise file extensions (e.g., `*.safetensors`, `*.bin`) to avoid unnecessarily routing smaller files through large-file storage.
35
104
-**Prioritize community access**: Xet substantially increases the efficiency and scale of large file transfers. Instead of structuring your repository to reduce its total size (or the size of individual files), organize it for collaborators and community users so they may easily navigate and retrieve the content they need.
36
105
37
-
###Current Limitations
106
+
## Current Limitations
38
107
39
108
While Xet brings fine-grained deduplication and enhanced performance to Git-based storage, some features and platform compatibilities are still in development. As a result, keep the following constraints in mind when working with a Xet-enabled repository:
40
109
41
-
-**64-bit systems only**: The `hf_xet` client currently requires a 64-bit architecture; 32-bit systems are not supported.
42
-
-**Git client integration (git-xet)**: Under active development - coming soon, stay tuned!
110
+
-**64-bit systems only**: The `hf_xet` client currently requires a 64-bit architecture; 32-bit systems are not supported.
0 commit comments