Organize Hub Xet docs (#1969)

* Reorganized Xet docs, hopefully this renders ok
* Add xet protocol subsection pages, finish toctree
* Fix links in xet proto contents pages
* quick fixes of some links
* Reorganize Storage Backend docs
* Fix TOC and link to xet protocol
* Fix storage-backends rename
* Update xet protocol to 1.0.0
* add xet protocol link to security
* Update docs/hub/_redirects.yml

add redirect for old storage-backends links

Co-authored-by: Julien Chaumond <[email protected]>

* Reorganizing TOC
* PR feedback

---------

Co-authored-by: Jared Sulzdorf <[email protected]>
Co-authored-by: Julien Chaumond <[email protected]>

Author: 3 people

docs/hub/_redirects.yml

@@ -18,4 +18,5 @@ api-webhook: webhooks
 adapter-transformers: adapters
 security-two-fa: security-2fa
 repositories-recommendations: storage-limits
-xet: storage-backends#xet
+xet: xet/index
+storage-backends: xet/index

docs/hub/_toctree.yml

@@ -47,8 +47,19 @@
     title: Repository Settings
   - local: storage-limits
     title: Storage Limits
-  - local: storage-backends
-    title: Storage Backends
+  - local: xet/index
+    title: Storage Backend (Xet)
+    sections:
+    - local: xet/overview
+      title: Xet History & Overview
+    - local: xet/using-xet-storage
+      title: Using Xet Storage
+    - local: xet/security
+      title: Security
+    - local: xet/legacy-git-lfs
+      title: Backwards Compatibility & Legacy
+    - local: xet/deduplication
+      title: Deduplication
   - local: repositories-pull-requests-discussions
     title: Pull Requests & Discussions
   - local: notifications

docs/hub/index.md

@@ -27,7 +27,7 @@ The Hugging Face Hub is a platform with over 2M models, 500k datasets, and 1M de
 <a class="transform no-underline! transition-colors hover:translate-x-px hover:text-gray-700" href="./repositories-getting-started">Getting Started</a>
 <a class="transform no-underline! transition-colors hover:translate-x-px hover:text-gray-700" href="./repositories-settings">Repository Settings</a>
 <a class="transform no-underline! transition-colors hover:translate-x-px hover:text-gray-700" href="./storage-limits">Storage Limits</a>
-<a class="transform no-underline! transition-colors hover:translate-x-px hover:text-gray-700" href="./storage-backends">Storage Backends</a>
+<a class="transform no-underline! transition-colors hover:translate-x-px hover:text-gray-700" href="./xet/index">Storage Backend (Xet)</a>
 <a class="transform no-underline! transition-colors hover:translate-x-px hover:text-gray-700" href="./repositories-pull-requests-discussions">Pull requests and Discussions</a>
 <a class="transform no-underline! transition-colors hover:translate-x-px hover:text-gray-700" href="./notifications">Notifications</a>
 <a class="transform no-underline! transition-colors hover:translate-x-px hover:text-gray-700" href="./collections">Collections</a>
@@ -122,7 +122,7 @@ On it, you'll be able to upload and discover...
 - Spaces: _interactive apps for demonstrating ML models directly in your browser_
 
 The Hub offers **versioning, commit history, diffs, branches, and over a dozen library integrations**! 
-All repositories build on [Xet](https://huggingface.co/join/xet), a new technology to efficiently store Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads.
+All repositories build on [Xet](./xet/index), a new technology to efficiently store Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads.
 
 You can learn more about the features that all repositories share in the [**Repositories documentation**](./repositories).
 

docs/hub/repositories.md

@@ -3,7 +3,7 @@
 Models, Spaces, and Datasets are hosted on the Hugging Face Hub as [Git repositories](https://git-scm.com/about), which means that version control and collaboration are core elements of the Hub. In a nutshell, a repository (also known as a **repo**) is a place where code and assets can be stored to back up your work, share it with the community, and work in a team. 
 
 Unlike other collaboration platforms, our Git repositories are optimized for Machine Learning and AI files – large binary files, usually in specific file formats like Parquet and Safetensors, and up to [Terabyte-scale sizes](https://huggingface.co/blog/from-files-to-chunks)!
-To achieve this, we built [Xet](./storage-backends), a modern custom storage system built specifically for AI/ML development, enabling chunk-level deduplication, smaller uploads, and faster downloads.
+To achieve this, we built [Xet](./xet/index), a modern custom storage system built specifically for AI/ML development, enabling chunk-level deduplication, smaller uploads, and faster downloads.
 
 <div class="flex justify-center">
 <img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/xet-speed.gif"/>
@@ -17,7 +17,7 @@ In these pages, you will go over the basics of getting started with Git and Xet
 - [Getting Started with Repositories](./repositories-getting-started)
 - [Settings](./repositories-settings)
 - [Storage Limits](./storage-limits)
-- [Storage Backends](./storage-backends)
+- [Storage Backend (Xet)](./xet/index)
 - [Pull Requests & Discussions](./repositories-pull-requests-discussions)
   - [Pull Requests advanced usage](./repositories-pull-requests-discussions#pull-requests-advanced-usage)
 - [Collections](./collections)