From 3c38c2e80c159be8aa4ed2dad4fc084c3e09f8e4 Mon Sep 17 00:00:00 2001 From: Victor Mustar Date: Tue, 15 Oct 2024 13:50:52 +0200 Subject: [PATCH 1/5] add doc --- docs/hub/_toctree.yml | 2 + docs/hub/spaces-zerogpu.md | 93 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 95 insertions(+) create mode 100644 docs/hub/spaces-zerogpu.md diff --git a/docs/hub/_toctree.yml b/docs/hub/_toctree.yml index 1338b5d17..e200c824e 100644 --- a/docs/hub/_toctree.yml +++ b/docs/hub/_toctree.yml @@ -238,6 +238,8 @@ title: Using Spaces for Organization Cards - local: spaces-gpus title: Spaces GPU Upgrades + - local: spaces-zerogpu + title: Spaces ZeroGPU - local: spaces-storage title: Spaces Persistent Storage - local: spaces-sdks-gradio diff --git a/docs/hub/spaces-zerogpu.md b/docs/hub/spaces-zerogpu.md new file mode 100644 index 000000000..eaafb0e3a --- /dev/null +++ b/docs/hub/spaces-zerogpu.md @@ -0,0 +1,93 @@ +# Spaces ZeroGPU: Dynamic GPU Allocation for Spaces + +ZeroGPU schema + +ZeroGPU introduces a new approach to GPU utilization in Hugging Face Spaces. This innovative hardware solution offers two primary benefits: + +1. **Free GPU Access**: Enables cost-effective GPU usage for Spaces. +2. **Multi-GPU Support**: Allows Spaces to leverage multiple GPUs concurrently on a single applications. + +ZeroGPU achieves these goals by implementing an efficient GPU allocation system, dynamically assigning and releasing GPUs as needed. This contrasts with traditional GPU Spaces, which are limited to a single dedicated GPU. + +## Using and hosting ZeroGPU Spaces + +- **Using existing ZeroGPU Spaces** + - ZeroGPU Spaces are available to use for free to all users. (Visit [the curated list](https://huggingface.co/spaces/enzostvs/zero-gpu-spaces)). + - [PRO users](https://huggingface.co/subscribe/pro) get x5 more daily usage quota and highest priority in GPU queues when using any ZeroGPU Spaces. +- **Hosting your own ZeroGPU Spaces** + - Personal accounts: [Subscribe to PRO](https://huggingface.co/settings/billing/subscription) to access ZeroGPU in the hardware options when creating a new Gradio SDK Space. + - Organizations: [Subscribe to the Enterprise Hub](https://huggingface.co/enterprise) to enable ZeroGPU Spaces for all organization members. + +## Technical Specifications + +- **GPU Type**: Nvidia A100 +- **Available VRAM**: 40GB per workload + +## Compatibility + +ZeroGPU Spaces are designed to be compatible with most PyTorch-based GPU Spaces. While compatibility is enhanced for high-level Hugging Face libraries like `transformers` and `diffusers`, users should be aware that: + +- Currently, ZeroGPU Spaces are exclusively compatible with the **Gradio SDK**. +- ZeroGPU Spaces may have limited compatibility compared to standard GPU Spaces. +- Unexpected issues may arise in some scenarios. + +### Supported Versions + +- Gradio: 4+ +- PyTorch: 2.0.1, 2.1.2, 2.2.2, 2.4.0 (Note: 2.3.x is not supported due to a [PyTorch bug](https://github.com/pytorch/pytorch/issues/122085)) +- Python: 3.10.13 + +## Getting started with ZeroGPU + +To utilize ZeroGPU in your Space, follow these steps: + +1. Make sure the ZeroGPU hardware is selected in your Space settings. +2. Import the `spaces` module. +3. Decorate GPU-dependent functions with `@spaces.GPU`. + +This decoration process allows the Space to request a GPU when the function is called and release it upon completion. + +### Example Usage + +```python +import spaces +from diffusers import DiffusionPipeline + +pipe = DiffusionPipeline.from_pretrained(...) +pipe.to('cuda') + +@spaces.GPU +def generate(prompt): + return pipe(prompt).images + +gr.Interface( + fn=generate, + inputs=gr.Text(), + outputs=gr.Gallery(), +).launch() +``` + +Note: The `@spaces.GPU` decorator is designed to be effect-free in non-ZeroGPU environments, ensuring compatibility across different setups. + +## Duration Management + +For functions expected to exceed the default 60-second runtime, specify a custom duration: + +```python +@spaces.GPU(duration=120) +def generate(prompt): + return pipe(prompt).images +``` + +This sets the maximum function runtime to 120 seconds. Specifying shorter durations for quicker functions can improve queue priority for Space visitors. + +## Limitations + +- **Personal accounts (PRO subscribers)**: Maximum of 10 ZeroGPU Spaces. +- **Organization accounts (Enterprise Hub)**: Maximum of 50 ZeroGPU Spaces. + +By leveraging ZeroGPU, developers can create more efficient and scalable Spaces, maximizing GPU utilization while minimizing costs. + +## Feedback + +You can share your feedback on Spaces ZeroGPU directly on the HF Hub: https://huggingface.co/spaces/zero-gpu-explorers/README/discussions From e684cc37538f8b2210810335c5c6cdfe137a966e Mon Sep 17 00:00:00 2001 From: Victor Mustar Date: Tue, 15 Oct 2024 13:52:29 +0200 Subject: [PATCH 2/5] spelling --- docs/hub/spaces-zerogpu.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hub/spaces-zerogpu.md b/docs/hub/spaces-zerogpu.md index eaafb0e3a..21d7bf278 100644 --- a/docs/hub/spaces-zerogpu.md +++ b/docs/hub/spaces-zerogpu.md @@ -5,7 +5,7 @@ ZeroGPU introduces a new approach to GPU utilization in Hugging Face Spaces. This innovative hardware solution offers two primary benefits: 1. **Free GPU Access**: Enables cost-effective GPU usage for Spaces. -2. **Multi-GPU Support**: Allows Spaces to leverage multiple GPUs concurrently on a single applications. +2. **Multi-GPU Support**: Allows Spaces to leverage multiple GPUs concurrently on a single application. ZeroGPU achieves these goals by implementing an efficient GPU allocation system, dynamically assigning and releasing GPUs as needed. This contrasts with traditional GPU Spaces, which are limited to a single dedicated GPU. From ab0511cdba90bf2b6468ffcf99dd76d4140789a5 Mon Sep 17 00:00:00 2001 From: Victor Mustar Date: Tue, 15 Oct 2024 13:55:29 +0200 Subject: [PATCH 3/5] Update spaces-zerogpu.md --- docs/hub/spaces-zerogpu.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/hub/spaces-zerogpu.md b/docs/hub/spaces-zerogpu.md index 21d7bf278..93faa02ee 100644 --- a/docs/hub/spaces-zerogpu.md +++ b/docs/hub/spaces-zerogpu.md @@ -71,7 +71,7 @@ Note: The `@spaces.GPU` decorator is designed to be effect-free in non-ZeroGPU e ## Duration Management -For functions expected to exceed the default 60-second runtime, specify a custom duration: +For functions expected to exceed the default 60-second of GPU runtime, you can specify a custom duration: ```python @spaces.GPU(duration=120) @@ -79,12 +79,12 @@ def generate(prompt): return pipe(prompt).images ``` -This sets the maximum function runtime to 120 seconds. Specifying shorter durations for quicker functions can improve queue priority for Space visitors. +This sets the maximum function runtime to 120 seconds. Specifying shorter durations for quicker functions will improve queue priority for Space visitors. -## Limitations +## Hosting Limitations -- **Personal accounts (PRO subscribers)**: Maximum of 10 ZeroGPU Spaces. -- **Organization accounts (Enterprise Hub)**: Maximum of 50 ZeroGPU Spaces. +- **Personal accounts ([PRO subscribers](https://huggingface.co/subscribe/pro))**: Maximum of 10 ZeroGPU Spaces. +- **Organization accounts ([Enterprise Hub](https://huggingface.co/enterprise))**: Maximum of 50 ZeroGPU Spaces. By leveraging ZeroGPU, developers can create more efficient and scalable Spaces, maximizing GPU utilization while minimizing costs. From 6dfd471ea0b92d45ffa25d8ee063876be4da103c Mon Sep 17 00:00:00 2001 From: Victor Mustar Date: Tue, 15 Oct 2024 14:04:21 +0200 Subject: [PATCH 4/5] intro --- docs/hub/spaces-zerogpu.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/hub/spaces-zerogpu.md b/docs/hub/spaces-zerogpu.md index 93faa02ee..fbf920840 100644 --- a/docs/hub/spaces-zerogpu.md +++ b/docs/hub/spaces-zerogpu.md @@ -2,12 +2,12 @@ ZeroGPU schema -ZeroGPU introduces a new approach to GPU utilization in Hugging Face Spaces. This innovative hardware solution offers two primary benefits: +ZeroGPU is a shared infrastructure that optimizes GPU usage for AI models and demos on Hugging Face Spaces. It dynamically allocates and releases NVIDIA A100 GPUs as needed, offering: 1. **Free GPU Access**: Enables cost-effective GPU usage for Spaces. 2. **Multi-GPU Support**: Allows Spaces to leverage multiple GPUs concurrently on a single application. -ZeroGPU achieves these goals by implementing an efficient GPU allocation system, dynamically assigning and releasing GPUs as needed. This contrasts with traditional GPU Spaces, which are limited to a single dedicated GPU. +Unlike traditional single-GPU allocations, ZeroGPU's efficient system lowers barriers for developers, researchers, and organizations to deploy AI models by maximizing resource utilization and power efficiency. ## Using and hosting ZeroGPU Spaces From 89035f981d837083e94cfbefa594be3277d3f106 Mon Sep 17 00:00:00 2001 From: Victor Mustar Date: Tue, 15 Oct 2024 14:05:09 +0200 Subject: [PATCH 5/5] image width --- docs/hub/spaces-zerogpu.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hub/spaces-zerogpu.md b/docs/hub/spaces-zerogpu.md index fbf920840..ad4aba412 100644 --- a/docs/hub/spaces-zerogpu.md +++ b/docs/hub/spaces-zerogpu.md @@ -1,6 +1,6 @@ # Spaces ZeroGPU: Dynamic GPU Allocation for Spaces -ZeroGPU schema +ZeroGPU schema ZeroGPU is a shared infrastructure that optimizes GPU usage for AI models and demos on Hugging Face Spaces. It dynamically allocates and releases NVIDIA A100 GPUs as needed, offering: