|
| 1 | +--- |
| 2 | +title: "Hugging Face" |
| 3 | +--- |
| 4 | + |
| 5 | +Hugging Face is the main platform for sharing open AI models. It provides inference in two ways. [Inference Providers](https://huggingface.co/docs/inference-providers/index) and [Inference Endpoints](https://huggingface.co/docs/inference-endpoints/index). |
| 6 | + |
| 7 | +## Inference Providers |
| 8 | + |
| 9 | +Inference Providers is a serverless service powered by external inference providers and routed through Hugging Face and paid per token. |
| 10 | + |
| 11 | +<Info> |
| 12 | + |
| 13 | +You can access your access token from [Hugging Face](https://huggingface.co/settings/tokens) and prioritize your [providers in settings](https://huggingface.co/settings/inference-providers/overview). |
| 14 | + |
| 15 | +</Info> |
| 16 | + |
| 17 | +<Tabs> |
| 18 | + <Tab title="YAML"> |
| 19 | + ```yaml title="config.yaml" |
| 20 | + name: My Config |
| 21 | + version: 0.0.1 |
| 22 | + schema: v1 |
| 23 | + |
| 24 | + models: |
| 25 | + - name: deepseek |
| 26 | + provider: huggingface-inference-providers |
| 27 | + model: deepseek-ai/DeepSeek-V3.2-Exp |
| 28 | + apiKey: <YOUR_HF_TOKEN> |
| 29 | + apiBase: https://router.huggingface.co/v1 |
| 30 | + ``` |
| 31 | + </Tab> |
| 32 | + <Tab title="JSON (Deprecated)"> |
| 33 | + ```json title="config.json" |
| 34 | + { |
| 35 | + "models": [ |
| 36 | + { |
| 37 | + "title": "deepseek", |
| 38 | + "provider": "huggingface-inference-providers", |
| 39 | + "model": "deepseek-ai/DeepSeek-V3.2-Exp", |
| 40 | + "apiKey": "<YOUR_HF_TOKEN>", |
| 41 | + "apiBase": "https://router.huggingface.co/v1" |
| 42 | + } |
| 43 | + ] |
| 44 | + } |
| 45 | + ``` |
| 46 | + </Tab> |
| 47 | +</Tabs> |
| 48 | + |
| 49 | +## Inference Endpoints |
| 50 | + |
| 51 | +Inference Endpoints is a dedicated service that allows you to run your open models dedicated hardware. It is a more advanced way to get inference from Hugging Face models where you have more control over the whole process. |
| 52 | + |
| 53 | +<Info> |
| 54 | + |
| 55 | +Before you can use Inference Endpoints, you need to create an endpoint. You can do this by going to [Inference Endpoints](https://endpoints.huggingface.co/burtenshaw/endpoints/dedicated) and clicking on "Create Endpoint". |
| 56 | + |
| 57 | +</Info> |
| 58 | + |
| 59 | +<Tabs> |
| 60 | + <Tab title="YAML"> |
| 61 | + ```yaml title="config.yaml" |
| 62 | + name: My Config |
| 63 | + version: 0.0.1 |
| 64 | + schema: v1 |
| 65 | + |
| 66 | + models: |
| 67 | + - name: deepseek |
| 68 | + provider: huggingface-inference-endpoints |
| 69 | + model: <ENDPOINT_ID> |
| 70 | + apiKey: <YOUR_HF_TOKEN> |
| 71 | + apiBase: https://<YOUR_ENDPOINT_ID>.aws.endpoints.huggingface.cloud |
| 72 | + ``` |
| 73 | + </Tab> |
| 74 | + <Tab title="JSON (Deprecated)"> |
| 75 | + ```json title="config.json" |
| 76 | + { |
| 77 | + "models": [ |
| 78 | + { |
| 79 | + "title": "deepseek", |
| 80 | + "provider": "huggingface-inference-endpoints", |
| 81 | + "model": "<ENDPOINT_ID>", |
| 82 | + "apiKey": "<YOUR_HF_TOKEN>", |
| 83 | + "apiBase": "https://<YOUR_ENDPOINT_ID>.aws.endpoints.huggingface.cloud" |
| 84 | + } |
| 85 | + ] |
| 86 | + } |
| 87 | + ``` |
| 88 | + </Tab> |
| 89 | +</Tabs> |
0 commit comments