Skip to content

Commit cd49e24

Browse files
committed
add section on specify provider vs auto
1 parent 9b7050b commit cd49e24

File tree

1 file changed

+106
-0
lines changed

1 file changed

+106
-0
lines changed

docs/inference-providers/guides/first-api-call.md

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,112 @@ Nice work! You've successfully used a production-grade AI model without any comp
124124

125125
The model you just used runs on professional infrastructure, handling scaling, optimization, and reliability automatically.
126126

127+
## Dive Deeper: Provider Selection
128+
129+
You might have noticed the `provider="auto"` parameter in the code examples above. This is a key feature of Inference Providers that gives you control over which infrastructure provider handles your request.
130+
131+
`auto` is powerful because:
132+
133+
1. It makes it easy to switch between providers, and to test different providers' performance for your use case.
134+
2. It also gives a fallback mechanism in case a provider is unavailable.
135+
136+
But if you want to be more specific, you can also specify a provider. Let's see how.
137+
138+
### Understanding Provider Selection
139+
140+
When you use `provider="auto"` (which is the default), the system automatically selects the first available provider for your chosen model based on your preference order in your [Inference Provider settings](https://hf.co/settings/inference-providers). This provides:
141+
142+
- **Automatic failover**: If one provider is unavailable, the system tries the next one
143+
- **Simplified setup**: No need to research which providers support your model
144+
- **Optimal routing**: The system handles provider selection for you
145+
146+
### Specifying a Specific Provider
147+
148+
Alternatively, you can explicitly choose a provider if you have specific requirements:
149+
150+
<hfoptions id="provider-selection-examples">
151+
152+
<hfoption id="python">
153+
154+
```python
155+
import os
156+
from huggingface_hub import InferenceClient
157+
158+
client = InferenceClient(api_key=os.environ["HF_TOKEN"])
159+
160+
# Using automatic provider selection (default)
161+
image_auto = client.text_to_image(
162+
"Astronaut riding a horse",
163+
model="black-forest-labs/FLUX.1-schnell",
164+
provider="auto" # This is the default
165+
)
166+
167+
# Using a specific provider
168+
image_fal = client.text_to_image(
169+
"Astronaut riding a horse",
170+
model="black-forest-labs/FLUX.1-schnell",
171+
provider="fal-ai" # Explicitly use Fal AI
172+
)
173+
174+
# Using another specific provider
175+
image_replicate = client.text_to_image(
176+
"Astronaut riding a horse",
177+
model="black-forest-labs/FLUX.1-schnell",
178+
provider="replicate" # Explicitly use Replicate
179+
)
180+
```
181+
182+
</hfoption>
183+
184+
<hfoption id="typescript">
185+
186+
```typescript
187+
import { InferenceClient } from "@huggingface/inference";
188+
189+
const client = new InferenceClient(process.env.HF_TOKEN);
190+
191+
// Using automatic provider selection (default)
192+
const imageAuto = await client.textToImage({
193+
model: "black-forest-labs/FLUX.1-schnell",
194+
inputs: "Astronaut riding a horse",
195+
provider: "auto", // This is the default
196+
parameters: { num_inference_steps: 5 },
197+
});
198+
199+
// Using a specific provider
200+
const imageFal = await client.textToImage({
201+
model: "black-forest-labs/FLUX.1-schnell",
202+
inputs: "Astronaut riding a horse",
203+
provider: "fal-ai", // Explicitly use Fal AI
204+
parameters: { num_inference_steps: 5 },
205+
});
206+
207+
// Using another specific provider
208+
const imageReplicate = await client.textToImage({
209+
model: "black-forest-labs/FLUX.1-schnell",
210+
inputs: "Astronaut riding a horse",
211+
provider: "replicate", // Explicitly use Replicate
212+
parameters: { num_inference_steps: 5 },
213+
});
214+
```
215+
216+
</hfoption>
217+
218+
</hfoptions>
219+
220+
### When to Use Each Approach
221+
222+
**Use `provider="auto"` when:**
223+
- You're just getting started with Inference Providers
224+
- You want the simplest setup and maximum reliability
225+
- You don't have specific infrastructure requirements
226+
- You want automatic failover if a provider is unavailable
227+
228+
**Use a specific provider when:**
229+
- You need consistent performance characteristics
230+
- You have specific billing or cost requirements
231+
- You want to test different providers' performance for your use case
232+
127233
## Next Steps
128234

129235
Now that you've seen how easy it is to use AI models, you might wonder:

0 commit comments

Comments
 (0)