Skip to content

Commit 335ef4d

Browse files
SBrandeisCopilotjulien-c
authored
[Inference] Improve error handling (huggingface#1504)
Introduces new error types to allow narrower error handling downstream Those errors hold more context, including the underlying http request / response when required This is a breakin change because we remove the `InferenceOutputError` type and change the error types thrown by the public methods. --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Julien Chaumond <[email protected]>
1 parent 615c348 commit 335ef4d

25 files changed

+747
-156
lines changed

‎packages/inference/README.md‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,108 @@ await textGeneration({
120120

121121
This will enable tree-shaking by your bundler.
122122

123+
### Error handling
124+
125+
The inference package provides specific error types to help you handle different error scenarios effectively.
126+
127+
#### Error Types
128+
129+
The package defines several error types that extend the base `Error` class:
130+
131+
- `InferenceClientError`: Base error class for all Hugging Face Inference errors
132+
- `InferenceClientInputError`: Thrown when there are issues with input parameters
133+
- `InferenceClientProviderApiError`: Thrown when there are API-level errors from providers
134+
- `InferenceClientHubApiError`: Thrown when there are API-levels errors from the Hugging Face Hub
135+
- `InferenceClientProviderOutputError`: Thrown when there are issues with providers' API responses format
136+
137+
### Example Usage
138+
139+
```typescript
140+
import { InferenceClient } from "@huggingface/inference";
141+
import {
142+
InferenceClientError,
143+
InferenceClientProviderApiError,
144+
InferenceClientProviderOutputError,
145+
InferenceClientHubApiError,
146+
} from "@huggingface/inference";
147+
148+
const client = new InferenceClient();
149+
150+
try {
151+
const result = await client.textGeneration({
152+
model: "gpt2",
153+
inputs: "Hello, I'm a language model",
154+
});
155+
} catch (error) {
156+
if (error instanceof InferenceClientProviderApiError) {
157+
// Handle API errors (e.g., rate limits, authentication issues)
158+
console.error("Provider API Error:", error.message);
159+
console.error("HTTP Request details:", error.request);
160+
console.error("HTTP Response details:", error.response);
161+
if (error instanceof InferenceClientHubApiError) {
162+
// Handle API errors (e.g., rate limits, authentication issues)
163+
console.error("Hub API Error:", error.message);
164+
console.error("HTTP Request details:", error.request);
165+
console.error("HTTP Response details:", error.response);
166+
} else if (error instanceof InferenceClientProviderOutputError) {
167+
// Handle malformed responses from providers
168+
console.error("Provider Output Error:", error.message);
169+
} else if (error instanceof InferenceClientInputError) {
170+
// Handle invalid input parameters
171+
console.error("Input Error:", error.message);
172+
} else {
173+
// Handle unexpected errors
174+
console.error("Unexpected error:", error);
175+
}
176+
}
177+
178+
/// Catch all errors from @huggingface/inference
179+
try {
180+
const result = await client.textGeneration({
181+
model: "gpt2",
182+
inputs: "Hello, I'm a language model",
183+
});
184+
} catch (error) {
185+
if (error instanceof InferenceClientError) {
186+
// Handle errors from @huggingface/inference
187+
console.error("Error from InferenceClient:", error);
188+
} else {
189+
// Handle unexpected errors
190+
console.error("Unexpected error:", error);
191+
}
192+
}
193+
```
194+
195+
### Error Details
196+
197+
#### InferenceClientProviderApiError
198+
199+
This error occurs when there are issues with the API request when performing inference at the selected provider.
200+
201+
It has several properties:
202+
- `message`: A descriptive error message
203+
- `request`: Details about the failed request (URL, method, headers)
204+
- `response`: Response details including status code and body
205+
206+
#### InferenceClientHubApiError
207+
208+
This error occurs when there are issues with the API request when requesting the Hugging Face Hub API.
209+
210+
It has several properties:
211+
- `message`: A descriptive error message
212+
- `request`: Details about the failed request (URL, method, headers)
213+
- `response`: Response details including status code and body
214+
215+
216+
#### InferenceClientProviderOutputError
217+
218+
This error occurs when a provider returns a response in an unexpected format.
219+
220+
#### InferenceClientInputError
221+
222+
This error occurs when input parameters are invalid or missing. The error message describes what's wrong with the input.
223+
224+
123225
### Natural Language Processing
124226
125227
#### Text Generation

‎packages/inference/src/errors.ts‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
import type { JsonObject } from "./vendor/type-fest/basic.js";
2+
3+
/**
4+
* Base class for all inference-related errors.
5+
*/
6+
export abstract class InferenceClientError extends Error {
7+
constructor(message: string) {
8+
super(message);
9+
this.name = "InferenceClientError";
10+
}
11+
}
12+
13+
export class InferenceClientInputError extends InferenceClientError {
14+
constructor(message: string) {
15+
super(message);
16+
this.name = "InputError";
17+
}
18+
}
19+
20+
interface HttpRequest {
21+
url: string;
22+
method: string;
23+
headers?: Record<string, string>;
24+
body?: JsonObject;
25+
}
26+
27+
interface HttpResponse {
28+
requestId: string;
29+
status: number;
30+
body: JsonObject | string;
31+
}
32+
33+
abstract class InferenceClientHttpRequestError extends InferenceClientError {
34+
httpRequest: HttpRequest;
35+
httpResponse: HttpResponse;
36+
constructor(message: string, httpRequest: HttpRequest, httpResponse: HttpResponse) {
37+
super(message);
38+
this.httpRequest = {
39+
...httpRequest,
40+
...(httpRequest.headers
41+
? {
42+
headers: {
43+
...httpRequest.headers,
44+
...("Authorization" in httpRequest.headers ? { Authorization: `Bearer [redacted]` } : undefined),
45+
/// redact authentication in the request headers
46+
},
47+
}
48+
: undefined),
49+
};
50+
this.httpResponse = httpResponse;
51+
}
52+
}
53+
54+
/**
55+
* Thrown when the HTTP request to the provider fails, e.g. due to API issues or server errors.
56+
*/
57+
export class InferenceClientProviderApiError extends InferenceClientHttpRequestError {
58+
constructor(message: string, httpRequest: HttpRequest, httpResponse: HttpResponse) {
59+
super(message, httpRequest, httpResponse);
60+
this.name = "ProviderApiError";
61+
}
62+
}
63+
64+
/**
65+
* Thrown when the HTTP request to the hub fails, e.g. due to API issues or server errors.
66+
*/
67+
export class InferenceClientHubApiError extends InferenceClientHttpRequestError {
68+
constructor(message: string, httpRequest: HttpRequest, httpResponse: HttpResponse) {
69+
super(message, httpRequest, httpResponse);
70+
this.name = "HubApiError";
71+
}
72+
}
73+
74+
/**
75+
* Thrown when the inference output returned by the provider is invalid / does not match the expectations
76+
*/
77+
export class InferenceClientProviderOutputError extends InferenceClientError {
78+
constructor(message: string) {
79+
super(message);
80+
this.name = "ProviderOutputError";
81+
}
82+
}

‎packages/inference/src/index.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
export { InferenceClient, InferenceClientEndpoint, HfInference } from "./InferenceClient.js";
2-
export { InferenceOutputError } from "./lib/InferenceOutputError.js";
2+
export * from "./errors.js";
33
export * from "./types.js";
44
export * from "./tasks/index.js";
55
import * as snippets from "./snippets/index.js";

‎packages/inference/src/lib/InferenceOutputError.ts‎

Lines changed: 0 additions & 8 deletions
This file was deleted.

‎packages/inference/src/lib/getInferenceProviderMapping.ts‎

Lines changed: 42 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import { HARDCODED_MODEL_INFERENCE_MAPPING } from "../providers/consts.js";
44
import { EQUIVALENT_SENTENCE_TRANSFORMERS_TASKS } from "../providers/hf-inference.js";
55
import type { InferenceProvider, InferenceProviderOrPolicy, ModelId } from "../types.js";
66
import { typedInclude } from "../utils/typedInclude.js";
7+
import { InferenceClientHubApiError, InferenceClientInputError } from "../errors.js";
78

89
export const inferenceProviderMappingCache = new Map<ModelId, InferenceProviderMapping>();
910

@@ -32,27 +33,46 @@ export async function fetchInferenceProviderMappingForModel(
3233
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
3334
inferenceProviderMapping = inferenceProviderMappingCache.get(modelId)!;
3435
} else {
35-
const resp = await (options?.fetch ?? fetch)(
36-
`${HF_HUB_URL}/api/models/${modelId}?expand[]=inferenceProviderMapping`,
37-
{
38-
headers: accessToken?.startsWith("hf_") ? { Authorization: `Bearer ${accessToken}` } : {},
36+
const url = `${HF_HUB_URL}/api/models/${modelId}?expand[]=inferenceProviderMapping`;
37+
const resp = await (options?.fetch ?? fetch)(url, {
38+
headers: accessToken?.startsWith("hf_") ? { Authorization: `Bearer ${accessToken}` } : {},
39+
});
40+
if (!resp.ok) {
41+
if (resp.headers.get("Content-Type")?.startsWith("application/json")) {
42+
const error = await resp.json();
43+
if ("error" in error && typeof error.error === "string") {
44+
throw new InferenceClientHubApiError(
45+
`Failed to fetch inference provider mapping for model ${modelId}: ${error.error}`,
46+
{ url, method: "GET" },
47+
{ requestId: resp.headers.get("x-request-id") ?? "", status: resp.status, body: error }
48+
);
49+
}
50+
} else {
51+
throw new InferenceClientHubApiError(
52+
`Failed to fetch inference provider mapping for model ${modelId}`,
53+
{ url, method: "GET" },
54+
{ requestId: resp.headers.get("x-request-id") ?? "", status: resp.status, body: await resp.text() }
55+
);
3956
}
40-
);
41-
if (resp.status === 404) {
42-
throw new Error(`Model ${modelId} does not exist`);
4357
}
44-
inferenceProviderMapping = await resp
45-
.json()
46-
.then((json) => json.inferenceProviderMapping)
47-
.catch(() => null);
48-
49-
if (inferenceProviderMapping) {
50-
inferenceProviderMappingCache.set(modelId, inferenceProviderMapping);
58+
let payload: { inferenceProviderMapping?: InferenceProviderMapping } | null = null;
59+
try {
60+
payload = await resp.json();
61+
} catch {
62+
throw new InferenceClientHubApiError(
63+
`Failed to fetch inference provider mapping for model ${modelId}: malformed API response, invalid JSON`,
64+
{ url, method: "GET" },
65+
{ requestId: resp.headers.get("x-request-id") ?? "", status: resp.status, body: await resp.text() }
66+
);
5167
}
52-
}
53-
54-
if (!inferenceProviderMapping) {
55-
throw new Error(`We have not been able to find inference provider information for model ${modelId}.`);
68+
if (!payload?.inferenceProviderMapping) {
69+
throw new InferenceClientHubApiError(
70+
`We have not been able to find inference provider information for model ${modelId}.`,
71+
{ url, method: "GET" },
72+
{ requestId: resp.headers.get("x-request-id") ?? "", status: resp.status, body: await resp.text() }
73+
);
74+
}
75+
inferenceProviderMapping = payload.inferenceProviderMapping;
5676
}
5777
return inferenceProviderMapping;
5878
}
@@ -83,7 +103,7 @@ export async function getInferenceProviderMapping(
83103
? EQUIVALENT_SENTENCE_TRANSFORMERS_TASKS
84104
: [params.task];
85105
if (!typedInclude(equivalentTasks, providerMapping.task)) {
86-
throw new Error(
106+
throw new InferenceClientInputError(
87107
`Model ${params.modelId} is not supported for task ${params.task} and provider ${params.provider}. Supported task: ${providerMapping.task}.`
88108
);
89109
}
@@ -104,7 +124,7 @@ export async function resolveProvider(
104124
): Promise<InferenceProvider> {
105125
if (endpointUrl) {
106126
if (provider) {
107-
throw new Error("Specifying both endpointUrl and provider is not supported.");
127+
throw new InferenceClientInputError("Specifying both endpointUrl and provider is not supported.");
108128
}
109129
/// Defaulting to hf-inference helpers / API
110130
return "hf-inference";
@@ -117,13 +137,13 @@ export async function resolveProvider(
117137
}
118138
if (provider === "auto") {
119139
if (!modelId) {
120-
throw new Error("Specifying a model is required when provider is 'auto'");
140+
throw new InferenceClientInputError("Specifying a model is required when provider is 'auto'");
121141
}
122142
const inferenceProviderMapping = await fetchInferenceProviderMappingForModel(modelId);
123143
provider = Object.keys(inferenceProviderMapping)[0] as InferenceProvider | undefined;
124144
}
125145
if (!provider) {
126-
throw new Error(`No Inference Provider available for model ${modelId}.`);
146+
throw new InferenceClientInputError(`No Inference Provider available for model ${modelId}.`);
127147
}
128148
return provider;
129149
}

‎packages/inference/src/lib/getProviderHelper.ts‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ import * as Replicate from "../providers/replicate.js";
4848
import * as Sambanova from "../providers/sambanova.js";
4949
import * as Together from "../providers/together.js";
5050
import type { InferenceProvider, InferenceProviderOrPolicy, InferenceTask } from "../types.js";
51+
import { InferenceClientInputError } from "../errors.js";
5152

5253
export const PROVIDERS: Record<InferenceProvider, Partial<Record<InferenceTask, TaskProviderHelper>>> = {
5354
"black-forest-labs": {
@@ -281,14 +282,18 @@ export function getProviderHelper(
281282
return new HFInference.HFInferenceTask();
282283
}
283284
if (!task) {
284-
throw new Error("you need to provide a task name when using an external provider, e.g. 'text-to-image'");
285+
throw new InferenceClientInputError(
286+
"you need to provide a task name when using an external provider, e.g. 'text-to-image'"
287+
);
285288
}
286289
if (!(provider in PROVIDERS)) {
287-
throw new Error(`Provider '${provider}' not supported. Available providers: ${Object.keys(PROVIDERS)}`);
290+
throw new InferenceClientInputError(
291+
`Provider '${provider}' not supported. Available providers: ${Object.keys(PROVIDERS)}`
292+
);
288293
}
289294
const providerTasks = PROVIDERS[provider];
290295
if (!providerTasks || !(task in providerTasks)) {
291-
throw new Error(
296+
throw new InferenceClientInputError(
292297
`Task '${task}' not supported for provider '${provider}'. Available tasks: ${Object.keys(providerTasks ?? {})}`
293298
);
294299
}

0 commit comments

Comments
 (0)