Implementing Consistent Error Handling Across Providers

This document provides instructions for implementing consistent error handling across all provider classes in the `src/api/providers/` directory. The goal is to ensure that all providers format errors in a consistent way that can be properly processed by `Cline.ts`.

## Background

The `Cline.ts` component expects errors to be formatted in a specific way to properly handle different error types, especially rate limit errors (429). Currently, only the `vscode-lm.ts` and `anthropic.ts` providers format errors in this way. We need to update all other providers to follow the same pattern.

## Error Format

Errors should be formatted as JSON strings wrapped in Error objects with the following structure:

```javascript
throw new Error(JSON.stringify({
    status: 429, // HTTP status code
    message: "Rate limit exceeded", // Human-readable message
    error: {
        metadata: {
            raw: errorObj.message || "Too many requests, please try again later"
        }
    },
    errorDetails: [{ // Optional, for rate limit errors
        "@type": "type.googleapis.com/google.rpc.RetryInfo",
        retryDelay: "30s"
    }]
}));
```

## Implementation Steps

For each provider in the `src/api/providers/` directory, follow these steps:

1. **Identify the `createMessage` Method**:
   - Locate the `createMessage` method in the provider class.
   - This method should be an async generator function that returns an `ApiStream`.

2. **Add Error Handling**:
   - Wrap the main logic of the method in a try-catch block.
   - In the catch block, implement error handling as shown below.

3. **Format Errors Consistently**:
   - Add specific handling for different error types:
     - Rate limit errors (429)
     - Authentication errors (401)
     - Bad request errors (400)
     - Other errors (500)
   - Return errors as JSON strings in Error objects.

4. **Update Tests**:
   - Update the tests for each provider to verify the new error handling.
   - Add tests for different error scenarios.

## Error Handling Template

Here's a template for the error handling code to add to each provider:

```typescript
try {
    // Existing code for creating and processing the stream
} catch (error) {
    // Format errors in a consistent way
    console.error("Provider API error:", error);

    // Handle rate limit errors specifically
    const errorObj = error as any;
    if (errorObj.status === 429 ||
        (errorObj.message && errorObj.message.toLowerCase().includes('rate limit'))) {

        throw new Error(JSON.stringify({
            status: 429,
            message: "Rate limit exceeded",
            error: {
                metadata: {
                    raw: errorObj.message || "Too many requests, please try again later"
                }
            },
            errorDetails: [{
                "@type": "type.googleapis.com/google.rpc.RetryInfo",
                retryDelay: "30s" // Default retry delay if not provided
            }]
        }));
    }

    // Handle authentication errors
    if (errorObj.status === 401 ||
        (errorObj.message && errorObj.message.toLowerCase().includes('api key'))) {
        throw new Error(JSON.stringify({
            status: 401,
            message: "Authentication error",
            error: {
                metadata: {
                    raw: errorObj.message || "Invalid API key or unauthorized access"
                }
            }
        }));
    }

    // Handle bad request errors
    if (errorObj.status === 400 ||
        (errorObj.error && errorObj.error.type === "invalid_request_error")) {
        throw new Error(JSON.stringify({
            status: 400,
            message: "Bad request",
            error: {
                metadata: {
                    raw: errorObj.message || "Invalid request parameters",
                    param: errorObj.error?.param
                }
            }
        }));
    }

    // Handle other errors
    if (error instanceof Error) {
        throw new Error(JSON.stringify({
            status: errorObj.status || 500,
            message: error.message,
            error: {
                metadata: {
                    raw: error.message
                }
            }
        }));
    } else if (typeof error === "object" && error !== null) {
        const errorDetails = JSON.stringify(error, null, 2);
        throw new Error(JSON.stringify({
            status: errorObj.status || 500,
            message: errorObj.message || errorDetails,
            error: {
                metadata: {
                    raw: errorDetails
                }
            }
        }));
    } else {
        // Handle primitive errors or other unexpected types
        throw new Error(JSON.stringify({
            status: 500,
            message: String(error),
            error: {
                metadata: {
                    raw: String(error)
                }
            }
        }));
    }
}
```

## Provider-Specific Considerations

Some providers may have specific error types or patterns that need special handling. Here are some provider-specific considerations:

### OpenAI and OpenAI-Native

- Check for OpenAI-specific error types like `openai.APIError`.
- Look for rate limit indicators in the error response.

### Bedrock

- Bedrock may have AWS-specific error types.
- Look for throttling indicators in the error message.

### Vertex

- Vertex may have Google Cloud-specific error types.
- Check for quota exceeded errors.

### Gemini

- Similar to Vertex, check for Google-specific error patterns.

### Mistral, DeepSeek, Ollama, LMStudio

- These providers may have their own error formats.
- Adapt the error handling to match their specific patterns.

## Testing

For each provider, update or add tests to verify the error handling:

1. **Test Rate Limit Errors**:
   - Mock a 429 status code error.
   - Verify the error is formatted correctly.

2. **Test Authentication Errors**:
   - Mock a 401 status code error.
   - Verify the error is formatted correctly.

3. **Test Bad Request Errors**:
   - Mock a 400 status code error.
   - Verify the error is formatted correctly.

4. **Test Other Errors**:
   - Mock a generic error.
   - Verify the error is formatted correctly.

## Example Test

Here's an example test for verifying rate limit error handling:

```typescript
it("should handle rate limit errors with proper format", async () => {
    mockCreate.mockImplementationOnce(() => {
        const error = new Error("Rate limit exceeded");
        error.status = 429;
        throw error;
    });

    const stream = handler.createMessage(systemPrompt, messages);
    try {
        for await (const _ of stream) {
            // consume stream
        }
        fail("Should have thrown an error");
    } catch (error) {
        expect(error).toBeInstanceOf(Error);
        const parsedError = JSON.parse((error as Error).message);
        expect(parsedError.status).toBe(429);
        expect(parsedError.message).toBe("Rate limit exceeded");
        expect(parsedError.errorDetails[0]["@type"]).toBe("type.googleapis.com/google.rpc.RetryInfo");
    }
});
```

## Providers to Update

Here's a list of all providers that need to be updated:

1. ✅ `anthropic.ts` (already updated)
2. ✅ `vscode-lm.ts` (already updated)
3. ✅ `openai.ts` (already updated)
4. ✅ `openai-native.ts` (already updated)
5. ✅  `bedrock.ts` (already updated)
6. ✅ `vertex.ts`(already updated)
7. ✅ `gemini.ts`(already updated)
8. ✅ `mistral.ts` (already updated)
9. ✅ `deepseek.ts` (already updated)
10. ✅  `ollama.ts` (already updated)
11. ✅ `lmstudio.ts` (already updated)
12. ✅ `openrouter.ts` (already updated)
13. ✅ `glama.ts`
14. ✅ `unbound.ts`
15. ✅ `requesty.ts`
16. ✅ `human-relay.ts`
17. ✅ `fake-ai.ts`

## Conclusion

By implementing consistent error handling across all providers, we ensure that `Cline.ts` can properly handle errors from any provider, especially rate limit errors. This will improve the user experience by providing better error messages and enabling automatic retries when appropriate.

Author: bramburn

src/api/providers/__tests__/anthropic.createMessage.test.ts

@@ -0,0 +1,370 @@
+// npx jest src/api/providers/__tests__/anthropic.createMessage.test.ts
+
+import { AnthropicHandler } from "../anthropic"
+import { ApiHandlerOptions } from "../../../shared/api"
+import { Anthropic } from "@anthropic-ai/sdk"
+import { ApiStreamChunk } from "../../transform/stream"
+import { fail } from "assert"
+
+// Define custom error type for tests
+interface ApiError extends Error {
+	status?: number
+}
+
+// Mock Anthropic SDK
+jest.mock("@anthropic-ai/sdk", () => {
+	return {
+		Anthropic: jest.fn().mockImplementation(() => ({
+			messages: {
+				create: jest.fn(),
+			},
+		})),
+	}
+})
+
+describe("AnthropicHandler.createMessage", () => {
+	let handler: AnthropicHandler
+	let mockCreate: jest.Mock
+	const mockOptions: ApiHandlerOptions = {
+		apiKey: "test-api-key",
+		apiModelId: "claude-3-5-sonnet-20241022",
+	}
+	const systemPrompt = "You are a helpful assistant."
+	const messages: Anthropic.Messages.MessageParam[] = [
+		{
+			role: "user",
+			content: [{ type: "text" as const, text: "Hello" }],
+		},
+	]
+
+	beforeEach(() => {
+		jest.clearAllMocks()
+		handler = new AnthropicHandler(mockOptions)
+		mockCreate = handler["client"].messages.create as jest.Mock
+
+		// Default mock implementation for successful streaming
+		mockCreate.mockResolvedValue({
+			async *[Symbol.asyncIterator]() {
+				yield {
+					type: "message_start",
+					message: {
+						usage: {
+							input_tokens: 100,
+							output_tokens: 50,
+							cache_creation_input_tokens: 20,
+							cache_read_input_tokens: 10,
+						},
+					},
+				}
+				yield {
+					type: "content_block_start",
+					index: 0,
+					content_block: {
+						type: "text",
+						text: "Hello",
+					},
+				}
+				yield {
+					type: "content_block_delta",
+					delta: {
+						type: "text_delta",
+						text: " world",
+					},
+				}
+			},
+		})
+	})
+
+	it("should handle successful streaming", async () => {
+		const stream = handler.createMessage(systemPrompt, messages)
+		const chunks: ApiStreamChunk[] = []
+
+		for await (const chunk of stream) {
+			chunks.push(chunk)
+		}
+
+		expect(chunks.length).toBeGreaterThan(0)
+		expect(chunks[0]).toEqual({
+			type: "usage",
+			inputTokens: 100,
+			outputTokens: 50,
+			cacheWriteTokens: 20,
+			cacheReadTokens: 10,
+		})
+		expect(chunks[1]).toEqual({
+			type: "text",
+			text: "Hello",
+		})
+		expect(chunks[2]).toEqual({
+			type: "text",
+			text: " world",
+		})
+	})
+
+	it("should handle standard errors with proper format", async () => {
+		mockCreate.mockImplementationOnce(() => {
+			throw new Error("Standard error")
+		})
+
+		const stream = handler.createMessage(systemPrompt, messages)
+		try {
+			for await (const _ of stream) {
+				// consume stream
+			}
+			fail("Should have thrown an error")
+		} catch (error) {
+			expect(error).toBeInstanceOf(Error)
+			const parsedError = JSON.parse((error as Error).message)
+			expect(parsedError.status).toBe(500)
+			expect(parsedError.message).toBe("Standard error")
+			expect(parsedError.error.metadata.raw).toBe("Standard error")
+		}
+	})
+
+	it("should handle rate limit errors with proper format", async () => {
+		mockCreate.mockImplementationOnce(() => {
+			const error = new Error("Rate limit exceeded") as ApiError
+			error.status = 429
+			throw error
+		})
+
+		const stream = handler.createMessage(systemPrompt, messages)
+		try {
+			for await (const _ of stream) {
+				// consume stream
+			}
+			fail("Should have thrown an error")
+		} catch (error) {
+			expect(error).toBeInstanceOf(Error)
+			const parsedError = JSON.parse((error as Error).message)
+			expect(parsedError.status).toBe(429)
+			expect(parsedError.message).toBe("Rate limit exceeded")
+			expect(parsedError.errorDetails[0]["@type"]).toBe("type.googleapis.com/google.rpc.RetryInfo")
+		}
+	})
+
+	it("should handle rate limit errors with message text", async () => {
+		mockCreate.mockImplementationOnce(() => {
+			throw new Error("You have exceeded your rate limit")
+		})
+
+		const stream = handler.createMessage(systemPrompt, messages)
+		try {
+			for await (const _ of stream) {
+				// consume stream
+			}
+			fail("Should have thrown an error")
+		} catch (error) {
+			expect(error).toBeInstanceOf(Error)
+			const parsedError = JSON.parse((error as Error).message)
+			expect(parsedError.status).toBe(429)
+			expect(parsedError.message).toBe("Rate limit exceeded")
+		}
+	})
+
+	it("should handle network errors with proper format", async () => {
+		mockCreate.mockImplementationOnce(() => {
+			const error = new Error("Network error") as ApiError
+			error.status = 503
+			throw error
+		})
+
+		const stream = handler.createMessage(systemPrompt, messages)
+		try {
+			for await (const _ of stream) {
+				// consume stream
+			}
+			fail("Should have thrown an error")
+		} catch (error) {
+			expect(error).toBeInstanceOf(Error)
+			const parsedError = JSON.parse((error as Error).message)
+			expect(parsedError.status).toBe(503)
+			expect(parsedError.message).toBe("Network error")
+		}
+	})
+
+	it("should handle authentication errors with proper format", async () => {
+		mockCreate.mockImplementationOnce(() => {
+			const error = new Error("Invalid API key") as ApiError
+			error.status = 401
+			throw error
+		})
+
+		const stream = handler.createMessage(systemPrompt, messages)
+		try {
+			for await (const _ of stream) {
+				// consume stream
+			}
+			fail("Should have thrown an error")
+		} catch (error) {
+			expect(error).toBeInstanceOf(Error)
+			const parsedError = JSON.parse((error as Error).message)
+			expect(parsedError.status).toBe(401)
+			expect(parsedError.message).toBe("Authentication error")
+		}
+	})
+
+	it("should handle object errors with proper format", async () => {
+		mockCreate.mockImplementationOnce(() => {
+			const error = {
+				status: 400,
+				message: "Bad request",
+				error: {
+					type: "invalid_request_error",
+					param: "model",
+				},
+			}
+			// Throw as object directly to test object error handling
+			throw error
+		})
+
+		const stream = handler.createMessage(systemPrompt, messages)
+		try {
+			for await (const _ of stream) {
+				// consume stream
+			}
+			fail("Should have thrown an error")
+		} catch (error) {
+			expect(error).toBeInstanceOf(Error)
+			const parsedError = JSON.parse((error as Error).message)
+			expect(parsedError.status).toBe(400)
+			expect(parsedError.message).toBe("Bad request")
+			expect(parsedError.error.metadata.param).toBe("model")
+		}
+	})
+
+	it("should handle errors during streaming", async () => {
+		// Mock a stream that throws after yielding some data
+		mockCreate.mockResolvedValueOnce({
+			async *[Symbol.asyncIterator]() {
+				yield {
+					type: "message_start",
+					message: {
+						usage: {
+							input_tokens: 100,
+							output_tokens: 50,
+						},
+					},
+				}
+				yield {
+					type: "content_block_start",
+					index: 0,
+					content_block: {
+						type: "text",
+						text: "This is the beginning of a response",
+					},
+				}
+				// Simulate error during streaming
+				throw new Error("Stream interrupted")
+			},
+		})
+
+		const stream = handler.createMessage(systemPrompt, messages)
+		const chunks: ApiStreamChunk[] = []
+
+		try {
+			for await (const chunk of stream) {
+				chunks.push(chunk)
+			}
+			fail("Expected error to be thrown")
+		} catch (error) {
+			expect(error).toBeInstanceOf(Error)
+			// Parse the error message as JSON
+			const parsedError = JSON.parse((error as Error).message)
+			expect(parsedError.status).toBe(500)
+			expect(parsedError.message).toBe("Stream interrupted")
+			// Verify we got some chunks before the error
+			expect(chunks.length).toBeGreaterThan(0)
+		}
+	})
+
+	it("should handle thinking mode for supported models", async () => {
+		const thinkingHandler = new AnthropicHandler({
+			apiKey: "test-api-key",
+			apiModelId: "claude-3-7-sonnet-20250219:thinking",
+			modelMaxThinkingTokens: 16384,
+		})
+
+		const thinkingMockCreate = thinkingHandler["client"].messages.create as jest.Mock
+
+		// Mock a stream with thinking content
+		thinkingMockCreate.mockResolvedValueOnce({
+			async *[Symbol.asyncIterator]() {
+				yield {
+					type: "message_start",
+					message: {
+						usage: {
+							input_tokens: 100,
+							output_tokens: 50,
+						},
+					},
+				}
+				yield {
+					type: "content_block_start",
+					index: 0,
+					content_block: {
+						type: "thinking",
+						thinking: "Let me think about this...",
+					},
+				}
+				yield {
+					type: "content_block_delta",
+					delta: {
+						type: "thinking_delta",
+						thinking: " I need to consider all options.",
+					},
+				}
+				yield {
+					type: "content_block_start",
+					index: 1,
+					content_block: {
+						type: "text",
+						text: "Here's my answer:",
+					},
+				}
+				// We need to make sure the text is properly captured
+				// The issue is that the handler might be ignoring empty text deltas
+				// Let's add a non-empty text delta
+				yield {
+					type: "content_block_delta",
+					delta: {
+						type: "text_delta",
+						text: " Additional text",
+					},
+				}
+			},
+		})
+
+		const stream = thinkingHandler.createMessage(systemPrompt, messages)
+		const chunks: ApiStreamChunk[] = []
+
+		for await (const chunk of stream) {
+			chunks.push(chunk)
+		}
+
+		// Verify we got thinking chunks
+		const reasoningChunks = chunks.filter((chunk) => chunk.type === "reasoning")
+		expect(reasoningChunks.length).toBeGreaterThan(0)
+		expect(reasoningChunks[0].text).toBe("Let me think about this...")
+		expect(reasoningChunks[1].text).toBe(" I need to consider all options.")
+
+		// Verify we also got text chunks
+		const textChunks = chunks.filter((chunk) => chunk.type === "text")
+		expect(textChunks.length).toBeGreaterThan(0)
+
+		// The first text chunk is a newline because chunk.index > 0 in the handler
+		expect(textChunks[0].text).toBe("\n")
+		// The second text chunk is from content_block_start
+		expect(textChunks[1].text).toBe("Here's my answer:")
+		// The third text chunk is from content_block_delta
+		expect(textChunks[2].text).toBe(" Additional text")
+
+		// Verify the API was called with thinking parameter
+		expect(thinkingMockCreate).toHaveBeenCalledWith(
+			expect.objectContaining({
+				thinking: { type: "enabled", budget_tokens: 16384 },
+			}),
+			expect.anything(),
+		)
+	})
+})