fix: handle empty provider response in _convert_chat_completion (#803)

## Summary

Some providers (observed via OpenRouter -> OpenAI Model -> Azure) return
HTTP 200 with an effectively empty `ChatCompletion` object when they
fail to process a request. The existing code in
`_convert_chat_completion` then crashes with an unhandled `TypeError` -
not a typed `AnyLLMError` — so consumers cannot catch it cleanly.

## Bug

When a provider returns an empty response, the OpenAI SDK parses it
into:

```
ChatCompletion(id=None, choices=None, created=None, model=None, object='chat.completion', ...)
```

The guard `if not isinstance(response.created, int)` catches `None`
(since `None` is not an `int`), then calls `int(None)` which raises:

```
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
```

This turns a provider-side issue into an untyped exception that
propagates up the call stack.

## How we found it

While testing request size limits in our LLM proxy service, we sent a
**large (~49 MB) fake base64 image** to `openai/gpt-4o-mini` via
OpenRouter:

```python
import base64, httpx

# 49 MB of garbage bytes encoded as base64 (~65 MB encoded)
fake_image = base64.b64encode(b"Y" * 49_000_000).decode()

# note: instead of openrouter.ai here was our proxy url which using any-llm sdk
response = httpx.post(
    "https://openrouter.ai/api/v1/chat/completions",
    headers={"Authorization": "Bearer <KEY>"},
    json={
        "model": "openai/gpt-4o-mini",
        "messages": [{
            "role": "user",
            "content": [
                {"type": "text", "text": "Analyze this image"},
                {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{fake_image}"}},
            ],
        }],
        "max_tokens": 10,
    },
    timeout=120,
)
```

**What happened:** The request reached the backing provider (Azure via
OpenRouter). Small fake images get a clean `400 Bad Request ("Invalid
image data")`. But the 49 MB payload overwhelmed the provider backend
(~25s processing), and it returned a **malformed 200 OK with an empty
body** instead of a proper error.

The OpenAI SDK parsed this into a `ChatCompletion` with all fields
`None`. When `any_llm` tried to normalize this object, `int(None)`
crashed.

**This is non-deterministic** - the same request may get routed to a
different provider backend that returns a proper 400. But when it hits a
backend that chokes, the crash is 100% reproducible.

## Fix

Add an early guard clause in `_convert_chat_completion` that detects
completely empty responses (`id`, `choices`, and `model` all `None`) and
raises `ProviderError` with a clear message.

**Why `AND` (all three None), not `OR`?** Some providers may
legitimately return one field as `None` in non-standard responses.
Requiring all three ensures we only catch truly empty/broken responses.

---------

Co-authored-by: daavoo <daviddelaiglesiacastro@gmail.com>

Author: lanesket

src/any_llm/providers/openai/utils.py

@@ -5,6 +5,7 @@
 from openai.types.chat.chat_completion import ChatCompletion as OpenAIChatCompletion
 
 from any_llm.constants import REASONING_FIELD_NAMES
+from any_llm.exceptions import ProviderError
 from any_llm.logging import logger
 from any_llm.types.completion import ChatCompletion
 
@@ -58,6 +59,17 @@ def _convert_chat_completion(response: OpenAIChatCompletion) -> ChatCompletion:
             response.object,
         )
         response.object = "chat.completion"
+
+    # Detect completely empty responses (all required fields None).
+    # Some providers return 200 OK with an empty body on malformed input
+    # instead of a proper error. Fail fast with a clear message.
+    if response.id is None and response.choices is None and response.model is None:
+        msg = (
+            "Provider returned an empty response with no id, choices, or model. "
+            "This usually means the provider failed to process the request."
+        )
+        raise ProviderError(msg)
+
     if not isinstance(response.created, int):
         # Sambanova returns a float instead of an int.
         logger.warning(

src/any_llm/providers/together/utils.py

@@ -35,7 +35,7 @@ def _create_openai_chunk_from_together_chunk(together_chunk: TogetherChatComplet
                 logger.warning("Together delta_content missing 'content' attribute: %s", delta_content)
             content = getattr(delta_content, "content", None)
             if getattr(delta_content, "role", None):
-                role = cast("Literal['assistant', 'user', 'system']", delta_content.role)  # type: ignore[attr-defined]
+                role = cast("Literal['assistant', 'user', 'system']", delta_content.role)
             if hasattr(delta_content, "reasoning") and delta_content.reasoning:
                 reasoning = Reasoning(content=delta_content.reasoning)
 

tests/unit/providers/test_openai_utils.py

@@ -0,0 +1,48 @@
+import pytest
+from openai.types.chat.chat_completion import ChatCompletion as OpenAIChatCompletion
+
+from any_llm.exceptions import ProviderError
+from any_llm.providers.openai.utils import _convert_chat_completion
+
+
+def test_convert_chat_completion_with_empty_response() -> None:
+    # Simulating the malformed response described in the PR
+    # ChatCompletion(id=None, choices=None, created=None, model=None, object='chat.completion', ...)
+    openai_response = OpenAIChatCompletion.model_construct(
+        id=None,
+        choices=None,
+        created=None,
+        model=None,
+        object="chat.completion",
+    )
+
+    with pytest.raises(ProviderError) as exc_info:
+        _convert_chat_completion(openai_response)
+
+    assert "Provider returned an empty response" in str(exc_info.value)
+
+
+def test_convert_chat_completion_with_partial_none_response() -> None:
+    # If not all THREE (id, choices, model) are None, it should NOT raise ProviderError early.
+    # It might fail later if other required fields like 'created' are missing or invalid,
+    # but the specific guard being tested here should not trigger.
+    openai_response = OpenAIChatCompletion.model_construct(
+        id="test-id",
+        choices=None,
+        created=1234567890,
+        model=None,
+        object="chat.completion",
+    )
+
+    # In this case, it will fail later during ChatCompletion.model_validate(normalized)
+    # because 'choices' is None, or during _normalize_openai_dict_response if it expect choices to be a list.
+
+    # Actually _normalize_openai_dict_response handles choices=None:
+    # choices = response_dict.get("choices")
+    # if isinstance(choices, list): ...
+
+    # But ChatCompletion.model_validate(normalized) will fail because choices is required.
+    from pydantic import ValidationError
+
+    with pytest.raises(ValidationError):
+        _convert_chat_completion(openai_response)