litellm.APIConnectionError: APIConnectionError: OpenAIException - 'async_generator' object has no attribute 'get'

[deepakdeore2004]

litellm gives below error when stream is set in curl

curl -sX POST https://litellm.example.com/v1/completions -H 'Authorization: Bearer sk-xxxxx' -H "Content-Type: application/json" -d '{
          "model": "llama31-instruct",
          "stream": "true",
          "prompt": "Hi, who is the CEO of google",
          "max_tokens": 40
         }'

{"error":{"message":"litellm.APIConnectionError: APIConnectionError: OpenAIException - 'async_generator' object has no attribute 'get'\nReceived Model Group=llama31-instruct\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"500"}}

below is configmap

    - litellm_params:
        api_base: http://llama31/v1
        api_key: sk-dummy
        model: openai/Llama-3.1-8B-Instruct
      model_name: llama31-instruct

litellm litellm-database:main-v1.48.16

calling bckend vLLM model directly returns streaming response but same command fails with litellm

what can be wrong here?

here are some debug logs:

04:47:02 - LiteLLM Router:DEBUG: router.py:2947 - Trying to fallback b/w models
04:47:02 - LiteLLM Proxy:ERROR: proxy_server.py:3539 - litellm.proxy.proxy_server.completion(): Exception occured - litellm.APIConnectionError: APIConnectionError: OpenAIException - 'async_generator' object has no attribute 'get'
Received Model Group=llama31-instruct
Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/litellm/main.py", line 4048, in atext_completion
    text_completion_response["id"] = response.get("id", None)
                                     ^^^^^^^^^^^^
AttributeError: 'async_generator' object has no attribute 'get'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 3436, in completion
    response = await llm_call
               ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 1911, in atext_completion
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 1899, in atext_completion
    response = await self.async_function_with_fallbacks(**kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 3079, in async_function_with_fallbacks
    raise original_exception
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2933, in async_function_with_fallbacks
    response = await self.async_function_with_retries(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 3209, in async_function_with_retries
    raise original_exception
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 3123, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 3216, in make_call
    response = await original_function(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2000, in _atext_completion
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 1987, in _atext_completion
    response = await response
               ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 1582, in wrapper_async
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 1397, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/main.py", line 4065, in atext_completion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2136, in exception_type
    raise e  # it's already mapped
    ^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 430, in exception_type
    raise APIConnectionError(
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: APIConnectionError: OpenAIException - 'async_generator' object has no attribute 'get'
Received Model Group=llama31-instruct
Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
INFO:     10.14.7.143:36522 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error