[Bug]: RAG CLI Chat does not work with GoogleGenAI

[lehoang318]

Bug Description

RAG CLI Chat does not work with GoogleGenAI.

• In details
  • GoogleGenAI._stream_chat() call asyncio.run

# line 403
next_msg, chat_kwargs, file_api_names = asyncio.run(
    prepare_chat_params(
        self.model, messages, self.file_mode, self._client, **params
    )
)

• RagCLI.start_chat_repl() is defined as async

async def start_chat_repl(self) -> None:
    """
    Start a REPL for chatting with the agent.
    """

• GoogleGenAI._stream_chat() is in the call flow from RagCLI.start_chat_repl(), therefore, RuntimeError shall be raised.

Version

v0.14.15

Steps to Reproduce

• Step 1: create a python wrapper for RAG CLI in which Settings.llm is set to GoogleGenAI
• Step 2: ingest several documents
• Step 3: run python rag_cli_wrapper.py --chat

Relevant Logs/Tracbacks

[dosubot]

Hi @lehoang318! I'm Dosu and I’m helping the llama_index team.

Thanks for the detailed bug report! I investigated this and found some relevant context:

The issue you're describing with asyncio.run() conflicts has been a known pattern in LlamaIndex. PR #20240 previously fixed a similar issue in GoogleGenAI's structured_predict method by replacing asyncio.run() with asyncio_run from llama_index.core.async_utils, which properly handles nested event loop contexts.

Looking at the current _stream_chat() implementation, it appears the code structure may have changed since v0.14.15. Could you share the actual traceback you're seeing? That would help confirm exactly where the asyncio.run() call is happening in your version.

If you're stuck on v0.14.15, a potential workaround would be to replace the asyncio.run() call with asyncio_run from llama_index.core.async_utils, which handles the nested event loop case gracefully. Upgrading to a newer version may also resolve the issue if subsequent releases applied similar fixes to the streaming chat methods.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[lehoang318]

@dosu
v0.14.15 is the latest version could be installed from pip.
Besides, I check the main branch, the issue is still there.
It seems that PR#20240 was not merged to main branch.