Add support for non stream API mode for OpenAI chat provider

[ikappaki]

To upvote this issue, give it a thumbs up. See this list for the most upvoted issues.

Currently, eca only appears to support the OpenAI provider in stream mode. However, some environments do not support streaming at all, which makes it impossible to use the OpenAI provider in those contexts.

At present, setting the stream option to false results in empty responses, indicating that non stream mode is not yet supported.

Could you please consider adding support for non stream API mode for OpenAI?

Thank you

[ericdallo]

@ikappaki when I started ECA I didn't mean to support non stream APIs, but I think this might be achievable, I will take a look and consider it.

Can you describe if you have interest in openai responses or chat API?

[ikappaki]

Hi @ericdallo, thanks for considering! I'm interested in the chat/completions API endpoint please.

[ericdallo]

@ikappaki I have something almost finished in the non-stream branch in case you wanna take a look, it's only missing small fixes regarding threads blocking.

[ikappaki]

Hi @ericdallo, I took a quick look at the branch, and it seems there’s a logic error in how the code handles the no-stream response. The prompt! function returns the response as a map, but sync-prompt! attempts to dereference it as an atom. Since it’s just a map, this causes an exception that isn’t caught, leaving the ECA chat window hanging with no indication of the error.

Here's a flowstorm screenshot showing prompt! (in green) just about to return a map value (clojure.lang.PersistentArrayMap visible on the top right):

The response was returned from base-chat-request! below:

I’m not yet familiar enough with the code to determine the correct handling here. Could you please have a look?

thanks

[ericdallo]

Ah I'm sorry, it's still not finished, I'm aware and fixed that, adding support for tools now, I will let you know when I have something testable

[ericdallo]

I pushed some improvements, it kinda works but failling the tool call loop, I need to think more how to make it loopable since it's sync, the code is not ideal yet.

[ericdallo]

@ikappaki I think I fixed the loop! still need to fix tests, but a hand helping test would be nice!

[ikappaki]

HI @ericdallo, thanks for completing the task 🙌I tested both the previous and latest commits, and everything works in the example session I tried. I’ll take a look at the test failures next and will reach out if I get stuck or need your input.

[ericdallo]

I will fix the test failures and merge the branch, don't worry :)
Thank you for testing!