LLM proxy

[xtrasmal]

Hi folks,

Has any of you tried to build a proxy for LLM's? Something like LiteLLM, but in Elixir?

[Puremag1c]

its not dificult to add proxy to "req" calls, and add to LLM Schema fields like use_proxy: true, type, port.. etc,
but need some experience to catch and fallback issues with connection in current app workflow.

im waiting this feature too, and if this is not done within a year, I will try to do it myself)
but I'm afraid that this will be a terrible implementation, so I'm waiting for more experienced ones =)