Support LiteLLM pip package https://github.com/BerriAI/litellm

[ishaan-jaff]

Starting a disucssion to add litellm python package support on Cloudflare Python workers https://github.com/BerriAI/litellm

[mitch-36]

+1

[Manouchehri]

Right now requirements.txt isn't supported by Python Workers.

https://developers.cloudflare.com/workers/languages/python/

Here's the list of supported packages:

https://developers.cloudflare.com/workers/languages/python/packages#supported-packages

Would LiteLLM work properly with that super restricted list?

[ishaan-jaff]

Having initial support for LiteLLM + OpenAI / Azure would be a good start

Would LiteLLM work properly with that super restricted list?

Yes - but LiteLLM would not be able to support VertexAI, Bedrock Models

[krrishdholakia]

We can move to making the calls to bedrock + vertex via http - this should loosen the requirement there. @Manouchehri

[krrishdholakia]

bedrock + vertex calls now use httpx. Still need the core packages for handling auth though.

[rawwerks]

+100 for adding litellm!

[krrishdholakia]

cc: @AdityaAtulTewari @hoodmane @fhanau

[sdawka]

Wow it's been a year! Does cloudflare not care about python workers, or is this a really hard problem? (If the latter, knowing why might be a learning opportunity in case anyone obliges.)

[rawwerks]

i think it's probably a combination of security and business priorities.

[dom96]

We're currently experimenting with a vendoring approach which should support just about any package, maybe you could give it a try with LiteLLM? Here is an example repo you can use as a base: https://github.com/danlapid/python-workers-mcp. We'd love your thoughts on what works well and what doesn't work well with this.