add support for anthropic, ai21, bedrock, palm, cohere, togetherai, etc.

[krrishdholakia]

Hi @rajivpoddar @prthkms

Noticed you're calling the openai chat completions endpoint. I'm working on litellm (simple library to standardize LLM API Calls - https://github.com/BerriAI/litellm) and wanted to see how i could be helpful.

My PR adds support for new LLM providers (anthropic, aws bedrock, azure openai, replicate, ai21, etc.) by replacing ChatOpenai with ChatLitellm.

Curious if you find this useful?

Happy to add additional tests/documentation if the initial PR looks good.

[rajivpoddar]

Thanks @krrishdholakia.

Does LiteLLM have support for grammars? The issue we have faced with other LLMs is that it does not respond back in JSON. It would be quite useful to have llama.cpp style GBNF grammars.

[krrishdholakia]

Interesting - is there a known way of applying grammars to openai/hosted models? @rajivpoddar