LiteLLM v1+ - code cleanup

[ishaan-jaff]

Starting this discussion to remove unused functions / features. It's crucial this package remains lite

[ishaan-jaff]

We should remove support for Batch completions: https://docs.litellm.ai/docs/completion/batching

Why ?

• Afaik none of our users use this. We've had 0 issues for batch_completions in the last 2-3 months
• It's poorly maintained, we don't improve it
• For high load, users can use the Router / LiteLLM Proxy

[ishaan-jaff]

Use Cache Context Manager

We should update our docs to use Cache context managers as the primary way of using Caching

Key Change on this Quick Start Doc

https://docs.litellm.ai/docs/caching/redis_cache#quick-start
Use

litellm.enable_cache(
    type: Optional[Literal["local", "redis"]] = "local",
    host: Optional[str] = None,
    port: Optional[str] = None,
    password: Optional[str] = None,
    supported_call_types: Optional[
        List[Literal["completion", "acompletion", "embedding", "aembedding"]]
    ] = ["completion", "acompletion", "embedding", "aembedding"],
    **kwargs,
)

Instead of

litellm.cache = Cache(type="redis", host=<host>, port=<port>, password=<password>)

Why change ?

• I find this a lot simpler to use. It's easy to enable/disable/update the cache params
• It came as a suggestion from a user - they needed to make their own context manager
  (thanks for the help @eliorc)

[ishaan-jaff]

Rename litellm.completion_cost() -> litellm.get_request_cost()

Why?

• you can use it for completion/embedding cost

[krrishdholakia]

why not just litellm.cost() ?

[ishaan-jaff]

works !

[ishaan-jaff]

Deprecate: litellm.completion_with_config

Why?

• not on our docs
• users can use the proxy, router for everything litellm.completion_with_config was doing

[ishaan-jaff]

since it's already not on our docs we can remove this - we'll just need to ensure it's on our release notes