Tool calling / instructor

[twardoch]

https://github.com/BerriAI/litellm/blob/main/litellm/tests/test_class.py shows some commented out code that references instructor: https://jxnl.github.io/instructor/

It would be great if litellm metadata (where you store the cost etc.) had some entry about known support of a given model for structured output, and allowed querying this, so that code could easily apply the right type of instructor patch. Basically a better way to interface with instructor — that would work also in scenarios like fallback models.

[krrishdholakia]

can you share some mock code for this? @twardoch