Instructor with llmlite doenst support vision for Claude 3 Haiku and Sonnet models

[enoch3712]

Hello all!

I have done some long testing and I concluded the following:

#claude-3-opus-20240229 - ok
#claude-3-haiku-20240307 - not ok
#claude-3-sonnet-20240229 - not ok
 resp = instructor.from_litellm(litellm.completion).chat.completions.create(
                model='claude-3-opus-20240229',
                messages=[
                    {
                        "role": "user",
                        "content": [
                            {
                                "type": "text",
                                "text": "What’s in this image?"
                            },
                            {
                                "type": "image_url",
                                "image_url": {
                                    "url": "data:image/jpeg;base64," + encode_image(file)
                                }
                            }
                        ]
                    }
                ],
                response_model=response_model,
            )

The source of models comes from this page:
https://docs.litellm.ai/docs/providers/anthropic

The error is always this one:

Thank you,

[jxnl]

1. its that a instructor issue or a anthropic / litellm issue?
2. The error is calling out the response model, but you don't include this in the example.

[enoch3712]

@jxnl I will close this question. Its better to follow this issue:

#617 (comment)