Opus 4 Broken Due to Expecting image/png

[sinabahram]

First, major thanks for this addon. Onto the bug.

Opus 4 returns and http 400 because it's being told to expect image/png but is presumably receiving jpeg or other mime types when shift+NVDA+Y is invoked. Exact message below.

error: HTTP Error 400: Bad Request. messages.0.content.0.image.source.base64.data: Image does not match the provided media type image/png

[cartertemm]

Hello @sinabahram,

Appreciate you opening an issue.

Unfortunately, try as I might, I can't reproduce this error from my end. In the Anthropic Vision docs, both JPEG and PNG mimetypes should work.

I do recall reading on HN that Anthropic was making some modifications to their API in light of increased usage, perhaps this was briefly broken in the process.

Can you try triggering a description again and see if you continue to receive the error? If you do

• Are you getting the same with any of the other models in the Claude family, or just 4 Opus?
• In what region are you located? I don't think they have gotten to the point of georestricting features, though stranger things have been known to happen, so I can try with a VPN.
• Are you on the latest version of the add-on?

Thanks again!

[sinabahram]

Major thanks for this response. Appreciate you.

Just want you to know my repro steps as this is still happening.

1. go to windos explorer where you have some .png and .jpeg files
2. hit ctrl+c on a .png file and then hit NVDA+shift+Y. Observe it works great.
3. then arrow to a .jpeg file, hit ctrl+C, then hit NVDA+Y. Observe it immediately returns the API error about expecting PNG.

The above seems consistent to me e.g. it complains it expects PNG files and when I give it one, it is happy and works but when I do not, it is not happy and doesn't work.