-
Notifications
You must be signed in to change notification settings - Fork 187
Description
In chat_model/bedrock_converse.py, you define a function to go from langchain message format to Converse API format. In particular when dealing with multimodal request, there is the need to determine the format of the image or file. For this you are doing:
"format": block["mimeType"].split("/")[1]
(example here)
Now here is the problem, the list of MimeTypes can be found here:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/MIME_types/Common_types
https://www.iana.org/assignments/media-types/media-types.xhtml
For markdown:
Correct mime type:text/markdown
Resulting "format": "markdown"
Expected format: md
For xslx:
Correct mime type:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Resulting "format": "vnd.openxmlformats-officedocument.spreadsheetml.sheet"
Expected format: xslx
Same for docx, xls,...
The expected format from Converse API must be in the following enums:
For image: 'png'|'jpeg'|'gif'|'webp'
For documents: 'pdf'|'csv'|'doc'|'docx'|'xls'|'xlsx'|'html'|'txt'|'md'
It's lucky it's working for pdf, images as the mime types as the mime types are application/pdf, image/png, etc...