Skip to content

ConverseAPI format incorrectly constructed from MimeType #574

@JuR-0

Description

@JuR-0

In chat_model/bedrock_converse.py, you define a function to go from langchain message format to Converse API format. In particular when dealing with multimodal request, there is the need to determine the format of the image or file. For this you are doing:

"format": block["mimeType"].split("/")[1]

(example here)

Now here is the problem, the list of MimeTypes can be found here:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/MIME_types/Common_types
https://www.iana.org/assignments/media-types/media-types.xhtml

For markdown:
Correct mime type:text/markdown
Resulting "format": "markdown"
Expected format: md

For xslx:
Correct mime type:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Resulting "format": "vnd.openxmlformats-officedocument.spreadsheetml.sheet"
Expected format: xslx

Same for docx, xls,...

The expected format from Converse API must be in the following enums:
For image: 'png'|'jpeg'|'gif'|'webp'
For documents: 'pdf'|'csv'|'doc'|'docx'|'xls'|'xlsx'|'html'|'txt'|'md'

It's lucky it's working for pdf, images as the mime types as the mime types are application/pdf, image/png, etc...

Metadata

Metadata

Assignees

No one assigned

    Labels

    bedrockbugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions