feat: add acceptedFileMimetypes to model config#2189
feat: add acceptedFileMimetypes to model config#2189tessaherself wants to merge 1 commit intohuggingface:mainfrom
Conversation
Allow models to declare which file MIME types they accept (e.g. "application/pdf", "image/*") via a new `acceptedFileMimetypes` field. The frontend merges these with existing multimodal MIME types to determine which upload options to show. This enables per-model file type support without coupling to a specific provider's file handling implementation. Refs: huggingface#482, huggingface#609, huggingface#1505, huggingface#1652 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a model declares `acceptedFileMimetypes` including "application/pdf", PDFs are now sent as OpenAI `file` content parts with base64 data instead of being wrapped in XML tags. This lets OpenAI (and compatible providers) process PDFs natively. Non-PDF files and models without `acceptedFileMimetypes` are unaffected — existing text extraction behavior is preserved. Depends on: huggingface#2189 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 182ee28286
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| ...(modelIsMultimodal | ||
| ? (currentModel.multimodalAcceptedMimetypes ?? [...IMAGE_MIME_ALLOWLIST_DEFAULT]) | ||
| : []), | ||
| ...(currentModel.acceptedFileMimetypes ?? []), |
There was a problem hiding this comment.
Including currentModel.acceptedFileMimetypes in the upload allowlist exposes file types (e.g. application/pdf) that the request pipeline does not actually forward to the model: prepareMessagesWithFiles only preserves images and text MIME types from TEXT_MIME_ALLOWLIST (src/lib/server/textGeneration/utils/prepareFiles.ts, prepareFiles). In this scenario users can successfully attach a file in the UI, but that attachment is silently omitted from the model input, leading to incorrect chat behavior.
Useful? React with 👍 / 👎.
1 participant
Summary
acceptedFileMimetypesfield to model config schema, allowing models to declare which file MIME types they accept (e.g.["application/pdf", "image/*"])acceptedFileMimetypeswith existingmultimodalAcceptedMimetypesto build the active MIME listThis is the foundation for native file handling per provider — models declare what they support, and endpoint adapters deliver files in the appropriate format.
Example config:
{ "name": "gpt-4o", "multimodal": true, "acceptedFileMimetypes": ["application/pdf"] }Related: #2188, #482, #609, #1505, #1652
Test plan
acceptedFileMimetypesbehaves as before (text + images only)acceptedFileMimetypes: ["application/pdf"]shows PDF in file picker🤖 Generated with Claude Code