Add Gemini(Vertex AI) Inline Data Part

[chanrute]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

I’d like to request a new feature to allow inlineData support for (Vertex AI) Gemini integration.

Currently, the JavaScript version provides a GeminiPartInlineData interface for sending PDFs or other binary data directly to Gemini.

ref: https://v03.api.js.langchain.com/interfaces/_langchain_google_vertexai.types.GeminiPartInlineData.html

Having a similar mechanism in Python would be extremely helpful for multi-modal use cases (e.g., attaching a PDF as inlineData) without resorting to workarounds like uploading or converting PDFs to images.

Motivation

Main motivation is that direct inlineData support simplifies PDF or multi-page document analysis with Vertex AI Gemini.

The current alternatives—such as cloud uploads or splitting PDFs into images—add overhead and complexity. A built-in inlineData feature in Python would ensure consistency between the JavaScript and Python libraries, saving developers time and effort.

Proposal (If applicable)

I propose implementing an equivalent to GeminiPartInlineData .Vertex AI Gemini reference is here.

This would introduce a new message part (e.g., "inlineData": {"mimeType": "...", "data": "..."}" ) so PDFs and similar content can be sent directly to Gemini.
This approach provides parity with the JS interface and makes multi-modal tasks much easier.