VideoContent support in the Gemini AI models (e.g. gemini-2.5-flash, gemini-2.5-pro)

[tskandratsyeu]

Hi all,
Firstly, thanks for creating this extension, it indeed makes the LLMs integration almost a breeze, at least in the text summarization and analysis tasks.

Alas, recently, I have stumbled across some impediment in implementing a video analysis tool with the ai-gemini integration. Based on these docs https://docs.quarkiverse.io/quarkus-langchain4j/dev/guide-passing-image.html, my understanding was that the Video should be processed in a pretty similar fashion. In the beginning, this was supported by the following PR #1325. Being fully confident that it works, I have updated to the 1.3.1 version of the extension. Needless to say, what my frustration was when I found out that actually the video content is being omitted during the UserMessage building under the hood of the generated AiService implementation class.

More concretely, by omitting video content I mean the prepareUserMessage method in following class https://github.com/quarkiverse/quarkus-langchain4j/blob/main/core/runtime/src/main/java/io/quarkiverse/langchain4j/runtime/aiservice/AiServiceMethodImplementationSupport.java#L960, where any video is not being processed at all. And the video is being wrapped by the TextContent instead, which seems not to be recognized by the model at all (at least in my tests).

Considering the above, my question is — is the VideoContent impossible to pass to ai-gemini models for the time being, or there is a different approach I am not aware of? Perhaps, if the former is the matter, I could prepare a PR with the video support extension, since it seems more or less straight-forward (considering the video is being prepared almost the same way as the Image and/or PDF Content).

Thanks in advance, and looking forward to hearing from you.

[geoand]

Can you please give #1881 a try?

[geoand]

Right again! Fixed

[tskandratsyeu]

Thanks, it works! I can see now that the video content is passed to the model API, and was even able to get the response out.
Although in the case of the gemini models, it works only with Video objects containing base64data, the url ones return Invalid argument error from the Gemini API.

While it works for one video, I would still really like the extension to have the possibility of supplying the model with several videos. I suppose it won't work if I supply my AiService interface method signature with two Video objects, right?
Like so:

@SystemMessage("...")
public interface MyAiService {
  @UserMessage("...")
  ResultDto doAnalysis(Video video1, Video video2);
}

UPD: just tried the implementation above and it just omits the second Video object unfortunately...

[geoand]

Thanks checking! I'll merge the PR. Please open a new issue asking for the ability to add multiple audio, video etc.

[tskandratsyeu]

Thanks, will do. I wonder when the extension version with the fix is expected to be published?

[geoand]

Thanks!

I will do a new release next week