Skip to content

vertexaiimageqa.mdx

Latest commit

 

History

History
79 lines (52 loc) · 2.5 KB

vertexaiimageqa.mdx

File metadata and controls

79 lines (52 loc) · 2.5 KB
title id slug description
VertexAIImageQA
vertexaiimageqa
/vertexaiimageqa
This component enables text generation (image captioning) using Google Vertex AI generative models.

VertexAIImageQA

This component enables text generation (image captioning) using Google Vertex AI generative models.
Mandatory run variables image: A ByteStream containing an image data

question: A string of a question about the image
Output variables replies: A list of strings containing answers generated by the model
API reference Google Vertex
GitHub link https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex

VertexAIImageQA supports the imagetext model.

Parameters Overview

VertexAIImageQA uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the official documentation.

Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.

You can find your project ID in the GCP resource manager or locally by running gcloud projects list in your terminal. For more info on the gcloud CLI, see its official documentation.

Usage

You need to install google-vertex-haystack package to use the VertexAIImageQA:

pip install google-vertex-haystack

On its own

Basic usage:

from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.generators.google_vertex import VertexAIImageQA

qa = VertexAIImageQA()

image = ByteStream.from_file_path("dog.jpg")

res = qa.run(image=image, question="What color is this dog")

print(res["replies"][0])

>>> white

You can also set the number of answers generated:

from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.generators.google_vertex import VertexAIImageQA

qa = VertexAIImageQA(
    number_of_results=3,
)
image = ByteStream.from_file_path("dog.jpg")

res = qa.run(image=image, question="Tell me something about this dog")

for answer in res["replies"]:
    print(answer)

>>> pomeranian
>>> white
>>> pomeranian puppy