Explore Image Embedding Drivers

[collindutter]

• I have read and agree to the contributing guidelines.

Is your feature request related to a problem? Please describe.
I can't use image embedding models like OpenAi's CLIP in Griptape.
Describe the solution you'd like
I want Embedding Drivers to support images.
Describe alternatives you've considered

• fastembed
• transformers

Additional context
The eventual goal is to use the embeddings with Vector Store Drivers which means I'd maybe need upsert_image_artifact and query_image methods. upsert_vector and query_vector may be sufficient for now.

[collindutter]

Up for discussion: do we feel that this is in scope for Griptape? And if so, should we make the distinction between Text Embedding Drivers and Image Embedding Drivers?

[shhlife]

What would the workaround/process be if we didn't include image embedding drivers?

[collindutter]

Basically what is described here, replacing the QdrantClient with QdrantVectorStoreDriver.

[shhlife]

Would we be able to do this with a LocalVectorStoreDriver as well? I feel like as long as we're doing text embeddings, and the future is multi-modal, we should handle image, audio, and video as well.

[shhlife]

I'm going to give it a try following the link you gave - but at a high level, I feel like we should offer the ability to embed things we have artifacts for.. at least common ones like images/audio/video especially as we go more multi-modal

[collindutter]

Tracking here