Is it possible to use CLIP in node-lama-cpp for embeddings, and if so, how?

[mau-io]

I'm currently working on a project where I need to generate embeddings for images, and I'm interested in exploring the capabilities of CLIP (Contrastive Language-Image Pretraining) for this purpose. I came across node-llama-cpp and was wondering if it supports CLIP directly..

Could anyone provide guidance on whether it's possible to use CLIP with node-llama-cpp? If direct integration is not possible, are there any suggested alternatives or workarounds to achieve similar embeddings functionality?

[giladgd]

The support for this in llama.cpp is not yet standardized enough to add robust support for it in node-llama-cpp,
but it is planned and will be added in the future when it'll be practical to evaluate embeddings together with tokens in llama.cpp.

You can watch #88 to get updated when it's available in node-llama-cpp.