Add @CacheResult to EmbeddingModel

[cmasantos]

I was wondering if we could add caching to embedding generation without having to wrap EmbeddingModel.java inside another CDI bean.

I have a use case where I need to generate embeddings for the same strings across multiple requests, so I’d like to cache them for a while. To do that, I’d wrap the embedding generation and add a cache there.

I’m not sure how that would work in terms of CDI, but it would be nice if we could somehow enable caching directly at the injection point of the EmbeddingModel interface.

[geoand]

That's interesting for sure! Unfortunately, defining caching at the injection point isn't a supported feature in Quarkus. I recommend opening an issue in the Quarkus repo

[cmasantos]

Thank you, I've created an issue in Quarkus: quarkusio/quarkus#49768

[jmartisk]

Hmm, I don't think this should be declared on injection points. If you have multiple injection points of the same embedding model, then each injection point could get a separate cache with different settings? That probably doesn't make sense. This is something that should be configured for an embedding model globally. It could be an interceptor that wraps the EmbeddingModel.embedAll method and, potentially returns cached values for some of the passed embeddings instead of requesting to compute new ones.

[geoand]

I think it's worth thinking about in Quarkus

[cmasantos]

then each injection point could get a separate cache with different settings?

That's a valid point, a config would also work and probably be more flexible.