Allow for local vectorizers and possibly generators

[rjalexa]

I would love to use Kragen but the data I must handle is sensitive and cannot leave premises (no OpenAI or other external APIs).

I have my own embedder container to build vectors for both the document and the queries and same for the generative function.

Could Kragen provide an interface for such local models?

Cheers from Italy and thank you

[jay-m-dev]

Hello @rjalexa, thank you for the feedback!
We are currently developing several optimizations for KRAGEN, some of which are:

1. Merging the Vectorization and Vector DB upload steps to make it more modular.
   • Vectorization will be performed during uploads to the Vector store using Vectorizer modules as handled by the Weaviate Vector Database. (e.g. the text2vec-transformers module will allow you to use local models for embedding)
2. We will work on implementing the ability to use a local model for the generative function.

These features are currently planned and will be implemented in the near future.