Use Bedrock embedding model via litellm?

[gmerritt]

I would like to use the AWS Bedrock embedding model amazon.titan-embed-text-v2:0 via LiteLLM, not directly via Bedrock.

Is this configurable via rag_api & LibreChat settings? Or would it require custom modification of the rag_api application?

Thanks!

[gmerritt]

This environment variable combination works for me!

EMBEDDINGS_PROVIDER=openai
EMBEDDINGS_MODEL=amazon.titan-embed-text-v2:0
RAG_OPENAI_API_KEY=sk-<litellm_key>
RAG_OPENAI_BASEURL=https://my.litellm.hostname/v1

The "trick" was wrapping my head around LiteLLM standardizing to present everything as OpenAI...even if (as in this example) the true endpoint is Bedrock.

[rromanchuk]

@gmerritt thanks for coming back to update this

Have you had a chance to look at https://docs.litellm.ai/docs/providers/bedrock_vector_store at least using the retrieval aspects LibreChat's perspective?