Supercharge the RAG API with an option for advanced Database backends

[Dual-0]

Hi everybody,

first of all, thank you for creating such an application. I'm currently migrating my LibreChat stack from a podman environment to Kubernetes, managed declaratively with FluxCD (GitOps). The last two days I'm focusing on optimizing the RAG API deployment.
My current goal is to move beyond the default pgvector setup to a more resilient and powerful database architecture. On my way I hit some limitations in the librechat-rag-api Helm chart. So I want to share my setup, the advantages, and a proposal for the chart to unlock a new level of performance and manageability for interested users.

A Modern, High-Performance Vector Database

Based on what I've learned so far about vector databases, the default Bitnami pgvector chart is great for getting started, but there could be something better and more robust for advanced environment. Inspired by this blog post I had to look into deploying the VectorChord extension, which is meant to be a powerful alternative to pgvector with a massive leap forward in cost-efficiency and performance for large-scale vector searches.

The beauty of this is that VectorChord isn't just for Kubernetes. It's a PostgreSQL extension. Anyone can run a high-performance VectorChord database using Docker or Podman on a VM, like I do before. The core requirement is that the LibreChat RAG API needs to be able to connect to it... more on that later.

Example: CloudNativePG with VectorChord on Kubernetes

In my Kubernetes environment, I use the CloudNativePG (CNPG) Operator to manage my VectorChord database. This provides true, declarative management of PostgreSQL clusters with high availability and automated failover.

This example defines the entire cluster, the specialized VectorChord image, the user, and the extensions in a single file:

# helmrelease.yaml for my CloudNativePG Cluster
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: librechat-rag-api-postgresql
  namespace: ai-playground
spec:
  chart:
    spec:
      chart: cluster # Using the CNPG operator's chart
  values:
    cluster:
      name: cnpg-cluster
      # Using the specialized VectorChord image
      imageName: ghcr.io/tensorchord/cloudnative-vectorchord:17-0.5.1
      # Using Longhorn as my current storage solution
      storage:
        size: 2Gi
        storageClass: longhorn
      # Declaratively create the 'rag' user and its password secret
      roles:
        - name: rag
          comment: RAG API
          login: true
          superuser: true
          passwordSecret:
            name: librechat-rag-api-db-secret
      # Declaratively create the DB and enable extensions
      postgresql:
        shared_preload_libraries:
          - "vchord.so"
      initdb:
        database: librechat-vectordb
        owner: rag
        secret:
          name: librechat-rag-api-postgres-user
        postInitSQL:
          - CREATE EXTENSION IF NOT EXISTS "vector";
          - CREATE EXTENSION IF NOT EXISTS "vchord" CASCADE;
          - CREATE EXTENSION IF NOT EXISTS "cube" CASCADE;
          - CREATE EXTENSION IF NOT EXISTS "earthdistance" CASCADE;

This is the kind of powerful and modern infrastructure we should be able to easily connect LibreChat to. The most compelling reason for this enhancement is that VectorChord is designed as a seamless, drop-in replacement for pgvector. From the application's perspective, the migration is essentially transparent. According to the VectorChord README.md, it is "fully compatible with pgvector data types and syntax." This means that LibreChat's RAG API, which already works with pgvector, requires zero code changes to benefit from VectorChord.

Unfortunately, integrating this database with the current Helm-Chart was a bit difficult due to a hardcoded DB_HOST in the ConfigMap. This effectively locks users into the in-chart PostgreSQL and prevents connections to any existing or external database.

The only solution I was able to use is a postRenderer patch to forcefully inject the correct DB_HOST at runtime.

# helmrelease.yaml for the RAG API
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: librechat-rag-api
  namespace: ai-playground
spec:
  values:
    global:
      librechat:
        # These two variables needs to be set
        existingSecretName: librechat-rag-api-secret
        existingSecretApiKey: RAG_OPENAI_API_KEY
    rag:
      configEnv:
        RAG_USE_FULL_CONTEXT: "true"
        EMBEDDINGS_PROVIDER: "google_genai"
        EMBEDDINGS_MODE: "gemini-embedding-001"
      # RAG_GOOGLE_API_KEY is in the existingSecret
      existingSecret: librechat-rag-api-secret
    # We must disable the in-chart postgres
    postgresql:
      enabled: false
    # ... other values
  
  # This post-renderer circumvents the hardcoded DB_HOST.
  # A JSON patch injects the correct DB_HOST into the container's `env` array,
  # which takes precedence over the incorrect value from `envFrom`.
  postRenderers:
    - kustomize:
        patches:
          - patch: |-
              - op: add
                path: /spec/template/spec/containers/0/env/-
                value:
                  name: DB_HOST
                  value: cnpg-cluster-rw.ai-playground.svc.cluster.local # My actual CNPG service endpoint
            target:
              kind: Deployment
              name: librechat-rag-api-librechat-rag-api

To unlock the full potential of the RAG API for everyone, I propose a simple enhancement to allow DB_HOST to be configured via values.yaml

This small change would eliminate the need for complex patches and allow anyone to connect the RAG API to a high-performance database—whether it's on Kubernetes, Docker or Podman.

Feedback is always welcome. Thanks.

[gbartolini]

Thanks for mentioning CloudNativePG (I am a maintainer and co-founder). My suggestion is to use the Database resource to manage extensions, instead of the post-init SQL stanza. See my blog article for an idea: https://www.gabrielebartolini.it/articles/2025/06/cnpg-recipe-18-getting-started-with-pgvector-on-kubernetes-using-cloudnativepg/