[Question]: Cannot get Nginx reverse proxy working with RAG

[matthewknill]

What is your question?

I have the following config for my Nginx config (replaced custom URLs with example.com or example.net):

map $host $backend {
    default                 127.0.0.1:8080; # Default backend
    sms.example.com 127.0.0.1:8081; # SMS Container
    chat.example.com 127.0.0.1:3080; # LibreChat Container
}

server {
    listen 80;
    server_name docker.example.com *.cloud.example.net;

    # Redirect all HTTP traffic to HTTPS
    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name docker.example.com *.cloud.example.net

    # SSL certificate settings
    ssl_certificate /etc/letsencrypt/live/docker.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/docker.example.com/privkey.pem;

    # SSL options (adjust as needed)
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_ciphers HIGH:!aNULL:!MD5;

    # Common proxy settings
    location / {
        proxy_pass http://$backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_cache_bypass $http_upgrade;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

And in my .env for LibreChat I have the following:

RAG_API_URL=http://host.docker.internal:8000
RAG_OPENAI_API_KEY=OPENAI_API_KEY

The above configuration works perfectly for everything except for when I attempt to upload files in which I get the message displayed in the screenshot.

Trying on my local machine, it RAG seems to work fine but I am a bit confused with http://host.docker.internal:8000 and what that should be set as to work when behind a reverse proxy. Do I need port 8000 accessible from client machine or should that be set differently?

More Details

Server Details:

• Ubuntu 20.04
• Hosted on Vultr

Client: Docker Engine - Community
Version: 27.1.1
API version: 1.46
Go version: go1.21.12
Git commit: 6312585
Built: Tue Jul 23 19:56:56 2024
OS/Arch: linux/amd64
Context: default

Server: Docker Engine - Community
Engine:
Version: 27.1.1
API version: 1.46 (minimum version 1.24)
Go version: go1.21.12
Git commit: cc13f95
Built: Tue Jul 23 19:56:56 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.19
GitCommit: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
runc:
Version: 1.7.19
GitCommit: v1.1.13-0-g58aa920
docker-init:
Version: 0.19.0
GitCommit: de40ad0

What is the main subject of your question?

No response

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

The RAG API should only be accessible to the LibreChat API, the one running on 3080 by default. It doesn't need to be behind the reverse proxy for the clients' sake.

It would be helpful to see the exact logs for why exactly the file is failing, also what are your startup logs in the LibreChat container? It will usually say if the RAG API is reachable.

[matthewknill]

Ah okay it does seem like it's not actually running:

0f8b90788118   ghcr.io/danny-avila/librechat-rag-api-dev-lite:latest   "python main.py"         3 weeks ago     Restarting (1) 33 seconds ago                                                         rag_api

It seems like I am getting a ton of these errors:

(Background on this error at: https://sqlalche.me/e/20/e3q8)
2024-08-15 06:55:08,251 - root - INFO - Initialized embeddings of type: <class 'langchain_openai.embeddings.base.OpenAIEmbeddings'>
/usr/local/lib/python3.10/site-packages/langchain_community/vectorstores/pgvector.py:322: LangChainPendingDeprecationWarning: Please use JSONB instead of JSON for metadata. This change will allow for more efficient querying that involves filtering based on metadata.Please note that filtering operators have been changed when using JSOB metadata to be prefixed with a $ sign to avoid name collisions with columns. If you're using an existing database, you will need to create adb migration for your metadata column to be JSONB and update your queries to use the new operators.
  warn_deprecated(
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 145, in __init__
    self._dbapi_connection = engine.raw_connection()
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3303, in raw_connection
    return self.pool.connect()
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 449, in connect
    return _ConnectionFairy._checkout(self)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
    rec = pool._do_get()
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 179, in _do_get
    with util.safe_reraise():
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
    return self._create_connection()
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
    return _ConnectionRecord(self)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 674, in __init__
    self.__connect()
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 900, in __connect
    with util.safe_reraise():
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 896, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 643, in connect
    return dialect.connect(*cargs, **cparams)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 616, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)
  File "/usr/local/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: connection to server at "vectordb" (172.18.0.4), port 5432 failed: FATAL:  database "mydatabase" does not exist


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/langchain_community/vectorstores/pgvector.py", line 383, in create_vector_extension
    session.execute(statement)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2306, in execute
    return self._execute_internal(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2181, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2050, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
  File "<string>", line 2, in _connection_for_bind
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
    ret_value = fn(self, *arg, **kw)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1144, in _connection_for_bind
    conn = bind.connect()
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3279, in connect
    return self._connection_cls(self)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 147, in __init__
    Connection._handle_dbapi_exception_noconnection(
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2443, in _handle_dbapi_exception_noconnection
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 145, in __init__
    self._dbapi_connection = engine.raw_connection()
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3303, in raw_connection
    return self.pool.connect()
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 449, in connect
    return _ConnectionFairy._checkout(self)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
    rec = pool._do_get()
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 179, in _do_get
    with util.safe_reraise():
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
    return self._create_connection()
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
    return _ConnectionRecord(self)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 674, in __init__
    self.__connect()
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 900, in __connect
    with util.safe_reraise():
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__

[danny-avila]

The vectordb is not running or also failing, do you have that container setup?

[matthewknill]

The following are my docker containers (vectordb seems to be up):

4a2348f633b4   ghcr.io/danny-avila/librechat-dev:latest                "docker-entrypoint.s…"   3 weeks ago     Up 3 weeks     0.0.0.0:3080->3080/tcp, :::3080->3080/tcp             LibreChat
0f8b90788118   ghcr.io/danny-avila/librechat-rag-api-dev-lite:latest   "python main.py"         3 weeks ago     Up 2 seconds                                                         rag_api
ca56479a54e1   mongo                                                   "docker-entrypoint.s…"   3 weeks ago     Up 3 weeks     27017/tcp                                             chat-mongodb
9c52c39f2406   ankane/pgvector:latest                                  "docker-entrypoint.s…"   3 weeks ago     Up 3 weeks     5432/tcp                                              vectordb

Looking at the logs for vectordb though, I am getting the following:

2024-09-05 23:10:03.150 UTC [31150] FATAL:  database "mydatabase" does not exist
2024-09-05 23:11:06.715 UTC [31151] FATAL:  database "mydatabase" does not exist

What is the mydatabase database and what do I need to do to fix it?

Thanks very much for your help...

[thonic]

The same thing hapenned to me @matthewknill, I believe the vectordb did not get created at some point during the setup for whatever reason (I ran our of disk space I think), so here is what I did:

docker compose down -v  # stop everything & drop pgdata2
docker compose up -d vectordb  # recreate the db

And then just start normally.