Cannot upload file to Anthropic, OpenAI endpoint used instead always

[michnovka]

What happened?

I have set up docker, and added Anthropic API key in .env file. I can chat fine, the AI respondes, but when I upload PDF, I get Error message.

From the log, it almost seems as if it tries to use key string "user_provided", but in .env this is not what I have placed. Also like I said, chat works.

Steps to Reproduce

1. Set up docker version
2. add to .env file Anthropic API key
3. chat and try to upload PDF file

What browsers are you seeing the problem on?

No response

Relevant log output

chat-meilisearch  | 2024-08-23T12:47:17.333475Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/b0807a64-dbe8-45a7-bb45-1590e7cc5bbf query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=85.6µs time.idle=143µs
chat-meilisearch  | 2024-08-23T12:47:17.348036Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=200µs time.idle=17.5ms
chat-meilisearch  | 2024-08-23T12:47:17.353827Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 38 }
chat-meilisearch  | 2024-08-23T12:47:17.361420Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
chat-meilisearch  | 2024-08-23T12:47:19.141908Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=693µs time.idle=11.0ms
chat-meilisearch  | 2024-08-23T12:47:19.152544Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 38 }
chat-meilisearch  | 2024-08-23T12:47:19.154958Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=146µs time.idle=3.62ms
chat-meilisearch  | 2024-08-23T12:47:19.155431Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/b0807a64-dbe8-45a7-bb45-1590e7cc5bbf query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=99.3µs time.idle=131µs
chat-meilisearch  | 2024-08-23T12:47:19.158114Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
chat-meilisearch  | 2024-08-23T12:47:19.162922Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=146µs time.idle=3.44ms
chat-meilisearch  | 2024-08-23T12:47:19.165035Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=137µs time.idle=3.57ms
chat-meilisearch  | 2024-08-23T12:47:19.167517Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 39 }
chat-meilisearch  | 2024-08-23T12:47:19.172737Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
chat-meilisearch  | 2024-08-23T12:47:19.184220Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 2, number_of_documents: 39 }
chat-meilisearch  | 2024-08-23T12:47:19.189516Z  INFO index_scheduler: A batch of tasks was successfully completed with 2 successful tasks and 0 failed tasks.
rag_api           | 2024-08-23 12:47:28,730 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 401 Unauthorized"
rag_api           | 2024-08-23 12:47:28,732 - root - ERROR - Error code: 401 - {'error': {'message': 'Incorrect API key provided: user_pro*ided. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}
rag_api           | 2024-08-23 12:47:28,734 - root - INFO - Request POST http://rag_api:8000/embed - 200
LibreChat         | 2024-08-23 12:47:28 error: Error embedding file File embedding failed.
LibreChat         | 2024-08-23 12:47:28 error: [/files] Error processing file: File embedding failed.

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[michnovka]

Ok, based on the log I now see that it clearly tries to upload to OpenAI, not anthropic. Which explains the error message, in .env file I do have

OPENAI_API_KEY=user_provided

but why would it not use Anthropic? This seems confusing

[tobwen]

I have exactly the same problem. The background is apparently that LibreChat has implemented its own RAG, which uses OpenAI as default. I am also wondering how I can deactivate RAG to ‘pass through’ an upload to the actual model

[danny-avila]

but why would it not use Anthropic? This seems confusing

Anthropic doesn't have an embeddings service.

Please see https://www.librechat.ai/docs/configuration/rag_api

[danny-avila]

It will be this week

[tobwen]

I really need the function to upload the file directly to the provider and not just for RAG processing.

Is there already a branch that I can help develop?

[banjavi]

Hey @danny-avila ! Just checking in to see if there’s been any update on this. We’re looking to skip the RAG UI panel and go with regular file uploads—either using the Local strategy (handled in ~/server/services/Files/Local.js and referenced in crud.js where files are stored on the server at /app/client/public/images) or the OpenAI strategy (files sent directly to OpenAI via the setup in ~/server/services/Files/strategies.js).

Basically, we’re trying to enable a context-based chat, similar to how ChatGPT handles file uploads, without involving the RAG flow. Any insights or updates on this would be super helpful! Thanks a lot!

[Meyer93]

@danny-avila . Hey are there any updates on this? I wasn't able to find anything?

[danny-avila]

There are several issues you can track:
#4616
#2755

they will be updated/closed when implemented