Cloudflare AI Gateway

[lunar-studio]

What happened?

Perhaps I'm doing something wrong, but I'm encountering issues trying to get Anthropic and Gemini (Generative) routed through CF AI Gateway. I can get OpenAI working just fine, just not these other two.

Version Information

ghcr.io/danny-avila/librechat-dev latest 2fa722380692 7 hours ago 1.56GB
ghcr.io/danny-avila/librechat-dev 8d938efaacda 21 hours ago 1.56GB
ghcr.io/danny-avila/librechat-dev 57563c4cf684 27 hours ago 1.56GB
ghcr.io/danny-avila/librechat-rag-api-dev latest 3011b13d8433 4 days ago 9.6GB
ghcr.io/danny-avila/librechat-rag-api-dev-lite latest 2f812738600a 4 days ago 1.67GB
ghcr.io/danny-avila/librechat latest 78a279684768 9 days ago 1.17GB
ghcr.io/danny-avila/librechat v0.8.0-rc2 78a279684768 9 days ago 1.17GB
ghcr.io/danny-avila/librechat 25d757180cb4 2 weeks ago 1.17GB

Steps to Reproduce

1. Lots of steps.

What browsers are you seeing the problem on?

Chrome

Relevant log output

been clearing hard cache. Made sure CF is purged. Keys are also within GF AI Gateway.
I also keep seeing "Please configure AI Gateway in the Cloudflare dashboard" although OpenAI is working fine.

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

It's possible CF AI gateway is not following OpenAI spec for Anthropic/Gemini, so you would have to use it as a reverse proxy for those endpoints.

#============#
# Anthropic  #
#============#

ANTHROPIC_API_KEY=user_provided
# ANTHROPIC_MODELS=claude-opus-4-20250514,claude-sonnet-4-20250514,claude-3-7-sonnet-20250219,claude-3-5-sonnet-20241022,claude-3-5-haiku-20241022,claude-3-opus-20240229,claude-3-sonnet-20240229,claude-3-haiku-20240307
# ANTHROPIC_REVERSE_PROXY=

#============#
# Google     #
#============#

GOOGLE_KEY=user_provided

# GOOGLE_REVERSE_PROXY=
# Some reverse proxies do not support the X-goog-api-key header, uncomment to pass the API key in Authorization header instead.
# GOOGLE_AUTH_HEADER=true

As for OpenAI compatible endpoints, you can use this example:
https://www.librechat.ai/docs/configuration/librechat_yaml/ai_endpoints/cloudflare

[lunar-studio]

Thanks Danny. That worked with some modifications provided below for other user reference.

Set this below in the portainer .env stack:

      ANTHROPIC_REVERSE_PROXY: "https://gateway.ai.cloudflare.com/v1/${CF_ACCOUNT_ID}/${CF_GATEWAY_ID}/anthropic"
      GOOGLE_REVERSE_PROXY: "https://gateway.ai.cloudflare.com/v1/${CF_ACCOUNT_ID}/${CF_GATEWAY_ID}/google-ai-studio"

Anthropic with ANTHROPIC_API_KEY set and ANTHROPIC_MODELS=claude-opus-4.1-20250805,claude-sonnet-4-20250522,claude-3.5-sonnet-20240621,claude-3.5-haiku-20241022 worked fine out of the box. I also had it set as a provider within the CF Gateway.

However I had to run troubleshooting with Google:

1. GOOGLE_KEY set in the panel was okay.
2. GOOGLE_MODELS=gemini-2.5-pro-preview-03-25,gemini-2.5-flash-preview,gemini-2.5-flash-lite had to be added to the Portainer config stack otherwise the UI would pull up with deprecated Gemini models repeatedly which don't work for various reasons - just having them alone in the panel would not update the UI.
3. I had to remove the Google provider key within Cloudflare Gateway which ultimately got it working properly. It was essentially double-passing the keys (I believe) to Google which was causing the following (redacted) errors:

Something went wrong. Here's the specific error message we encountered: An error occurred while processing the request: [GoogleGenerativeAI Error]: Error fetching from https://gateway.ai.cloudflare.com/v1${CF_ACCOUNT_ID}/${CF_GATEWAY_ID}/google-ai-studio/v1beta/models/gemini-2.5-pro-preview-03-25:streamGenerateContent?alt=sse: [401 Unauthorized] Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.