Skip to content

Add Caddy LLM proxy and API key enforcement#373

Merged
NoorChasib merged 1 commit intomainfrom
llm-server-auth
Mar 17, 2026
Merged

Add Caddy LLM proxy and API key enforcement#373
NoorChasib merged 1 commit intomainfrom
llm-server-auth

Conversation

@NoorChasib
Copy link
Collaborator

Introduce a path-based Caddy reverse proxy for the LLM server: add a Caddyfile with model routes and a models.json manifest, plus a llm_proxy service in the controller compose file (exposes 80/443, adds caddy volumes). Require and document LLM_API_KEY and LLM_DOMAIN in the .env template. Enforce API auth in the llm container entrypoint (fail startup if LLM_API_KEY unset) and pass the key to model processes via --api-key. Minor models.conf comment update to reflect GLM 4.1V 9B.

🎯 Summary

ZUBA-

🔰 Checklist

  • I have read and agree with the following checklist
  • I have performed a self-review of my code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation where required.
  • I have tested my changes to the best of my ability.
  • I have consulted with the team if introducing a new dependency.

Introduce a path-based Caddy reverse proxy for the LLM server: add a Caddyfile with model routes and a models.json manifest, plus a llm_proxy service in the controller compose file (exposes 80/443, adds caddy volumes). Require and document LLM_API_KEY and LLM_DOMAIN in the .env template. Enforce API auth in the llm container entrypoint (fail startup if LLM_API_KEY unset) and pass the key to model processes via --api-key. Minor models.conf comment update to reflect GLM 4.1V 9B.
Copy link
Collaborator

@dbarkowsky dbarkowsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all seems straightforward.
Would appreciate a little walkthrough sometime in the next few weeks though to fully see how it's used in practice with hosted models.

Copy link
Collaborator

@Bottle7 Bottle7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@NoorChasib NoorChasib merged commit 39b7fef into main Mar 17, 2026
6 checks passed
@NoorChasib NoorChasib deleted the llm-server-auth branch March 17, 2026 21:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants