Skip to content

[PSP-449] Add on-prem support to model-engine #725

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

[PSP-449] Add on-prem support to model-engine #725

wants to merge 5 commits into from

Conversation

@TarunRavikumar
Copy link
Collaborator

@TarunRavikumar TarunRavikumar commented Oct 28, 2025 •
edited
Loading

Add On-Premise Deployment Support

This PR adds support for on-premise deployments using Redis, MinIO storage.

Key Changes

  • New on-prem configuration: Added onprem.yaml config file with settings for MinIO, Redis, and private registries
  • Redis-based infrastructure: Implemented Redis task queues and on-prem queue endpoint delegate
  • S3-compatible storage: Added support for MinIO and custom S3 endpoints with configurable addressing styles
  • Container registry flexibility: Support for private registries with OnPremDockerRepository
  • Database configuration: Environment variable-based PostgreSQL connection for on-prem deployments
  • Improved logging: Enhanced error handling and debug logs in S3 file storage gateway

Why?

  • We have contractual requirements to move our stack on-prem.
  • MCPx needs model engine running locally for on-prem and ci/cd testing.

Next Steps

  • There's probably more work we need to do in llm-engine to fully bring it to on-prem support. I've only tested that model engine can create new launch endpoints (and healthy pods) from our input docker image. This was what was needed for mcpx-go.

Linear

https://linear.app/scale-epd/issue/PSP-449/llm-engine-on-prem-support

@socket-security
Copy link

socket-security bot commented Oct 28, 2025 •
edited
Loading

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security		 Vulnerability Quality Maintenance License
Updatedpsycopg2-binary@​2.9.3 ⏵ 2.9.10100 +110010010070

View full report

@TarunRavikumar TarunRavikumar changed the title Add on-prem support to model-engine [PSP-449] Add on-prem support to model-engine Oct 28, 2025
MicahFulton
MicahFulton reviewed Oct 29, 2025
View reviewed changes
pr.md
@@ -0,0 +1,20 @@
# Add On-Premise Deployment Support
Copy link
Collaborator

@MicahFulton MicahFulton Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need this file?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MicahFulton MicahFulton MicahFulton left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@TarunRavikumar @MicahFulton