Blog Post Submission: mlflow-modal - Extending MLflow Deployments to Serverless GPU Infrastructure

[debu-sinha]

Blog Post Submission

Post Type

• Deep Dive
• How-To
• Use Case
• Tips / Best Practices
• Features

Topics

• GenAI
• Advanced
• Deployment
• Core

Title

mlflow-modal: Extending MLflow Deployments to Serverless GPU Infrastructure

Abstract

MLflow serves approximately 29 million downloads per month (PyPI Stats, Feb 2026). This post announces and documents mlflow-modal, a deployment plugin I built to extend MLflow's deployment ecosystem to Modal's serverless GPU infrastructure.

The plugin adds a new deployment target so MLflow models can run on serverless GPUs without managing infrastructure:

1. New deployment target - deploy MLflow models to Modal with mlflow deployments create -t modal
2. Full Deployments API support - create, update, delete, predict, list endpoints
3. Production features - GPU selection (T4 through H100), autoscaling, streaming predictions
4. Environment handling - automatic detection of uv/pip/conda, private PyPI support

The plugin is published on PyPI (pip install mlflow-modal, v0.5.0) and validated against Modal's 1.0 API.

Target Length

~3000 words (flagship length per maintainer guidance)

Related Artifacts

• Package: mlflow-modal on PyPI (v0.5.0)
• GitHub: mlflow-modal repository
• MLflow Docs: Deployments API, Plugins
• Related PR: Add Modal as a supported deployment target with full documentation mlflow#20032 (upstream documentation)

Provenance

• Plugin architecture, design, and implementation: Debu Sinha (@debu-sinha)
• Technical validation: Modal team

Consent Acknowledgment

• I will request technical acknowledgment from Modal before PR

Additional Context

This plugin fills a gap in MLflow's deployment ecosystem -- there was no serverless GPU deployment target. For developers who want pay-per-use GPU inference with scale-to-zero and sub-second cold starts, this is a straightforward path from MLflow model to production endpoint.

This was specifically requested as a "2-3 page" flagship post by MLflow blog maintainer.

[debu-sinha]

Quick update: the upstream documentation PR (mlflow/mlflow#20032) has been merged as of Feb 13. Modal is now listed as a supported deployment target in the official MLflow docs.

I have a draft of the blog post ready and will open a PR shortly. The post covers the full plugin workflow (deploy, predict, GPU config, auto-scaling, streaming) with runnable code examples. Planning to have the Modal team review for technical accuracy before finalizing.