Skip to content

Add (partial) support for Podman #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
anfredette merged 7 commits into from
Jan 9, 2026
Merged

Add (partial) support for Podman #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
e2c7731
Add support for Podman
namasl Dec 5, 2025
f9a1628
Update docs
namasl Dec 5, 2025
e456110
Allow kind cluster to work with podman
namasl Dec 5, 2025
06f02c8
Update definition of container tool
namasl Dec 5, 2025
be64513
Revert podman with kind
namasl Dec 8, 2025
de3a02f
Add kind support warning
namasl Dec 8, 2025
a461ddc
fix: Improve container runtime detection and documentation
anfredette Jan 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 44 additions & 21 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,22 @@ else
$(error Unsupported platform: $(UNAME_S). Please use macOS or Linux (or WSL2 on Windows))
endif

# Container runtime detection
# - CONTAINER_TOOL: Used for PostgreSQL, simulator builds, and general container operations
# Can be overridden: CONTAINER_TOOL=podman make db-start
# - KIND always requires Docker (it creates containers to simulate K8s nodes)
#
# Auto-detection prefers docker if running (for KIND compatibility), falls back to podman if running
# Only selects a runtime if its daemon is actually running
CONTAINER_TOOL ?= $(shell \
if command -v docker >/dev/null 2>&1 && docker info >/dev/null 2>&1; then \
echo docker; \
elif command -v podman >/dev/null 2>&1 && podman info >/dev/null 2>&1; then \
echo podman; \
else \
echo ""; \
fi)

# Configuration
REGISTRY ?= quay.io
REGISTRY_ORG ?= vllm-assistant
Expand Down Expand Up @@ -71,8 +87,15 @@ help: ## Display this help message

check-prereqs: ## Check if required tools are installed
@printf "$(BLUE)Checking prerequisites...$(NC)\n"
@command -v docker >/dev/null 2>&1 || (ecprintfho "$(RED)✗ docker not found$(NC). Install from https://www.docker.com/products/docker-desktop\n" && exit 1)
@printf "$(GREEN)✓ docker found$(NC)\n"
@[ -n "$(CONTAINER_TOOL)" ] || (printf "$(RED)✗ Docker or Podman not found$(NC).\n" && exit 1)
@printf "$(GREEN)✓ $(CONTAINER_TOOL) found (container runtime)$(NC)\n"
@if [ "$(CONTAINER_TOOL)" = "podman" ]; then \
if command -v docker >/dev/null 2>&1; then \
printf "$(GREEN)✓ docker also found (required for KIND cluster)$(NC)\n"; \
else \
printf "$(YELLOW)⚠ Docker not found - KIND cluster commands will not work$(NC)\n"; \
fi; \
fi
@command -v kubectl >/dev/null 2>&1 || (printf "$(RED)✗ kubectl not found$(NC). Run: brew install kubectl\n" && exit 1)
@printf "$(GREEN)✓ kubectl found$(NC)\n"
@command -v kind >/dev/null 2>&1 || (printf "$(RED)✗ kind not found$(NC). Run: brew install kind\n" && exit 1)
Expand All @@ -89,8 +112,8 @@ check-prereqs: ## Check if required tools are installed
printf "$(YELLOW)⚠ Warning: Python $$PY_VERSION detected. Some dependencies (psycopg2-binary) may not have wheels yet.$(NC)\n"; \
printf "$(YELLOW) Recommend using Python 3.13 for best compatibility.$(NC)\n"; \
fi
@docker info >/dev/null 2>&1 || (printf "$(RED)✗ Docker daemon not running$(NC). Start Docker Desktop\n" && exit 1)
@printf "$(GREEN)✓ Docker daemon running$(NC)\n"
@$(CONTAINER_TOOL) info >/dev/null 2>&1 || (printf "$(RED)✗ Docker or Podman daemon not running$(NC).\n" && exit 1)
@printf "$(GREEN)✓ $(CONTAINER_TOOL) daemon running$(NC)\n"
@command -v docker-compose >/dev/null 2>&1 || docker compose version >/dev/null 2>&1 || (printf "$(RED)✗ docker compose not found$(NC). Install Docker Compose or update Docker Desktop\n" && exit 1)
@printf "$(GREEN)✓ docker compose found$(NC)\n"
@printf "$(GREEN)All prerequisites satisfied!$(NC)\n"
Expand Down Expand Up @@ -232,28 +255,28 @@ open-backend: ## Open backend API docs in browser

build-simulator: ## Build vLLM simulator Docker image
@printf "$(BLUE)Building simulator image...$(NC)\n"
cd $(SIMULATOR_DIR) && docker build -t vllm-simulator:latest -t $(SIMULATOR_FULL_IMAGE) .
cd $(SIMULATOR_DIR) && $(CONTAINER_TOOL) build -t vllm-simulator:latest -t $(SIMULATOR_FULL_IMAGE) .
@printf "$(GREEN)✓ Simulator image built:$(NC)\n"
@printf " - vllm-simulator:latest\n"
@printf " - $(SIMULATOR_FULL_IMAGE)\n"

push-simulator: build-simulator ## Push simulator image to Quay.io
@printf "$(BLUE)Pushing simulator image to $(SIMULATOR_FULL_IMAGE)...$(NC)\n"
@# Check if logged in to Quay.io
@if ! docker login quay.io --get-login > /dev/null 2>&1; then \
@if ! $(CONTAINER_TOOL) login quay.io --get-login > /dev/null 2>&1; then \
printf "$(YELLOW)Not logged in to Quay.io. Please login:$(NC)\n"; \
docker login quay.io || (printf "$(RED)✗ Login failed$(NC)\n" && exit 1); \
$(CONTAINER_TOOL) login quay.io || (printf "$(RED)✗ Login failed$(NC)\n" && exit 1); \
else \
printf "$(GREEN)✓ Already logged in to Quay.io$(NC)\n"; \
fi
@printf "$(BLUE)Pushing image...$(NC)\n"
docker push $(SIMULATOR_FULL_IMAGE)
$(CONTAINER_TOOL) push $(SIMULATOR_FULL_IMAGE)
@printf "$(GREEN)✓ Image pushed to $(SIMULATOR_FULL_IMAGE)$(NC)\n"

pull-simulator: ## Pull simulator image from Quay.io
@printf "$(BLUE)Pulling simulator image from $(SIMULATOR_FULL_IMAGE)...$(NC)\n"
docker pull $(SIMULATOR_FULL_IMAGE)
docker tag $(SIMULATOR_FULL_IMAGE) vllm-simulator:latest
$(CONTAINER_TOOL) pull $(SIMULATOR_FULL_IMAGE)
$(CONTAINER_TOOL) tag $(SIMULATOR_FULL_IMAGE) vllm-simulator:latest
@printf "$(GREEN)✓ Image pulled and tagged as vllm-simulator:latest$(NC)\n"

docker-build: ## Build all Docker images
Expand Down Expand Up @@ -360,15 +383,15 @@ clean-deployments: ## Delete all InferenceServices from cluster

db-start: ## Start PostgreSQL container for benchmark data
@printf "$(BLUE)Starting PostgreSQL...$(NC)\n"
@if docker ps -a --format '{{.Names}}' | grep -q '^compass-postgres$$'; then \
if docker ps --format '{{.Names}}' | grep -q '^compass-postgres$$'; then \
@if $(CONTAINER_TOOL) ps -a --format '{{.Names}}' | grep -q '^compass-postgres$$'; then \
if $(CONTAINER_TOOL) ps --format '{{.Names}}' | grep -q '^compass-postgres$$'; then \
printf "$(YELLOW)PostgreSQL already running$(NC)\n"; \
else \
docker start compass-postgres; \
$(CONTAINER_TOOL) start compass-postgres; \
printf "$(GREEN)✓ PostgreSQL started$(NC)\n"; \
fi \
else \
docker run --name compass-postgres -d \
$(CONTAINER_TOOL) run --name compass-postgres -d \
-e POSTGRES_PASSWORD=compass \
-e POSTGRES_DB=compass \
-p 5432:5432 \
Expand All @@ -380,18 +403,18 @@ db-start: ## Start PostgreSQL container for benchmark data

db-stop: ## Stop PostgreSQL container
@printf "$(BLUE)Stopping PostgreSQL...$(NC)\n"
@docker stop compass-postgres 2>/dev/null || true
@$(CONTAINER_TOOL) stop compass-postgres 2>/dev/null || true
@printf "$(GREEN)✓ PostgreSQL stopped$(NC)\n"

db-remove: db-stop ## Stop and remove PostgreSQL container
@printf "$(BLUE)Removing PostgreSQL container...$(NC)\n"
@docker rm compass-postgres 2>/dev/null || true
@$(CONTAINER_TOOL) rm compass-postgres 2>/dev/null || true
@printf "$(GREEN)✓ PostgreSQL container removed$(NC)\n"

db-init: db-start ## Initialize PostgreSQL schema
@printf "$(BLUE)Initializing PostgreSQL schema...$(NC)\n"
@sleep 2
@docker exec -i compass-postgres psql -U postgres -d compass < scripts/schema.sql
@$(CONTAINER_TOOL) exec -i compass-postgres psql -U postgres -d compass < scripts/schema.sql
@printf "$(GREEN)✓ Schema initialized$(NC)\n"

db-load-synthetic: db-start ## Load synthetic benchmark data (appends)
Expand Down Expand Up @@ -434,19 +457,19 @@ db-convert-pgdump: db-start ## Convert PostgreSQL dump to JSON format
@printf "$(GREEN)✓ Created $(PGDUMP_OUTPUT)$(NC)\n"

db-shell: ## Open PostgreSQL shell
@docker exec -it compass-postgres psql -U postgres -d compass
@$(CONTAINER_TOOL) exec -it compass-postgres psql -U postgres -d compass

db-query-traffic: ## Query unique traffic patterns from database
@printf "$(BLUE)Querying unique traffic patterns...$(NC)\n"
@docker exec -i compass-postgres psql -U postgres -d compass -c \
"SELECT DISTINCT prompt_tokens, output_tokens, COUNT(*) as num_benchmarks \
@$(CONTAINER_TOOL) exec -i compass-postgres psql -U postgres -d compass -c \
"SELECT DISTINCT mean_input_tokens, mean_output_tokens, COUNT(*) as num_benchmarks \
FROM exported_summaries \
GROUP BY prompt_tokens, output_tokens \
ORDER BY prompt_tokens, output_tokens;"

db-query-models: ## Query available models in database
@printf "$(BLUE)Querying available models...$(NC)\n"
@docker exec -i compass-postgres psql -U postgres -d compass -c \
@$(CONTAINER_TOOL) exec -i compass-postgres psql -U postgres -d compass -c \
"SELECT DISTINCT model_hf_repo, hardware, hardware_count, COUNT(*) as num_benchmarks \
FROM exported_summaries \
GROUP BY model_hf_repo, hardware, hardware_count \
Expand Down
116 changes: 115 additions & 1 deletion docs/DEVELOPER_GUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,125 @@ make check-prereqs
```

This checks for:
- Docker Desktop (running)
- Docker or Podman (running)
- Python 3.11+
- Ollama
- kubectl
- KIND

### Container Runtime Support

Compass supports both **Docker** and **Podman** as container runtimes.

#### Compatibility Matrix

| Component | Docker | Podman | Notes |
|-----------|--------|--------|-------|
| PostgreSQL (`db-*` targets) | ✅ | ✅ | Works with either |
| Simulator build/push/pull | ✅ | ✅ | Works with either |
| KIND cluster (`cluster-*` targets) | ✅ | ❌ | KIND requires Docker |
| Docker Compose (`docker-*` targets) | ✅ | ⚠️ | Requires `podman-compose` |

#### Auto-Detection Behavior

The Makefile automatically detects which container runtime is available **and running**:
- **If Docker daemon is running:** Docker is used (for KIND compatibility)
- **If only Podman daemon is running:** Podman is used automatically
- **If neither daemon is running:** Commands will fail with a helpful error

This means if you quit Docker Desktop, the Makefile will automatically use Podman (if its machine is running), and vice versa.

#### Setting the Container Tool

**Option 1: Per-command override**
```bash
CONTAINER_TOOL=podman make db-start
CONTAINER_TOOL=podman make db-init
CONTAINER_TOOL=podman make db-load-guidellm
```

**Option 2: Export for your shell session**
```bash
export CONTAINER_TOOL=podman
make db-start
make db-init
make db-load-guidellm
# All commands in this session will use Podman
```

**Option 3: Create a `.env` file (persistent)**
```bash
echo "CONTAINER_TOOL=podman" >> .env
```
The Makefile automatically loads `.env`, so all subsequent `make` commands will use Podman. The `.env` file is in `.gitignore` so it won't affect other developers.

#### Using Podman on macOS

Podman on macOS requires a Linux VM to run containers:

```bash
# First time setup - initialize the Podman machine
podman machine init

# Start the Podman machine (required before each use, or after reboot)
podman machine start
```

**Important:** When starting the Podman machine, you may see this message:
```
Another process was listening on the default Docker API socket address.
You can still connect Docker API clients by setting DOCKER_HOST using the
following command in your terminal session:

export DOCKER_HOST='unix:///var/folders/.../podman-machine-default-api.sock'
```

**Do NOT set `DOCKER_HOST`** - this redirects the Docker CLI to Podman, which causes confusion. Instead, use `CONTAINER_TOOL=podman` as described above.

#### Port Conflicts

If you switch between Docker and Podman, you may encounter port conflicts:

```
Error: listen tcp :5432: bind: address already in use
```

**To resolve:**
1. Stop and remove the container in the other runtime:
```bash
# If switching TO Podman, clean up Docker first:
docker stop compass-postgres && docker rm compass-postgres

# If switching TO Docker, clean up Podman first:
podman stop compass-postgres && podman rm compass-postgres
```

2. If the port is still in use, check what's holding it:
```bash
lsof -i :5432
```

3. You may need to restart Docker Desktop if it has a stale port binding.

#### Mixed Docker/Podman Setup

You can use Podman for simple containers while keeping Docker for KIND:

1. Keep Docker Desktop installed (required for KIND clusters)
2. Use `CONTAINER_TOOL=podman` or `.env` for database operations
3. KIND cluster commands (`make cluster-*`) will use Docker automatically via `scripts/kind-cluster.sh`

Example workflow:
```bash
# Use Podman for database
export CONTAINER_TOOL=podman
make db-init
make db-load-guidellm

# KIND still uses Docker (no change needed)
make cluster-start
```

### Initial Setup

Create virtual environments and install dependencies:
Expand Down Expand Up @@ -504,6 +617,7 @@ Creates `vllm-simulator:latest` Docker image.
### Testing the Simulator Locally

```bash
# Can use podman instead of docker
docker run -p 8080:8080 \
-e MODEL_NAME=mistralai/Mistral-7B-Instruct-v0.3 \
-e GPU_TYPE=NVIDIA-L4 \
Expand Down
14 changes: 7 additions & 7 deletions scripts/kind-cluster.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,25 +22,25 @@ NC='\033[0m' # No Color

# Helper functions
print_header() {
echo -e "${BLUE}╔═══════════════════════════════════════════════════════════╗${NC}"
echo -e "${BLUE}║ $1${NC}"
echo -e "${BLUE}╚═══════════════════════════════════════════════════════════╝${NC}"
printf "${BLUE}╔═══════════════════════════════════════════════════════════╗${NC}\n"
printf "${BLUE}║ $1${NC}\n"
printf "${BLUE}╚═══════════════════════════════════════════════════════════╝${NC}\n"
}

print_step() {
echo -e "${GREEN}▶ $1${NC}"
printf "${GREEN}▶ $1${NC}\n"
}

print_error() {
echo -e "${RED}✗ ERROR: $1${NC}"
printf "${RED}✗ ERROR: $1${NC}\n"
}

print_warning() {
echo -e "${YELLOW}⚠ WARNING: $1${NC}"
printf "${YELLOW}⚠ WARNING: $1${NC}\n"
}

print_success() {
echo -e "${GREEN}✓ $1${NC}"
printf "${GREEN}✓ $1${NC}\n"
}

# Check prerequisites
Expand Down
4 changes: 2 additions & 2 deletions simulator/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Returns appropriate canned responses based on prompt patterns:
## Building

```bash
# Build the Docker image
# Build the Docker image (replace with podman as needed)
docker build -t vllm-simulator:latest .

# Test locally
Expand Down Expand Up @@ -96,7 +96,7 @@ The simulator is used automatically when `simulator_mode: true` is set in the de
### Load into KIND cluster

```bash
# Build image
# Build image (use podman as needed)
docker build -t vllm-simulator:latest .

# Load into KIND
Expand Down