Skip to content

podman-env CRI-O prepends '/' to unqualified image names causing InvalidImageName errors #21251

@elasticdotventures

Description

@elasticdotventures

🐛 Critical Bug Report: CRI-O Image Name Resolution Failure

Summary

When using minikube with podman driver and CRI-O runtime, Kubernetes incorrectly prepends a forward slash (/) to unqualified image names during resolution, causing InvalidImageName errors and preventing pods from starting.

Environment

  • Driver: podman
  • Container Runtime: CRI-O
  • Issue Scope: Image name resolution in CRI-O → podman integration
  • Impact: Critical - prevents custom application deployment

Problem Description

Expected Behavior: app4dog/middleware:latest → resolves correctly
Actual Behavior: app4dog/middleware:latest → becomes /app4dog/middleware:latest

Error Message

Failed to apply default image tag "/app4dog/middleware:latest": couldn't parse image name "/app4dog/middleware:latest": invalid reference format

Root Cause Analysis

  1. Image Storage: podman build -t app4dog/middleware:latest stores as localhost/app4dog/middleware:latest
  2. K8s Reference: Deployment references app4dog/middleware:latest (unqualified)
  3. CRI-O Resolution: Incorrectly prepends / instead of resolving to localhost/ prefix
  4. Validation Failure: Malformed name /app4dog/middleware:latest fails parsing

Reproduction Steps

  1. minikube start --driver=podman --container-runtime=cri-o
  2. eval $(minikube podman-env)
  3. podman build -t app4dog/middleware:latest .
  4. Deploy K8s pod referencing app4dog/middleware:latest
  5. Observe InvalidImageName error

Related Issues

Impact Assessment

  • Severity: Critical - breaks podman + CRI-O development workflows
  • Scope: Affects custom image deployment (not pre-built images)
  • Regression: Long-standing issue in CRI-O integration

Technical Context

This is separate from podman-env connectivity issues. The bug occurs in the CRI-O runtime's image name resolution logic, specifically in how /etc/containers/registries.conf handles unqualified registry names.

Location: Likely in pkg/minikube/cruntime/crio.go registry configuration or CRI-O config generation.

Next Steps

Working on engineered fix that:

  • ✅ Resolves CRI-O image name resolution
  • ✅ Maintains Docker compatibility (99.99% of users)
  • ✅ Integrates with existing podman-env improvements
  • ✅ Low-impact, surgical change

/kind bug
/priority critical-urgent
/sig node

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.priority/critical-urgentHighest priority. Must be actively worked on as someone's top priority right now.sig/nodeCategorizes an issue or PR as relevant to SIG Node.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions