Skip to content

Latest commit

 

History

History
146 lines (100 loc) · 3.45 KB

5.md

File metadata and controls

146 lines (100 loc) · 3.45 KB

Docker Environment Configuration Best Practices

Date: 2024-11-19 Topic: Cross-platform dependencies and build optimization

Background

Today I tackled cross-platform dependency issues and Docker build optimization. Running on MacBook Pro with M3 chip added some platform-specific challenges.

Core Issues

  1. Docker environment configuration on ARM64
  2. Cross-platform dependency conflicts (Windows packages in Linux)
  3. Network access optimization for China

Handling Cross-Platform Dependencies

Problem: Windows-specific packages in requirements.txt break Linux builds.

Solution:

# Remove Windows-specific packages
RUN sed -i '/pywin32/d' requirements.txt

# Set pip mirror source
RUN pip config set global.index-url https://mirrors.huaweicloud.com/repository/pypi/simple/

# Install dependencies
RUN pip install --no-cache-dir -r requirements.txt

Registry Configuration

{
  "registry-mirrors": [
    "https://mirror.baidubce.com",
    "https://hub-mirror.c.163.com",
    "https://mirror.ccs.tencentyun.com"
  ]
}

In China, properly configuring image registries is essential for reasonable build times.

Multi-Stage Build Optimization

# Builder stage
FROM python:3.10 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --user -r requirements.txt

# Production stage
FROM python:3.10-slim
COPY --from=builder /root/.local /root/.local
WORKDIR /app

Benefits:

  • Smaller final image
  • Build tools not included in production
  • Better layer caching

Build Cache Optimization

Order matters for caching:

# Dependencies first (rarely change)
COPY requirements.txt .
RUN pip install -r requirements.txt

# Source code last (frequently changes)
COPY . .

If you copy source code before installing dependencies, every code change invalidates the dependency cache.

Docker Architecture Overview

Docker Ecosystem
├── Container Runtime
│   ├── Resource Isolation
│   └── Process Management
├── Image Management
│   ├── Image Building
│   └── Image Storage
└── Network Management
    ├── Container Network
    └── Service Discovery

Runtime Optimization

Key areas:

  • Resource limits (CPU, memory)
  • Log handling
  • Health checks

Example health check:

HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
  CMD curl -f http://localhost:8000/health || exit 1

Today's Reflection

The M3 chip's ARM64 architecture introduced some unexpected challenges. Many Docker images are built for x86_64, and while Docker Desktop can emulate x86, performance suffers.

The solution was finding native ARM64 images or building our own. Python's official images have ARM64 variants, which made things easier. But some third-party images required workarounds.

Another lesson: the order of Dockerfile instructions is not arbitrary. Docker's layer caching is powerful, but only if you structure your Dockerfile correctly. Put stable things first, changing things last.

Cross-platform dependency handling (removing Windows packages for Linux builds) is a hack, but sometimes hacks are necessary. The alternative - maintaining separate requirements files per platform - creates maintenance burden.

Further Learning

  • ARM64 vs x86_64 container considerations
  • Docker BuildKit advanced features
  • Container resource management
  • Health check strategies