Best practices for handling Nginx PID/Permissions with Melange & Apko?

[taha2samy]

Hi everyone 👋,

I’m working on building a production-grade, minimal Nginx image using the Wolfi toolchain. My goal is to create a secure "distroless" equivalent by compiling Nginx from source using Melange and assembling the final runtime with Apko.

I have successfully compiled the binary, but I am hitting a wall regarding runtime permissions and volatile paths (specifically the PID file and logs) when running as a non-root user.

The Constraints & The Problem:

1. Melange Linter: The linter correctly flags any attempt to package directories under /run, as this is a volatile filesystem.
2. Runtime Errors: I've removed /run from the Melange build artifact and tried to define the directory structure and permissions in the apko.yaml paths section instead. However, the container fails immediately upon startup with (13: Permission denied) when Nginx tries to create the PID file or write logs, despite my attempts to set ownership to the nginx user (UID 101).

My Setup:

• Build: Custom Nginx compilation with strict flags.
• User: Running as nginx (UID 101).
• Target: pid /run/nginx.pid;

Configuration Files:

Here is my nginx.melange.yaml:

package:
  name: nginx
  version: 1.27.0
  epoch: 0
  description: "nginx web server"
  copyright:
    - license: BSD-2-Clause
  dependencies:
    runtime:
      - ca-certificates

environment:
  contents:
    repositories:
      - https://packages.wolfi.dev/os
    keyring:
      - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
    packages:
      - wolfi-base
      - build-base
      - openssl-dev
      - pcre2-dev
      - zlib-dev
      - linux-headers

pipeline:
  - uses: fetch
    with:
      uri: http://nginx.org/download/nginx-${{package.version}}.tar.gz
      expected-sha256: b7230e3cf87eaa2d4b0bc56aadc920a960c7873b9991a1b66ffcc08fc650129c
  - runs: |
      set -e
      tar xzf nginx-${{package.version}}.tar.gz
      cd nginx-${{package.version}}
      
      # Configure paths
      ./configure \
        --prefix=/usr \
        --sbin-path=/usr/bin/nginx \
        --conf-path=/etc/nginx/nginx.conf \
        --error-log-path=/var/log/nginx/error.log \
        --http-log-path=/var/log/nginx/access.log \
        --pid-path=/run/nginx.pid \
        --lock-path=/run/nginx.lock \
        --user=nginx \
        --group=nginx \
        --with-http_ssl_module \
        --with-http_v2_module \
        --http-client-body-temp-path=/var/cache/nginx/client_temp \
        --http-proxy-temp-path=/var/cache/nginx/proxy_temp \
        
      make
      make install DESTDIR=${{targets.destdir}}

      # Cleanup volatile paths to satisfy linter
      rm -rf ${{targets.destdir}}/run

      # Create cache/log dirs
      mkdir -p ${{targets.destdir}}/var/log/nginx
      mkdir -p ${{targets.destdir}}/var/cache/nginx

  - uses: strip

subpackages:
  - name: nginx-config
    pipeline:
      - runs: |
          mkdir -p ${{targets.subpkgdir}}/etc/nginx
          # Config setup...

And my apko.yaml where I attempt to fix permissions:

contents:
  packages:
    - nginx
    - nginx-config
    # ... other deps

paths:
  # Trying to explicitly provision /run for the PID file
  - path: /run
    type: directory
    uid: 101
    gid: 101
    permissions: 0o777 # Even loose permissions aren't working as expected
  
  - path: /var/log/nginx
    type: directory
    uid: 101
    gid: 101
    permissions: 0o755
    recursive: true

accounts:
  users:
    - username: nginx
      uid: 101
      gid: 101
  run-as: nginx

[taha2samy]

is anyone here ?
what is wrong ?