mkdir permission denied in Docker v3.6.6 when writing to fs #20910
Description
Describe the bug
In dockerized version 3.6.6 the ingester does not seem to have permissions to create new folders in the mounted volume.
level=error ts=2026-02-21T15:32:09.147328855Z caller=flush.go:262 component=ingester loop=31 org_id=fake msg="failed to flush" retries=6 err="failed to flush chunks: store put chunk: mkdir fake: permission denied, num_chunks: 2, labels: {compose_service=\"node_exporter\", container=\"prometheus-node_exporter-1\", container_network=\"host\", service_name=\"prometheus-node_exporter-1\"}"
To Reproduce
Steps to reproduce the behavior:
- Started Loki (version 3.6.6)
- Started Alloy (v1.13.1) to collect logs from running containers
- Restarted loki container
- Query:
{container="prometheus-loki-1"}
Compose config
loki:
image: grafana/loki:latest
command: -config.file=/etc/loki/local-config.yaml
ports:
- "3100:3100"
volumes:
- loki_data:/loki
- ./loki-config.yaml:/etc/loki/local-config.yaml:ro
volumes:
loki_data:
Expected behavior
Loki should be able to write to the mounted docker volume
Environment:
- Infrastructure: ubuntu server 24.04
- Deployment tool: docker compose
Screenshots, Promtail config, or terminal output
Content of mounted volume. Tried chmod -R 775, chown -R 10001:10001 and chmod g+s. None worked.
drwxrwsr-x 8 10001 10001 4096 Feb 21 14:23 .
drwx-----x 3 root root 4096 Feb 21 14:14 ..
drwxrwxr-x 4 10001 10001 4096 Feb 21 14:23 compactor
drwxrwxr-x 2 10001 10001 4096 Feb 18 08:47 rules
drwxrwxr-x 2 10001 10001 4096 Feb 18 08:47 rules-temp
drwxrwxr-x 7 10001 10001 4096 Feb 21 15:27 tsdb-shipper-active
drwxrwxr-x 3 10001 10001 4096 Feb 21 14:23 tsdb-shipper-cache
drwxrwxr-x 4 10001 10001 4096 Feb 21 16:22 wal