FEAT Traefik logs with Axiom using Vector

[pascalandy]

You are a senior DevOps engineer. Your task is to generate a complete set of configuration files and documentation for integrating Traefik logs with Axiom using Vector, all managed within a Docker Compose setup.

The goal is to have Traefik write its access logs in JSON format to a shared volume, and have a Vector service read those logs from the volume and forward them to a specified Axiom dataset.

Generate the following three artifacts:

1. A docker-compose.yml file.
2. A Vector configuration file named vector.yaml.
3. A README.md file explaining the setup.

Requirements for docker-compose.yml

• Define three services: traefik, vector, and a sample service like whoami that traefik can route to.
• Define a named volume (e.g., traefik-logs) to be shared between the traefik and vector services.
• Traefik Service:
  • Use the official traefik image.
  • Configure the command arguments to enable the access log.
  • Set the access log format to json.
  • Set the access log file path to a file inside the shared volume (e.g., /logs/access.log).
  • Mount the shared volume to /logs.
  • Map ports 80:80 and 8080:8080.
• Vector Service:
  • Use the official timberio/vector image.
  • Mount the shared traefik-logs volume to a path inside the container (e.g., /var/log/traefik) so it can read the log file.
  • Mount the local vector.yaml configuration file into the container (e.g., to /etc/vector/vector.yaml).
  • Pass the Axiom Dataset and API Token as environment variables from an .env file. Use AXIOM_DATASET and AXIOM_TOKEN as the variable names.
• Whoami Service:
  • Use the traefik/whoami image.
  • Add labels for Traefik to discover and route traffic to this service.

Requirements for vector.yaml

• Source:
  • Define a file source named traefik_logs.
  • Set the include path to the location of the access log file inside the Vector container (e.g., /var/log/traefik/access.log).
• Sink:
  • Define an axiom sink named axiom_sink.
  • Set the input to come from the traefik_logs source.
  • Set the dataset and token fields to read from the environment variables ${AXIOM_DATASET} and ${AXIOM_TOKEN} respectively.
  • Enable gzip compression.

Requirements for README.md

• Provide a clear title: "Traefik to Axiom Logging with Vector".
• Include a brief overview of the architecture (Traefik -> Shared Volume -> Vector -> Axiom).
• Prerequisites:
  • List required tools: Docker and Docker Compose.
  • List required credentials: An Axiom Dataset name and an Axiom API Ingest Token.
• Setup Instructions:
  1. Explain that the user must create an .env file in the same directory.
  2. Provide a template for the .env file with AXIOM_DATASET and AXIOM_TOKEN placeholders.
  3. Provide the command to start the services: docker-compose up -d.
• Verification:
  • Explain how to generate traffic (e.g., by visiting http://localhost).
  • Instruct the user to check their Axiom dataset for incoming logs.
• File Structure:
  • Show a simple tree diagram of the required files:

    .
    ├── docker-compose.yml
    ├── vector.yaml
    └── .env
    

[pascalandy]

also consider, buffering size : https://jensknipper.de/blog/traefik-logging/

[pascalandy]

this is DevOps. Wrong repo