Gradual CPU & Memory Leak on Rocky Linux (File Source -> Blackhole Sink)

[Space0726]

Question

Hi,

I am encountering an issue with gradual CPU and Memory usage increase while running Vector on a Rocky Linux server (Non-containerized environment).

Environment:

• Vector Version: 0.51.1

• Installation Method: Binary (vector-0.51.1-x86_64-unknown-linux-gnu.tar.gz)

• OS: Rocky Linux 9.3 (Blue Onyx)

• Kernel: 5.14.0-362.24.1.el9_3.0.1.x86_64

• Hardware: Intel Xeon (Cascadelake) 2 Cores / 4GB RAM / 50GB Disk

Context: The host runs a Java-based TCP application using Logback for logging. Vector is configured to tail these logs.

Log Rotation Strategy: The logs are rotated daily at midnight. The rotation process is as follows:

1. Rename: The active log file is renamed with the date suffix (e.g., app.log -> app.2025-12-17.log).

2. Move: The renamed file is moved to a date-specific directory (e.g., /logs/2025-12-17/).

3. Create: A fresh log file is created for the application to continue writing.

The Problem & Debugging Attempts: I initially used a kafka sink, but noticed a continuous creep in both CPU and Memory usage. To isolate the issue, I have:

1. Changed the sink to blackhole.

2. Minimized transforms logic.

3. Tried Jemalloc tuning: applied MALLOC_CONF="background_thread:true,dirty_decay_ms:1000,muzzy_decay_ms:1000", but the issue persists.

Despite these changes (Source -> Minimal Transform -> Blackhole), the resource usage continues to grow over time.

Metrics: I am monitoring Vector using prometheus_exporter and Grafana.

Any insights on how to debug this further or what might be causing this leak in a file-source setup (especially with the rename-and-move rotation strategy) would be greatly appreciated.

Thanks!

Vector Config

in log.yml

sources:
  log_source:
    type: file
    include:
      - "/home/msg-service/logs/msg-service.8081.log"
      - "/home/msg-service/logs/msg-service.8082.log"
    glob_minimum_cooldown_ms: 10000
    rotate_wait_secs: 30

transforms:
  log_transform:
    type: remap
    inputs:
      - log_source
    drop_on_error: true
    drop_on_abort: true
    reroute_dropped: true
    source: |
      ."service.name" = "msg-service"

  event_transform:
    type: remap
    inputs:
      - log_transform
    source: |
      ."service.name" = "msg-service-event"

sinks:
  dropped_sink:
    type: blackhole
    inputs:
      - log_transform.dropped

  log_sink:
    type: blackhole
    inputs:
      - log_transform

  event_sink:
    type: blackhole
    inputs:
      - event_transform

in vector.yml

data_dir: "/usr/local/vector/data_dir"

api:
  enabled: true

expire_metrics_secs: 30

sources:
  internal_metrics_source:
    type: internal_metrics
    scrape_interval_secs: 10

sinks:
  exporter_sink:
    type: prometheus_exporter
    inputs:
      - internal_metrics_source
    flush_period_secs: 10

Vector Logs

No response