out_splunk tries to send data with empty payload #10313
Description
Bug Report
Describe the bug
out_splunk tries to send POST request when the payload is empty.
To Reproduce
- Steps to reproduce the problem:
Run Fluent Bit with the following config:
pipeline:
inputs:
- name: event_type
type: metrics
processors:
metrics:
- name: metrics_selector
metric_name: /./
action: exclude
storage.type: filesystem
outputs:
- name: stdout
match: '*'
processors:
metrics:
- name: metrics_selector
metric_name: /./
action: exclude
- name: splunk
match: '*'
host: localhost
port: 8090
splunk_token: token
service:
log_level: info
storage.path: ./storage
The log will show that attempts to send data were performed.
fluent-bit -c drop.yml
Fluent Bit v4.0.2
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
______ _ _ ______ _ _ ___ _____
| ___| | | | | ___ (_) | / || _ |
| |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __/ /| || |/' |
| _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| || /| |
| | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /\___ |\ |_/ /
\_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ |_(_)___/
[2025/05/08 11:27:55] [ info] [fluent bit] version=4.0.2, commit=5dd2b0b438, pid=13482
[2025/05/08 11:27:55] [ info] [storage] ver=1.1.6, type=memory+filesystem, sync=normal, checksum=off, max_chunks_up=128
[2025/05/08 11:27:55] [ info] [storage] backlog input plugin: storage_backlog.1
[2025/05/08 11:27:55] [ info] [simd ] disabled
[2025/05/08 11:27:55] [ info] [cmetrics] version=1.0.0
[2025/05/08 11:27:55] [ info] [ctraces ] version=0.6.5
[2025/05/08 11:27:55] [ info] [input:event_type:event_type.0] initializing
[2025/05/08 11:27:55] [ info] [input:event_type:event_type.0] storage_strategy='filesystem' (memory + filesystem)
[2025/05/08 11:27:55] [ info] [input:event_type:event_type.0] thread instance initialized
[2025/05/08 11:27:55] [ info] [input:storage_backlog:storage_backlog.1] initializing
[2025/05/08 11:27:55] [ info] [input:storage_backlog:storage_backlog.1] storage_strategy='memory' (memory only)
[2025/05/08 11:27:55] [ info] [input:storage_backlog:storage_backlog.1] queue memory limit: 95.4M
[2025/05/08 11:27:55] [ info] [output:stdout:stdout.0] worker #0 started
[2025/05/08 11:27:55] [ info] [output:splunk:splunk.1] worker #0 started
[2025/05/08 11:27:55] [ info] [output:splunk:splunk.1] worker #1 started
[2025/05/08 11:27:55] [ info] [sp] stream processor started
[2025/05/08 11:27:57] [ info] [input:event_type:event_type.0] [OK] collector_time
[2025/05/08 11:27:58] [error] [net] TCP connection failed: localhost:8090 (Connection refused)
[2025/05/08 11:27:58] [error] [net] TCP connection failed: localhost:8090 (Connection refused)
[2025/05/08 11:27:58] [ warn] [engine] failed to flush chunk '13482-1746718078.238015000.flb', retry in 9 seconds: task_id=0, input=event_type.0 > output=splunk.1 (out_id=1)
[2025/05/08 11:27:59] [ info] [input:event_type:event_type.0] [OK] collector_time
[2025/05/08 11:28:00] [error] [net] TCP connection failed: localhost:8090 (Connection refused)
[2025/05/08 11:28:00] [error] [net] TCP connection failed: localhost:8090 (Connection refused)
[2025/05/08 11:28:00] [ warn] [engine] failed to flush chunk '13482-1746718080.237270000.flb', retry in 10 seconds: task_id=1, input=event_type.0 > output=splunk.1 (out_id=1)
[2025/05/08 11:28:01] [ info] [input:event_type:event_type.0] [OK] collector_time
^C[2025/05/08 11:28:02] [engine] caught signal (SIGINT)
[2025/05/08 11:28:02] [ info] [input] pausing storage_backlog.1
[2025/05/08 11:28:02] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2025/05/08 11:28:02] [ info] [output:stdout:stdout.0] thread worker #0 stopped
[2025/05/08 11:28:02] [ info] [output:splunk:splunk.1] thread worker #0 stopping...
[2025/05/08 11:28:02] [ info] [output:splunk:splunk.1] thread worker #0 stopped
[2025/05/08 11:28:02] [ info] [output:splunk:splunk.1] thread worker #1 stopping...
[2025/05/08 11:28:02] [ info] [output:splunk:splunk.1] thread worker #1 stopped
Inspecting the chunk files will show that they're empty:
chunk-inspector dump -file storage/event_type.0/13482-1746718078.238015000.flb -v
Filename storage/event_type.0/13482-1746718078.238015000.flb OK
Metadata Length: 16
12 bytes read from Metadata: [event_type.0]
File size: 16384 bytes
16344 bytes read from User Content: [��meta��cmetrics��external��processing��static_labels��metrics�]
The content below is optional, not needed for reproduction or verification of the issue
If the data is sent to an actual Splunk endpoint, or a mocked one, it will show that the content is 0. This is from a mock Splunk endpoint:
2025-05-08T11:27:45.247-04:00 INFO 13338 --- [.0-8090-exec-10] c.c.s.controller.SplunkMockController : Got request to /services/collector/event
host: localhost:8090
content-length: 0
user-agent: Fluent-Bit
authorization: Splunk token
connection: keep-alive
Expected behavior
The out_splunk shouldn't try to send empty payloads.
Your Environment
- Version used: Latest from master branch (commit=5dd2b0b438)