Skip to content

Commit bf4591a

Browse files
committed
pipeline: input: add prometheus textfile plugin doc
Signed-off-by: Eduardo Silva <[email protected]>
1 parent bf69b7e commit bf4591a

File tree

2 files changed

+113
-0
lines changed

2 files changed

+113
-0
lines changed

SUMMARY.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,7 @@
121121
* [Process metrics](pipeline/inputs/process.md)
122122
* [Prometheus remote write](pipeline/inputs/prometheus-remote-write.md)
123123
* [Prometheus scrape Metrics](pipeline/inputs/prometheus-scrape-metrics.md)
124+
* [Prometheus text file](pipeline/inputs/prometheus-textfile.md)
124125
* [Random](pipeline/inputs/random.md)
125126
* [Serial interface](pipeline/inputs/serial-interface.md)
126127
* [Splunk](pipeline/inputs/splunk.md)
Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# Prometheus Textfile
2+
3+
The **prometheus_textfile** input plugin allows Fluent Bit to read metrics from Prometheus text format files (.prom files) on the local filesystem. This plugin is particularly useful for collecting custom metrics that are written to files by external applications or scripts, similar to the Prometheus Node Exporter textfile collector.
4+
5+
## Configuration Parameters
6+
7+
| Key | Description | Default |
8+
|-----|-------------|---------|
9+
| path | File or directory path pattern. Supports glob patterns with `*` wildcard (e.g., `/var/lib/prometheus/*.prom`) | |
10+
| scrape_interval | Interval in seconds between file scans | 10s |
11+
12+
## Getting Started
13+
14+
### Simple Configuration
15+
16+
The following configuration will monitor `/var/lib/prometheus/textfile` directory for `.prom` files every 15 seconds:
17+
18+
```yaml
19+
pipeline:
20+
inputs:
21+
- name: prometheus_textfile
22+
tag: custom_metrics
23+
path: '/var/lib/prometheus/textfile/*.prom'
24+
scrape_interval: 15
25+
outputs:
26+
- name: prometheus_exporter
27+
match: custom_metrics
28+
host: 192.168.100.61
29+
port: 2021
30+
```
31+
32+
## Prometheus Text Format
33+
34+
The plugin expects files to be in the standard Prometheus text exposition format. Here's an example of a valid `.prom` file:
35+
36+
```
37+
# HELP custom_counter_total A custom counter metric
38+
# TYPE custom_counter_total counter
39+
custom_counter_total{instance="server1",job="myapp"} 42
40+
41+
# HELP custom_gauge A custom gauge metric
42+
# TYPE custom_gauge gauge
43+
custom_gauge{environment="production"} 1.23
44+
45+
# HELP custom_histogram_bucket A custom histogram
46+
# TYPE custom_histogram_bucket histogram
47+
custom_histogram_bucket{le="0.1"} 10
48+
custom_histogram_bucket{le="0.5"} 25
49+
custom_histogram_bucket{le="1.0"} 40
50+
custom_histogram_bucket{le="+Inf"} 50
51+
custom_histogram_sum 125.5
52+
custom_histogram_count 50
53+
```
54+
55+
## Use Cases
56+
57+
### Custom Application Metrics
58+
59+
Applications can write custom metrics to `.prom` files, and this plugin will collect them:
60+
61+
```bash
62+
# Script writes metrics to file
63+
echo "# HELP app_requests_total Total HTTP requests" > /var/lib/prometheus/textfile/app.prom
64+
echo "# TYPE app_requests_total counter" >> /var/lib/prometheus/textfile/app.prom
65+
echo "app_requests_total{status=\"200\"} 1500" >> /var/lib/prometheus/textfile/app.prom
66+
echo "app_requests_total{status=\"404\"} 23" >> /var/lib/prometheus/textfile/app.prom
67+
```
68+
69+
### Batch Job Metrics
70+
Cron jobs or batch processes can write completion metrics:
71+
72+
```bash
73+
#!/bin/bash
74+
# Backup script writes completion metrics
75+
BACKUP_START=$(date +%s)
76+
# ... perform backup ...
77+
BACKUP_END=$(date +%s)
78+
DURATION=$((BACKUP_END - BACKUP_START))
79+
80+
cat > /var/lib/prometheus/textfile/backup.prom << EOF
81+
# HELP backup_duration_seconds Time taken to complete backup
82+
# TYPE backup_duration_seconds gauge
83+
backup_duration_seconds ${DURATION}
84+
85+
# HELP backup_last_success_timestamp_seconds Last successful backup timestamp
86+
# TYPE backup_last_success_timestamp_seconds gauge
87+
backup_last_success_timestamp_seconds ${BACKUP_END}
88+
EOF
89+
```
90+
91+
### System Integration
92+
External monitoring tools can write metrics that Fluent Bit will collect and forward.
93+
94+
## Integration with Other Plugins
95+
96+
### OpenTelemetry Destination
97+
98+
```yaml
99+
pipeline:
100+
inputs:
101+
- name: prometheus_textfile
102+
tag: textfile_metrics
103+
path: /var/lib/prometheus/textfile
104+
- name: node_exporter_metrics
105+
tag: system_metrics
106+
scrape_interval: 15
107+
outputs:
108+
- name: opentelemetry
109+
match: '*'
110+
host: 192.168.56.4
111+
port: 2021
112+
```

0 commit comments

Comments
 (0)