|
| 1 | +# Prometheus Textfile |
| 2 | + |
| 3 | +The **prometheus_textfile** input plugin allows Fluent Bit to read metrics from Prometheus text format files (.prom files) on the local filesystem. This plugin is particularly useful for collecting custom metrics that are written to files by external applications or scripts, similar to the Prometheus Node Exporter textfile collector. |
| 4 | + |
| 5 | +## Configuration Parameters |
| 6 | + |
| 7 | +| Key | Description | Default | |
| 8 | +|-----|-------------|---------| |
| 9 | +| path | File or directory path pattern. Supports glob patterns with `*` wildcard (e.g., `/var/lib/prometheus/*.prom`) | | |
| 10 | +| scrape_interval | Interval in seconds between file scans | 10s | |
| 11 | + |
| 12 | +## Getting Started |
| 13 | + |
| 14 | +### Simple Configuration |
| 15 | + |
| 16 | +The following configuration will monitor `/var/lib/prometheus/textfile` directory for `.prom` files every 15 seconds: |
| 17 | + |
| 18 | +```yaml |
| 19 | +pipeline: |
| 20 | + inputs: |
| 21 | + - name: prometheus_textfile |
| 22 | + tag: custom_metrics |
| 23 | + path: '/var/lib/prometheus/textfile/*.prom' |
| 24 | + scrape_interval: 15 |
| 25 | + outputs: |
| 26 | + - name: prometheus_exporter |
| 27 | + match: custom_metrics |
| 28 | + host: 192.168.100.61 |
| 29 | + port: 2021 |
| 30 | +``` |
| 31 | +
|
| 32 | +## Prometheus Text Format |
| 33 | +
|
| 34 | +The plugin expects files to be in the standard Prometheus text exposition format. Here's an example of a valid `.prom` file: |
| 35 | + |
| 36 | +``` |
| 37 | +# HELP custom_counter_total A custom counter metric |
| 38 | +# TYPE custom_counter_total counter |
| 39 | +custom_counter_total{instance="server1",job="myapp"} 42 |
| 40 | + |
| 41 | +# HELP custom_gauge A custom gauge metric |
| 42 | +# TYPE custom_gauge gauge |
| 43 | +custom_gauge{environment="production"} 1.23 |
| 44 | + |
| 45 | +# HELP custom_histogram_bucket A custom histogram |
| 46 | +# TYPE custom_histogram_bucket histogram |
| 47 | +custom_histogram_bucket{le="0.1"} 10 |
| 48 | +custom_histogram_bucket{le="0.5"} 25 |
| 49 | +custom_histogram_bucket{le="1.0"} 40 |
| 50 | +custom_histogram_bucket{le="+Inf"} 50 |
| 51 | +custom_histogram_sum 125.5 |
| 52 | +custom_histogram_count 50 |
| 53 | +``` |
| 54 | +
|
| 55 | +## Use Cases |
| 56 | +
|
| 57 | +### Custom Application Metrics |
| 58 | +
|
| 59 | +Applications can write custom metrics to `.prom` files, and this plugin will collect them: |
| 60 | +
|
| 61 | +```bash |
| 62 | +# Script writes metrics to file |
| 63 | +echo "# HELP app_requests_total Total HTTP requests" > /var/lib/prometheus/textfile/app.prom |
| 64 | +echo "# TYPE app_requests_total counter" >> /var/lib/prometheus/textfile/app.prom |
| 65 | +echo "app_requests_total{status=\"200\"} 1500" >> /var/lib/prometheus/textfile/app.prom |
| 66 | +echo "app_requests_total{status=\"404\"} 23" >> /var/lib/prometheus/textfile/app.prom |
| 67 | +``` |
| 68 | + |
| 69 | +### Batch Job Metrics |
| 70 | +Cron jobs or batch processes can write completion metrics: |
| 71 | + |
| 72 | +```bash |
| 73 | +#!/bin/bash |
| 74 | +# Backup script writes completion metrics |
| 75 | +BACKUP_START=$(date +%s) |
| 76 | +# ... perform backup ... |
| 77 | +BACKUP_END=$(date +%s) |
| 78 | +DURATION=$((BACKUP_END - BACKUP_START)) |
| 79 | + |
| 80 | +cat > /var/lib/prometheus/textfile/backup.prom << EOF |
| 81 | +# HELP backup_duration_seconds Time taken to complete backup |
| 82 | +# TYPE backup_duration_seconds gauge |
| 83 | +backup_duration_seconds ${DURATION} |
| 84 | +
|
| 85 | +# HELP backup_last_success_timestamp_seconds Last successful backup timestamp |
| 86 | +# TYPE backup_last_success_timestamp_seconds gauge |
| 87 | +backup_last_success_timestamp_seconds ${BACKUP_END} |
| 88 | +EOF |
| 89 | +``` |
| 90 | + |
| 91 | +### System Integration |
| 92 | +External monitoring tools can write metrics that Fluent Bit will collect and forward. |
| 93 | + |
| 94 | +## Integration with Other Plugins |
| 95 | + |
| 96 | +### OpenTelemetry Destination |
| 97 | + |
| 98 | +```yaml |
| 99 | +pipeline: |
| 100 | + inputs: |
| 101 | + - name: prometheus_textfile |
| 102 | + tag: textfile_metrics |
| 103 | + path: /var/lib/prometheus/textfile |
| 104 | + - name: node_exporter_metrics |
| 105 | + tag: system_metrics |
| 106 | + scrape_interval: 15 |
| 107 | + outputs: |
| 108 | + - name: opentelemetry |
| 109 | + match: '*' |
| 110 | + host: 192.168.56.4 |
| 111 | + port: 2021 |
| 112 | +``` |
0 commit comments