Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@
* [Process metrics](pipeline/inputs/process.md)
* [Prometheus remote write](pipeline/inputs/prometheus-remote-write.md)
* [Prometheus scrape Metrics](pipeline/inputs/prometheus-scrape-metrics.md)
* [Prometheus text file](pipeline/inputs/prometheus-textfile.md)
* [Random](pipeline/inputs/random.md)
* [Serial interface](pipeline/inputs/serial-interface.md)
* [Splunk](pipeline/inputs/splunk.md)
Expand Down
112 changes: 112 additions & 0 deletions pipeline/inputs/prometheus-textfile.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Prometheus Textfile

Check warning on line 1 in pipeline/inputs/prometheus-textfile.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Spelling] Spelling check: 'Textfile'? Raw Output: {"message": "[FluentBit.Spelling] Spelling check: 'Textfile'?", "location": {"path": "pipeline/inputs/prometheus-textfile.md", "range": {"start": {"line": 1, "column": 14}}}, "severity": "INFO"}

Check warning on line 1 in pipeline/inputs/prometheus-textfile.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Headings] 'Prometheus Textfile' should use sentence-style capitalization. Raw Output: {"message": "[FluentBit.Headings] 'Prometheus Textfile' should use sentence-style capitalization.", "location": {"path": "pipeline/inputs/prometheus-textfile.md", "range": {"start": {"line": 1, "column": 3}}}, "severity": "INFO"}

The **prometheus_textfile** input plugin allows Fluent Bit to read metrics from Prometheus text format files (.prom files) on the local filesystem. This plugin is particularly useful for collecting custom metrics that are written to files by external applications or scripts, similar to the Prometheus Node Exporter textfile collector.

Check warning on line 3 in pipeline/inputs/prometheus-textfile.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Spelling] Spelling check: 'textfile'? Raw Output: {"message": "[FluentBit.Spelling] Spelling check: 'textfile'?", "location": {"path": "pipeline/inputs/prometheus-textfile.md", "range": {"start": {"line": 3, "column": 317}}}, "severity": "INFO"}

Check warning on line 3 in pipeline/inputs/prometheus-textfile.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Simplicity] Avoid words like "useful" that imply ease of use, because the user may find this action difficult. Raw Output: {"message": "[FluentBit.Simplicity] Avoid words like \"useful\" that imply ease of use, because the user may find this action difficult.", "location": {"path": "pipeline/inputs/prometheus-textfile.md", "range": {"start": {"line": 3, "column": 177}}}, "severity": "WARNING"}

Check warning on line 3 in pipeline/inputs/prometheus-textfile.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Spelling] Spelling check: 'prometheus_textfile'? Raw Output: {"message": "[FluentBit.Spelling] Spelling check: 'prometheus_textfile'?", "location": {"path": "pipeline/inputs/prometheus-textfile.md", "range": {"start": {"line": 3, "column": 7}}}, "severity": "INFO"}

## Configuration Parameters

Check warning on line 5 in pipeline/inputs/prometheus-textfile.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Headings] 'Configuration Parameters' should use sentence-style capitalization. Raw Output: {"message": "[FluentBit.Headings] 'Configuration Parameters' should use sentence-style capitalization.", "location": {"path": "pipeline/inputs/prometheus-textfile.md", "range": {"start": {"line": 5, "column": 4}}}, "severity": "INFO"}

| Key | Description | Default |
|-----|-------------|---------|
| path | File or directory path pattern. Supports glob patterns with `*` wildcard (e.g., `/var/lib/prometheus/*.prom`) | |

Check failure on line 9 in pipeline/inputs/prometheus-textfile.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Latin] Use 'for example' instead of 'e.g.,'. Raw Output: {"message": "[FluentBit.Latin] Use 'for example' instead of 'e.g.,'.", "location": {"path": "pipeline/inputs/prometheus-textfile.md", "range": {"start": {"line": 9, "column": 84}}}, "severity": "ERROR"}
| scrape_interval | Interval in seconds between file scans | 10s |

Check warning on line 10 in pipeline/inputs/prometheus-textfile.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Units] Put a nonbreaking space between the number and the unit in '10s'. Raw Output: {"message": "[FluentBit.Units] Put a nonbreaking space between the number and the unit in '10s'.", "location": {"path": "pipeline/inputs/prometheus-textfile.md", "range": {"start": {"line": 10, "column": 62}}}, "severity": "INFO"}

Check warning on line 10 in pipeline/inputs/prometheus-textfile.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Spelling] Spelling check: 'scrape_interval'? Raw Output: {"message": "[FluentBit.Spelling] Spelling check: 'scrape_interval'?", "location": {"path": "pipeline/inputs/prometheus-textfile.md", "range": {"start": {"line": 10, "column": 3}}}, "severity": "INFO"}

## Getting Started

Check warning on line 12 in pipeline/inputs/prometheus-textfile.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Headings] 'Getting Started' should use sentence-style capitalization. Raw Output: {"message": "[FluentBit.Headings] 'Getting Started' should use sentence-style capitalization.", "location": {"path": "pipeline/inputs/prometheus-textfile.md", "range": {"start": {"line": 12, "column": 4}}}, "severity": "INFO"}

### Simple Configuration

Check warning on line 14 in pipeline/inputs/prometheus-textfile.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Headings] 'Simple Configuration' should use sentence-style capitalization. Raw Output: {"message": "[FluentBit.Headings] 'Simple Configuration' should use sentence-style capitalization.", "location": {"path": "pipeline/inputs/prometheus-textfile.md", "range": {"start": {"line": 14, "column": 5}}}, "severity": "INFO"}

The following configuration will monitor `/var/lib/prometheus/textfile` directory for `.prom` files every 15 seconds:

```yaml
pipeline:
inputs:
- name: prometheus_textfile
tag: custom_metrics
path: '/var/lib/prometheus/textfile/*.prom'
scrape_interval: 15
outputs:
- name: prometheus_exporter
match: custom_metrics
host: 192.168.100.61
port: 2021
```

## Prometheus Text Format

The plugin expects files to be in the standard Prometheus text exposition format. Here's an example of a valid `.prom` file:

```
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]

# HELP custom_counter_total A custom counter metric
# TYPE custom_counter_total counter
custom_counter_total{instance="server1",job="myapp"} 42

# HELP custom_gauge A custom gauge metric
# TYPE custom_gauge gauge
custom_gauge{environment="production"} 1.23

# HELP custom_histogram_bucket A custom histogram
# TYPE custom_histogram_bucket histogram
custom_histogram_bucket{le="0.1"} 10
custom_histogram_bucket{le="0.5"} 25
custom_histogram_bucket{le="1.0"} 40
custom_histogram_bucket{le="+Inf"} 50
custom_histogram_sum 125.5
custom_histogram_count 50
```

## Use Cases

### Custom Application Metrics

Applications can write custom metrics to `.prom` files, and this plugin will collect them:

```bash
# Script writes metrics to file
echo "# HELP app_requests_total Total HTTP requests" > /var/lib/prometheus/textfile/app.prom
echo "# TYPE app_requests_total counter" >> /var/lib/prometheus/textfile/app.prom
echo "app_requests_total{status=\"200\"} 1500" >> /var/lib/prometheus/textfile/app.prom
echo "app_requests_total{status=\"404\"} 23" >> /var/lib/prometheus/textfile/app.prom
```

### Batch Job Metrics
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[markdownlint] reported by reviewdog 🐶
MD022/blanks-around-headings Headings should be surrounded by blank lines [Expected: 1; Actual: 0; Below] [Context: "### Batch Job Metrics"]

Cron jobs or batch processes can write completion metrics:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[markdownlint-fix] reported by reviewdog 🐶

Suggested change
Cron jobs or batch processes can write completion metrics:
Cron jobs or batch processes can write completion metrics:


```bash
#!/bin/bash
# Backup script writes completion metrics
BACKUP_START=$(date +%s)
# ... perform backup ...
BACKUP_END=$(date +%s)
DURATION=$((BACKUP_END - BACKUP_START))

cat > /var/lib/prometheus/textfile/backup.prom << EOF
# HELP backup_duration_seconds Time taken to complete backup
# TYPE backup_duration_seconds gauge
backup_duration_seconds ${DURATION}

# HELP backup_last_success_timestamp_seconds Last successful backup timestamp
# TYPE backup_last_success_timestamp_seconds gauge
backup_last_success_timestamp_seconds ${BACKUP_END}
EOF
```

### System Integration
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[markdownlint] reported by reviewdog 🐶
MD022/blanks-around-headings Headings should be surrounded by blank lines [Expected: 1; Actual: 0; Below] [Context: "### System Integration"]

External monitoring tools can write metrics that Fluent Bit will collect and forward.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[markdownlint-fix] reported by reviewdog 🐶

Suggested change
External monitoring tools can write metrics that Fluent Bit will collect and forward.
External monitoring tools can write metrics that Fluent Bit will collect and forward.


## Integration with Other Plugins

### OpenTelemetry Destination

```yaml
pipeline:
inputs:
- name: prometheus_textfile
tag: textfile_metrics
path: /var/lib/prometheus/textfile
- name: node_exporter_metrics
tag: system_metrics
scrape_interval: 15
outputs:
- name: opentelemetry
match: '*'
host: 192.168.56.4
port: 2021
```
Loading