Skip to content

Commit df6c963

Browse files
committed
feat(influxdb3): add TOML configuration documentation to plugin library
- Add TOML configuration section to plugins-library index explaining usage - Add TOML config tables to all official plugin documentation files - Standardize TOML section format across plugins with config_file_path parameter - Update system-metrics plugin moved from examples to official - Remove redundant config_file_path from individual parameter tables - Ensure consistent placement before Installation/Requirements sections - Fix linting: replace 'e.g.' with 'for example' in system-metrics.md This completes the TOML configuration documentation updates from PR 6244
1 parent 55e128c commit df6c963

13 files changed

+458
-340
lines changed

content/shared/influxdb3-plugins/plugins-library/_index.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,46 @@ Plugins in this library include a JSON metadata schema in a docstring header tha
1515

1616
- the [InfluxDB 3 Explorer UI](/influxdb3/explorer/) to display and configure the plugin
1717
- automated testing and validation of plugins in the repository
18+
19+
## Using TOML Configuration Files
20+
21+
Many plugins in this library support using TOML configuration files to specify all plugin arguments. This is useful for complex configurations or when you want to version control your plugin settings.
22+
23+
### Important Requirements
24+
25+
**To use TOML configuration files, you must set the `PLUGIN_DIR` environment variable in the {{% product-name %}} host environment.** This is required in addition to the `--plugin-dir` flag when starting {{% product-name %}}:
26+
27+
- `--plugin-dir` tells {{% product-name %}} where to find plugin Python files
28+
- `PLUGIN_DIR` environment variable tells the plugins where to find TOML configuration files
29+
30+
### Set up TOML Configuration
31+
32+
1. **Start {{% product-name %}} with the PLUGIN_DIR environment variable set**:
33+
```bash
34+
PLUGIN_DIR=~/.plugins influxdb3 serve --node-id node0 --object-store file --data-dir ~/.influxdb3 --plugin-dir ~/.plugins
35+
```
36+
37+
2. **Copy or create a TOML configuration file in your plugin directory**:
38+
```bash
39+
# Example: copy a plugin's configuration template
40+
cp plugin_config_example.toml ~/.plugins/my_config.toml
41+
```
42+
43+
3. **Edit the TOML file** to match your requirements. The TOML file should contain all the arguments defined in the plugin's argument schema.
44+
45+
4. **Create a trigger with the `config_file_path` argument**:
46+
When creating a trigger, specify the `config_file_path` argument to point to your TOML configuration file.
47+
48+
- Specify only the filename (not the full path)
49+
- The file must be located under `PLUGIN_DIR`
50+
51+
```bash
52+
influxdb3 create trigger \
53+
--database mydb \
54+
--plugin-filename plugin_name.py \
55+
--trigger-spec "every:1d" \
56+
--trigger-arguments config_file_path=my_config.toml \
57+
my_trigger_name
58+
```
59+
60+
For more information on using TOML configuration files, see the project [README](https://github.com/influxdata/influxdb3_plugins/blob/master/README.md).

content/shared/influxdb3-plugins/plugins-library/examples/wal-plugin.md

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -10,15 +10,9 @@ The plugin can optionally double-count rows for a specified table to demonstrate
1010
|-----------|------|---------|-------------|
1111
| `double_count_table` | string | none | Table name for which to double the row count in write reports (for testing/demonstration) |
1212

13-
## Requirements
13+
## Installation steps
1414

15-
### Software requirements
16-
- InfluxDB 3 Core or InfluxDB 3 Enterprise with Processing Engine enabled
17-
- No additional Python packages required (uses built-in libraries)
18-
19-
### Installation steps
20-
21-
1. Start InfluxDB 3 with plugin support:
15+
1. Start {{% product-name %}} with the Processing Engine enabled (`--plugin-dir /path/to/plugins`)
2216
```bash
2317
influxdb3 serve \
2418
--node-id node0 \

content/shared/influxdb3-plugins/plugins-library/official/basic-transformation.md

Lines changed: 35 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -31,22 +31,43 @@ The plugin supports both scheduled batch processing of historical data and real-
3131
| `excluded_fields` | string | none | Dot-separated list of fields to exclude |
3232
| `filters` | string | none | Query filters. Format: `'field:"operator value"'` |
3333

34-
### Advanced parameters
34+
### TOML configuration
3535

36-
| Parameter | Type | Default | Description |
37-
|-----------|------|---------|-------------|
38-
| `config_file_path` | string | none | Path to TOML config file relative to PLUGIN_DIR |
36+
| Parameter | Type | Default | Description |
37+
|--------------------|--------|---------|----------------------------------------------------------------------------------|
38+
| `config_file_path` | string | none | TOML config file path relative to `PLUGIN_DIR` (required for TOML configuration) |
39+
40+
*To use a TOML configuration file, set the `PLUGIN_DIR` environment variable and specify the `config_file_path` in the trigger arguments.* This is in addition to the `--plugin-dir` flag when starting InfluxDB 3.
41+
42+
#### Example TOML configurations
3943

40-
## Requirements
44+
- [basic_transformation_config_scheduler.toml](https://github.com/influxdata/influxdb3_plugins/blob/master/influxdata/basic_transformation/basic_transformation_config_scheduler.toml) - for scheduled triggers
45+
- [basic_transformation_config_data_writes.toml](https://github.com/influxdata/influxdb3_plugins/blob/master/influxdata/basic_transformation/basic_transformation_config_data_writes.toml) - for data write triggers
46+
47+
```bash
48+
influxdb3 create trigger \
49+
--database mydb \
50+
--plugin-filename basic_transformation.py \
51+
--trigger-spec "every:1d" \
52+
--trigger-arguments config_file_path=basic_transformation_config_scheduler.toml \
53+
basic_transform_trigger
54+
```
4155

42-
### Software requirements
43-
- InfluxDB 3 Core or Enterprise with Processing Engine enabled
44-
- Python packages:
45-
- `pint` (for unit conversions)
56+
For more information on using TOML configuration files, see the Using TOML Configuration Files section in the [influxdb3_plugins
57+
/README.md](https://github.com/influxdata/influxdb3_plugins/blob/master/README.md).
4658

47-
### Installation steps
59+
## Schema requirements
4860

49-
1. Start InfluxDB 3 with plugin support:
61+
The plugin assumes that the table schema is already defined in the database, as it relies on this schema to retrieve field and tag names required for processing.
62+
63+
> [!WARNING]
64+
> #### Requires existing schema
65+
>
66+
> By design, the plugin returns an error if the schema doesn't exist or doesn't contain the expected columns.
67+
68+
## Installation steps
69+
70+
1. Start {{% product-name %}} with the Processing Engine enabled (`--plugin-dir /path/to/plugins`)
5071
```bash
5172
influxdb3 serve \
5273
--node-id node0 \
@@ -56,6 +77,9 @@ The plugin supports both scheduled batch processing of historical data and real-
5677
```
5778

5879
2. Install required Python packages:
80+
81+
- `pint` (for unit conversions)
82+
5983
```bash
6084
influxdb3 install package pint
6185
```
@@ -174,52 +198,6 @@ influxdb3 create trigger \
174198
status_updater
175199
```
176200

177-
## Using TOML Configuration Files
178-
179-
This plugin supports using TOML configuration files to specify all plugin arguments. This is useful for complex configurations or when you want to version control your plugin settings.
180-
181-
### Important Requirements
182-
183-
**To use TOML configuration files, you must set the `PLUGIN_DIR` environment variable in the InfluxDB 3 host environment.** This is required in addition to the `--plugin-dir` flag when starting InfluxDB 3:
184-
185-
- `--plugin-dir` tells InfluxDB 3 where to find plugin Python files
186-
- `PLUGIN_DIR` environment variable tells the plugins where to find TOML configuration files
187-
188-
### Setting Up TOML Configuration
189-
190-
1. **Start InfluxDB 3 with the PLUGIN_DIR environment variable set**:
191-
```bash
192-
PLUGIN_DIR=~/.plugins influxdb3 serve --node-id node0 --object-store file --data-dir ~/.influxdb3 --plugin-dir ~/.plugins
193-
```
194-
195-
2. **Copy the example TOML configuration file to your plugin directory**:
196-
```bash
197-
cp basic_transformation_config_scheduler.toml ~/.plugins/
198-
# or for data writes:
199-
cp basic_transformation_config_data_writes.toml ~/.plugins/
200-
```
201-
202-
3. **Edit the TOML file** to match your requirements. The TOML file contains all the arguments defined in the plugin's argument schema (see the JSON schema in the docstring at the top of basic_transformation.py).
203-
204-
4. **Create a trigger using the `config_file_path` argument**:
205-
```bash
206-
influxdb3 create trigger \
207-
--database mydb \
208-
--plugin-filename basic_transformation.py \
209-
--trigger-spec "every:1d" \
210-
--trigger-arguments config_file_path=basic_transformation_config_scheduler.toml \
211-
basic_transform_trigger
212-
```
213-
214-
### Important Notes
215-
- The `PLUGIN_DIR` environment variable must be set when starting InfluxDB 3 for TOML configuration to work
216-
- When using `config_file_path`, specify only the filename (not the full path)
217-
- The TOML file must be located in the directory specified by `PLUGIN_DIR`
218-
- All parameters in the TOML file will override any command-line arguments
219-
- Example TOML configuration files are provided:
220-
- `basic_transformation_config_scheduler.toml` - for scheduled triggers
221-
- `basic_transformation_config_data_writes.toml` - for data write triggers
222-
223201
## Code overview
224202

225203
### Files

content/shared/influxdb3-plugins/plugins-library/official/downsampler.md

Lines changed: 18 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -36,24 +36,32 @@ Each downsampled record includes metadata about the original data points compres
3636
| `target_database` | string | "default" | Database for storing downsampled data |
3737
| `max_retries` | integer | 5 | Maximum number of retries for write operations |
3838
| `batch_size` | string | "30d" | Time interval for batch processing (HTTP mode only) |
39-
| `config_file_path` | string | none | Path to TOML config file relative to PLUGIN_DIR |
4039

41-
### Metadata columns
40+
### TOML configuration
41+
42+
| Parameter | Type | Default | Description |
43+
|--------------------|--------|---------|----------------------------------------------------------------------------------|
44+
| `config_file_path` | string | none | TOML config file path relative to `PLUGIN_DIR` (required for TOML configuration) |
45+
46+
*To use a TOML configuration file, set the `PLUGIN_DIR` environment variable and specify the `config_file_path` in the trigger arguments.* This is in addition to the `--plugin-dir` flag when starting InfluxDB 3.
47+
48+
#### Example TOML configuration
49+
50+
[downsampling_config_scheduler.toml](downsampling_config_scheduler.toml)
51+
52+
For more information on using TOML configuration files, see the Using TOML Configuration Files section in the [influxdb3_plugins
53+
/README.md](/README.md).
54+
55+
## Schema management
4256

4357
Each downsampled record includes three additional metadata columns:
4458
- `record_count`—the number of original points compressed into this single downsampled row
4559
- `time_from`—the minimum timestamp among the original points in the interval
4660
- `time_to`—the maximum timestamp among the original points in the interval
4761

48-
## Requirements
62+
## Installation steps
4963

50-
### Software requirements
51-
- InfluxDB 3 Core or Enterprise with Processing Engine enabled
52-
- Python packages: No additional packages required
53-
54-
### Installation steps
55-
56-
1. Start InfluxDB 3 with plugin support:
64+
1. Start {{% product-name %}} with the Processing Engine enabled (`--plugin-dir /path/to/plugins`)
5765
```bash
5866
influxdb3 serve \
5967
--node-id node0 \
@@ -181,46 +189,6 @@ curl -X POST http://localhost:8181/api/v3/engine/downsample \
181189
}'
182190
```
183191

184-
## Using TOML Configuration Files
185-
186-
This plugin supports using TOML configuration files for complex configurations.
187-
188-
### Important Requirements
189-
190-
**To use TOML configuration files, you must set the `PLUGIN_DIR` environment variable in the InfluxDB 3 host environment:**
191-
192-
```bash
193-
PLUGIN_DIR=~/.plugins influxdb3 serve --node-id node0 --object-store file --data-dir ~/.influxdb3 --plugin-dir ~/.plugins
194-
```
195-
196-
### Example TOML Configuration
197-
198-
```toml
199-
# downsampling_config_scheduler.toml
200-
source_measurement = "cpu"
201-
target_measurement = "cpu_hourly"
202-
target_database = "analytics"
203-
interval = "1h"
204-
window = "6h"
205-
calculations = "avg"
206-
specific_fields = "usage_user.usage_system.usage_idle"
207-
max_retries = 3
208-
209-
[tag_values]
210-
host = ["server1", "server2", "server3"]
211-
```
212-
213-
### Create trigger using TOML config
214-
215-
```bash
216-
influxdb3 create trigger \
217-
--database mydb \
218-
--plugin-filename downsampler.py \
219-
--trigger-spec "every:1h" \
220-
--trigger-arguments config_file_path=downsampling_config_scheduler.toml \
221-
downsample_trigger
222-
```
223-
224192
## Code overview
225193

226194
### Files

content/shared/influxdb3-plugins/plugins-library/official/forecast-error-evaluator.md

Lines changed: 20 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -68,18 +68,24 @@ It includes debounce logic to suppress transient anomalies and supports multi-ch
6868
| `twilio_from_number` | string | required | Twilio sender number (for example, `"+1234567890"`) |
6969
| `twilio_to_number` | string | required | Recipient number (for example, `"+0987654321"`) |
7070

71-
## Requirements
71+
### TOML configuration
7272

73-
### Software requirements
74-
- InfluxDB 3 Core or Enterprise with Processing Engine enabled
75-
- Notification Sender Plugin for InfluxDB 3 (required for notifications)
76-
- Python packages:
77-
- `pandas` (for data processing)
78-
- `requests` (for HTTP notifications)
73+
| Parameter | Type | Default | Description |
74+
|--------------------|--------|---------|----------------------------------------------------------------------------------|
75+
| `config_file_path` | string | none | TOML config file path relative to `PLUGIN_DIR` (required for TOML configuration) |
76+
77+
*To use a TOML configuration file, set the `PLUGIN_DIR` environment variable and specify the `config_file_path` in the trigger arguments.* This is in addition to the `--plugin-dir` flag when starting InfluxDB 3.
78+
79+
#### Example TOML configuration
80+
81+
[forecast_error_config_scheduler.toml](https://github.com/influxdata/influxdb3_plugins/blob/master/influxdata/forecast_error_evaluator/forecast_error_config_scheduler.toml)
82+
83+
For more information on using TOML configuration files, see the Using TOML Configuration Files section in the [influxdb3_plugins
84+
/README.md](https://github.com/influxdata/influxdb3_plugins/blob/master/README.md).
7985

80-
### Installation steps
86+
## Installation steps
8187

82-
1. Start InfluxDB 3 with plugin support:
88+
1. Start {{% product-name %}} with the Processing Engine enabled (`--plugin-dir /path/to/plugins`)
8389
```bash
8490
influxdb3 serve \
8591
--node-id node0 \
@@ -89,12 +95,17 @@ It includes debounce logic to suppress transient anomalies and supports multi-ch
8995
```
9096

9197
2. Install required Python packages:
98+
99+
- `pandas` (for data processing)
100+
- `requests` (for HTTP notifications)
101+
92102
```bash
93103
influxdb3 install package pandas
94104
influxdb3 install package requests
95105
```
96106

97107
3. Install the Notification Sender Plugin (required):
108+
98109
```bash
99110
# Ensure notifier plugin is available in ~/.plugins/
100111
```

content/shared/influxdb3-plugins/plugins-library/official/influxdb-to-iceberg.md

Lines changed: 29 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -53,27 +53,40 @@ The plugin supports both scheduled batch transfers of historical data and on-dem
5353
- The `time` column is converted to `datetime64[us]` for Iceberg compatibility
5454
- Tables are created in format: `<namespace>.<table_name>`
5555

56-
## Requirements
56+
## Schema requirements
5757

58-
### Software requirements
59-
- InfluxDB 3 Core or Enterprise with Processing Engine enabled
60-
- Python packages:
61-
- `pandas` (for data manipulation)
62-
- `pyarrow` (for Parquet support)
63-
- `pyiceberg[catalog-options]` (for Iceberg integration)
58+
The plugin assumes that the Iceberg table schema is already defined in the database, as it relies on this schema to retrieve field and tag names required for processing.
6459

65-
### Installation steps
60+
> [!WARNING]
61+
> #### Requires existing schema
62+
>
63+
> By design, the plugin returns an error if the schema doesn't exist or doesn't contain the expected columns.
6664
67-
1. Start InfluxDB 3 with plugin support:
68-
```bash
69-
influxdb3 serve \
70-
--node-id node0 \
71-
--object-store file \
72-
--data-dir ~/.influxdb3 \
73-
--plugin-dir ~/.plugins
74-
```
65+
### TOML configuration
66+
67+
| Parameter | Type | Default | Description |
68+
|--------------------|--------|---------|----------------------------------------------------------------------------------|
69+
| `config_file_path` | string | none | TOML config file path relative to `PLUGIN_DIR` (required for TOML configuration) |
70+
71+
*To use a TOML configuration file, set the `PLUGIN_DIR` environment variable and specify the `config_file_path` in the trigger arguments.* This is in addition to the `--plugin-dir` flag when starting InfluxDB 3.
72+
73+
#### Example TOML configuration
74+
75+
[influxdb_to_iceberg_config_scheduler.toml](https://github.com/influxdata/influxdb3_plugins/blob/master/influxdata/influxdb_to_iceberg/influxdb_to_iceberg_config_scheduler.toml)
76+
77+
For more information on using TOML configuration files, see the Using TOML Configuration Files section in the [influxdb3_plugins
78+
/README.md](https://github.com/influxdata/influxdb3_plugins/blob/master/README.md).
79+
80+
## Installation steps
81+
82+
1. Start {{% product-name %}} with the Processing Engine enabled (`--plugin-dir /path/to/plugins`)
7583

7684
2. Install required Python packages:
85+
86+
- `pandas` (for data manipulation)
87+
- `pyarrow` (for Parquet support)
88+
- `pyiceberg[catalog-options]` (for Iceberg integration)
89+
7790
```bash
7891
influxdb3 install package pandas
7992
influxdb3 install package pyarrow

0 commit comments

Comments
 (0)