Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 58 additions & 31 deletions development/golang-output-plugins.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
# Golang Output Plugins
# Golang output plugins

Fluent Bit currently supports integration of Golang plugins built as shared objects for output plugins only. The interface for the Golang plugins is currently under development but is functional.
Fluent Bit supports integration of Golang plugins built as shared objects for output plugins only. The interface for the Golang plugins is currently under development but is functional.

## Getting Started
## Get started

Compile Fluent Bit with Golang support, e.g:
Compile Fluent Bit with Golang support:

```text
$ cd build/
$ cmake -DFLB_DEBUG=On -DFLB_PROXY_GO=On ../
$ make
```shell
cd build/
cmake -DFLB_DEBUG=On -DFLB_PROXY_GO=On ../
make
```

Once compiled, we can see a new option in the binary `-e` which stands for _external plugin_, e.g:
Once compiled, you can see the new `-e` option in the binary which stands for _external plugin_.

```text
$ bin/fluent-bit -h
Expand All @@ -30,13 +30,13 @@ Available Options
...
```

## Build a Go Plugin
## Build a Go plugin

The _fluent-bit-go_ package is available to assist developers in creating Go plugins.
The `fluent-bit-go` package is available to assist developers in creating Go plugins.

[https://github.com/fluent/fluent-bit-go](https://github.com/fluent/fluent-bit-go)

At a minimum, a Go plugin looks like this:
A minimum Go plugin looks like the following:

```go
package main
Expand Down Expand Up @@ -70,17 +70,17 @@ func main() {
}
```

the code above is a template to write an output plugin, it's really important to keep the package name as `main` and add an explicit `main()` function. This is a requirement as the code will be build as a shared library.
The previous code is a template to write an output plugin. It's important to keep the package name as `main` and add an explicit `main()` function. This is a requirement as the code will be build as a shared library.

To build the code above, use the following line:
To build the code, use the following line:

```bash
$ go build -buildmode=c-shared -o out_gstdout.so out_gstdout.go
go build -buildmode=c-shared -o out_gstdout.so out_gstdout.go
```

Once built, a shared library called `out\_gstdout.so` will be available. It's really important to double check the final .so file is what we expect. Doing a `ldd` over the library we should see something similar to this:
Once built, a shared library called `out_gstdout.so` will be available. Confirm the final `.so` file is as expected. When you use the `ldd` over the library should see something similar to this:

```text
```shell
$ ldd out_gstdout.so
linux-vdso.so.1 => (0x00007fff561dd000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc4aeef0000)
Expand All @@ -91,23 +91,23 @@ $ ldd out_gstdout.so
## Run Fluent Bit with the new plugin

```bash
$ bin/fluent-bit -e /path/to/out_gstdout.so -i cpu -o gstdout
bin/fluent-bit -e /path/to/out_gstdout.so -i cpu -o gstdout
```

## Configuration file

Fluent Bit can load / run Golang plugin using two configuration file.
Fluent Bit can load and run Golang plugins using two configuration files.

- Plugins configuration file
- [Main configuration file](../administration/configuring-fluent-bit/classic-mode/configuration-file.md)

### Plugins configuration file

| Key | Description | Default Value|
| ---- | ----------- | ------------ |
| Path | A path for a Golang plugin. | |
| Key | Description |
| ---- | ----------- |
| Path | A path for a Golang plugin. |

#### Example
#### Plugin file example

```python
[PLUGINS]
Expand All @@ -118,11 +118,11 @@ Fluent Bit can load / run Golang plugin using two configuration file.

The keys for Golang plugin available as of this version are described in the following table:

| Key | Description | Default Value|
| ---- | ----------- | ------------ |
| Plugins_file | Path for a plugins configuration file. A _plugins_ configuration file allows to define paths for external plugins, for an example [see here](https://github.com/fluent/fluent-bit/blob/master/conf/plugins.conf). | |
| Key | Description |
| ---- | ----------- |
| `Plugins_file` | Path for a plugins configuration file. A plugins configuration file let you define paths for external plugins. For example, [see here](https://github.com/fluent/fluent-bit/blob/master/conf/plugins.conf). |

#### Example
#### Main configuration file example

The following is an example of a main configuration file.

Expand All @@ -139,13 +139,40 @@ The following is an example of a main configuration file.

#### Config key constraint

Some config keys are reserved by Fluent Bit and must not be used by a custom plugin, they are: `alias`,`host`,`ipv6`,`listen`,`log_level`,`log_suppress_interval`,`match`,`match_regex`,`mem_buf_limit`,`port`,`retry_limit`,`routable`,`storage.pause_on_chunks_overlimit`, `storage.total_limit_size`, `storage.type`, `tag`,`threaded`,`tls`,`tls.ca_file`, `tls.ca_path`, `tls.crt_file`, `tls.debug`, `tls.key_file`, `tls.key_passwd`, `tls.verify`, `tls.vhost`, `workers`
The following configuration keys are reserved by Fluent Bit and must not be used by a custom plugin:

- `alias`
- `host`
- `ipv6`
- `listen`
- `log_level`
- `log_suppress_interval`
- `match`
- `match_regex`
- `mem_buf_limit`
- `port`
- `retry_limit`
- `routable`
- `storage.pause_on_chunks_overlimit`
- `storage.total_limit_size`
- `storage.type`
- `tag`
- `threaded`
- `tls`
- `tls.ca_file`
- `tls.ca_path`
- `tls.crt_file`
- `tls.debug`
- `tls.key_file`
- `tls.key_passwd`
- `tls.verify`
- `tls.vhost`
- `workers`

### Run using a configuration file

We can load a main configuration file using `-c` option.
Note: No need to specify a plugins configuration file from command line.
You can load a main configuration file using `-c` option. You don't need to specify a plugins configuration file from command line.

```text
fluent-bit -c fluent-bit.conf
```
```