Skip to content

Conversation

@alexakreizinger
Copy link
Member

bear with me... this is going to be the first PR in a series of PRs trying to clarify some confusing stuff related to parsers, processors, and other YAML configuration-related topics. but I'm starting with this because it was the tiny thread that kept unraveling the more I pulled at it.

long story short, I thought it was odd that one of the configuration file pages stated that YAML configuration files don't support parsers. then I realized that it's because YAML configuration files don't have a dedicated parsers section—which is to say, that parsers are defined on a per-plugin basis.

so this PR attempts to perform the minimum viable amount of cleanup to clear up that confusion. that said, I'm going to need to revisit some of the docs in this section to cut through other things that are potentially misleading and to consolidate duplicated/scattered info.

Signed-off-by: Alexa Kreizinger <[email protected]>
Signed-off-by: Alexa Kreizinger <[email protected]>
Signed-off-by: Alexa Kreizinger <[email protected]>
@alexakreizinger alexakreizinger requested review from a team as code owners November 4, 2025 03:29
Copy link
Contributor

@cosmo0920 cosmo0920 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a comment for the top level elements of YAML configuration.

Comment on lines +12 to +18
- `env`
- `includes`
- `service`
- `pipeline`
- `inputs`
- `outputs`
- `filters`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parsers can handle in YAML type of configurations:

This is an example configuration of YAML but it's still valid for the format:

env:
  FLB_ENV: "1"

parsers:
  - name: json
    format: json

plugins:
  - /opt/fluent-bit/plugins/custom.so

service:
  flush: 1
  log_level: info

pipeline:
  inputs:
    - name: tail
      path: /var/log/syslog
  outputs:
    - name: stdout

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 what does an example configuration like this do? does it apply a JSON parser to all inputs?

Copy link
Contributor

@cosmo0920 cosmo0920 Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, parsers, multiline_parsers, and plugins are able to be handled in the same YAML configurations.
But in classix format, parsers, multiline_parsers, and plugins should be handled with parsers_file and plugins_file and loaded as separated other files.
This could be an advantage of YAML format of Fluent BIt configurations.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, that makes sense :) so those are optional sections you can use to define custom parsers/plugins/etc

of the other sections that you can include in a YAML config, are any of them required? I'm wondering if it would be helpful to specify which ones you always need to include versus which ones are optional.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The suggested sections are all of optional. So, it's not needed to put in every configuration.

@eschabell eschabell added waiting-for-user Waiting for user/contributors feedback or requested changes waiting-on-review Waiting on a review from mainteners labels Nov 4, 2025
Copy link
Contributor

@esmerel esmerel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one minor nit, but looks good to me
🎉 ship it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting-for-user Waiting for user/contributors feedback or requested changes waiting-on-review Waiting on a review from mainteners

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants