-
Notifications
You must be signed in to change notification settings - Fork 540
fix confusing/misleading info about parser support in YAML config #2133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
fix confusing/misleading info about parser support in YAML config #2133
Conversation
Signed-off-by: Alexa Kreizinger <[email protected]>
Signed-off-by: Alexa Kreizinger <[email protected]>
Signed-off-by: Alexa Kreizinger <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a comment for the top level elements of YAML configuration.
| - `env` | ||
| - `includes` | ||
| - `service` | ||
| - `pipeline` | ||
| - `inputs` | ||
| - `outputs` | ||
| - `filters` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
parsers can handle in YAML type of configurations:
This is an example configuration of YAML but it's still valid for the format:
env:
FLB_ENV: "1"
parsers:
- name: json
format: json
plugins:
- /opt/fluent-bit/plugins/custom.so
service:
flush: 1
log_level: info
pipeline:
inputs:
- name: tail
path: /var/log/syslog
outputs:
- name: stdout
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤔 what does an example configuration like this do? does it apply a JSON parser to all inputs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean, parsers, multiline_parsers, and plugins are able to be handled in the same YAML configurations.
But in classix format, parsers, multiline_parsers, and plugins should be handled with parsers_file and plugins_file and loaded as separated other files.
This could be an advantage of YAML format of Fluent BIt configurations.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah, that makes sense :) so those are optional sections you can use to define custom parsers/plugins/etc
of the other sections that you can include in a YAML config, are any of them required? I'm wondering if it would be helpful to specify which ones you always need to include versus which ones are optional.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The suggested sections are all of optional. So, it's not needed to put in every configuration.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one minor nit, but looks good to me
🎉 ship it
Co-authored-by: Lynette Miles <[email protected]> Signed-off-by: Alexa Kreizinger <[email protected]>
bear with me... this is going to be the first PR in a series of PRs trying to clarify some confusing stuff related to parsers, processors, and other YAML configuration-related topics. but I'm starting with this because it was the tiny thread that kept unraveling the more I pulled at it.
long story short, I thought it was odd that one of the configuration file pages stated that YAML configuration files don't support parsers. then I realized that it's because YAML configuration files don't have a dedicated parsers section—which is to say, that parsers are defined on a per-plugin basis.
so this PR attempts to perform the minimum viable amount of cleanup to clear up that confusion. that said, I'm going to need to revisit some of the docs in this section to cut through other things that are potentially misleading and to consolidate duplicated/scattered info.