es_out: support Upstream Servers with configuration overriding

[mabrarov]

Implementation of Upstream feature for the Elasticsearch output plugin.

Refer to fluent/fluent-bit#7608 for the changes in Fluent Bit code.

Summary by CodeRabbit

• Documentation
  • Added Elasticsearch to the list of plugins that support Upstream.
  • Updated Elasticsearch docs with full Upstream configuration support, examples for Classic mode and YAML, and detailed per-key override guidance.
  • Added AWS Sigv4 authentication guidance and notes on how AWS/TLS settings interact with Upstream overrides.
  • Clarified write-operation behavior and field-name handling in Upstream contexts.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[mabrarov]

This pull request is waiting for review of fluent/fluent-bit#7608

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[cnorris-cs]

I left suggestions, but I've added my technical writing-based approval based on that to keep this moving.

Primarily, I was looking to help the table be more consistent and readable, while implementing some style-based suggestions for consistency.

[mabrarov]

Hi @cnorris-cs,

I applied some (!) parts of your suggestions as well as introduced additional changes for consistency with your suggestions. Please take a look at this pull request one more time when you have time. Some of the changes are implemented in dedicated commits (for ease of revert in case of concerns) - please let me know if I need to squash all commits into a single one.

Thank you for your review.

[esmerel]

Couple of minor stylistic suggestions for consistency, but these docs changes look good to me otherwise.

[eschabell]

@esmerel can you look at the Vale check failures, I'm seeing this error before:

Raw Output:
{"message": "[FluentBit.Headings] 'Classic mode Configuration File with Upstream' should use sentence-style capitalization.", "location": {"path": "pipeline/outputs/elasticsearch.md", "range": {"start": {"line": 163, "column": 6}}}, "severity": "INFO"}
Error: reviewdog: Too many results (annotations) in diff.
You may miss some annotations due to GitHub limitation for annotation created by logging command.
Please check GitHub Actions log console to see all results.

[esmerel]

Making suggestions to correct vale warnings.

[coderabbitai]

📝 Walkthrough

Walkthrough

Added Upstream support documentation for Elasticsearch: updated plugin list, introduced Upstream-aware parameter overrides, clarified AWS Sigv4/TLS interactions with Upstream NODE overrides, and added Classic/YAML configuration examples and usage snippets.

Changes

Cohort / File(s)	Summary
Upstream Plugin Reference administration/configuring-fluent-bit/classic-mode/upstream-servers.md	Added Elasticsearch to the list of output plugins documented as supporting Upstream.
Elasticsearch Output Documentation pipeline/outputs/elasticsearch.md	Reworked configuration table to indicate per-key "Overrides allowed"; added Overrides table, AWS Sigv4 + Upstream/TLS notes; added Classic and YAML Upstream configuration examples and sample upstream_servers blocks; clarified write operations and field-name behavior in Upstream context.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 I hop through docs with floppy cheer,

Elasticsearch now runs Upstream near;
YAML, classic configs all in view,
Nodes, overrides—ready to pursue! ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically describes the main change: adding Upstream Servers support with configuration overriding to the Elasticsearch output plugin documentation.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

---

📜 Recent review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9f9a5ee and 92e2433.

📒 Files selected for processing (1)

• pipeline/outputs/elasticsearch.md

🚧 Files skipped from review as they are similar to previous changes (1)

• pipeline/outputs/elasticsearch.md

_{✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Fix all issues with AI agents

In `@pipeline/outputs/elasticsearch.md`:
- Around line 207-218: The YAML example uses an unquoted asterisk for the match
value ("match: *"), which YAML treats as an anchor reference; update the example
so the match key uses a quoted literal asterisk (e.g., change match: * to match:
"*" ) to ensure the YAML parses as intended and preserves the wildcard
semantics.
- Around line 226-249: The YAML has an invalid unquoted wildcard in the pipeline
output config: change the match field from an unquoted asterisk to a quoted
string (e.g., change match: * to match: "*" or match: "'*'") in the pipeline
outputs block so the YAML parser accepts the wildcard value; update the pipeline
outputs entry with the quoted wildcard for the match key.

📜 Review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d8aa628 and 9f9a5ee.

📒 Files selected for processing (2)

• administration/configuring-fluent-bit/classic-mode/upstream-servers.md
• pipeline/outputs/elasticsearch.md

🧰 Additional context used

🧠 Learnings (1)

📚 Learning: 2025-12-12T14:30:18.461Z

Learnt from: kalavt
Repo: fluent/fluent-bit-docs PR: 2294
File: pipeline/inputs/kafka.md:147-168
Timestamp: 2025-12-12T14:30:18.461Z
Learning: In Fluent Bit v4.0.4+, when using AWS MSK IAM authentication (rdkafka.sasl.mechanism: aws_msk_iam), the rdkafka.security.protocol is automatically set to SASL_SSL and the AWS region is auto-detected from the broker hostname for standard MSK endpoints. The aws_msk_iam_cluster_arn parameter was removed - users only need to set rdkafka.sasl.mechanism: aws_msk_iam (and optionally aws_region for custom DNS/PrivateLink).

Applied to files:

• pipeline/outputs/elasticsearch.md

🔇 Additional comments (7)

administration/configuring-fluent-bit/classic-mode/upstream-servers.md (1)

8-8: LGTM! Elasticsearch correctly added to Upstream-supported plugins.

The addition properly documents Elasticsearch as an Upstream-capable plugin, consistent with the extensive Upstream configuration documentation added in the elasticsearch.md file.

pipeline/outputs/elasticsearch.md (6)

13-20: LGTM! Clear documentation of Upstream override capability.

The introduction and table header effectively explain which configuration parameters can be overridden in Upstream NODE sections.

---

21-58: LGTM! Comprehensive override documentation for Elasticsearch parameters.

The "Allows overrides" column correctly identifies which parameters can be overridden in Upstream NODE sections, with helpful explanations for Host and Port requirements.

---

66-68: LGTM! Important clarification about AWS Sigv4 and Upstream interaction.

This section correctly distinguishes between TLS parameters for AWS authentication (not overridable in NODE) and TLS parameters for Elasticsearch connections (overridable in NODE), preventing potential configuration confusion.

---

163-163: Note: Vale style check flagged this heading.

Per the PR objectives, Vale reported that this heading should use sentence-style capitalization. However, "Upstream" appears to be used as a proper noun (feature name) throughout the documentation, so the current capitalization may be intentional. Please verify whether this should be "Classic mode configuration file with upstream" or if "Upstream" should remain capitalized as a feature name.

---

165-201: LGTM! Well-documented Classic mode Upstream configuration.

The Classic mode configuration examples and Upstream server definitions are syntactically correct and clearly demonstrate how to configure Elasticsearch with Upstream load balancing.

---

220-224: LGTM! Helpful explanation of YAML Upstream configuration options.

The explanation clearly documents that Upstream configuration can use either Classic mode format or YAML format, and can optionally be embedded in the main configuration file when using YAML.

_{✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.}