metricbeat beat stats metricset fetch failure_store metric

[leehinman]

Proposed commit message

In Metricbeat beat modules, stats metricset collect the failure_store metric that was added in #48068

Having failure_store as a metric in elastic-agent internal monitoring is a use case for this.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

Will add failure_store field to libbeat.output.events

Author's Checklist

• [ ]

How to test this PR locally

1. Configure metricbeat to monitor another beat
2. Check that target beat is exporting failure_store

curl http://localhost:5066/stats | jq '.libbeat.output.events'

3. Make sure documents in metricbeat have failure_store

Related issues

• Relates [beats receivers] Ship the Elasticsearch exporter's failure store metrics to Fleet elastic-agent#10642

Use cases

Screenshots

Logs

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @leehinman? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[github-actions]

✅ Vale Linting Results

No issues found on modified lines!

---

The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

[github-actions]

🔍 Preview links for changed docs

• docs/reference/metricbeat/exported-fields-beat.md

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9d5b7ffe-8d03-4749-845a-5f5621c53dbb

📥 Commits

Reviewing files that changed from the base of the PR and between af631bd and 87438f9.

📒 Files selected for processing (7)

• changelog/fragments/1773425590-add-failure_store-metric-to-beats-stats-metricset.yaml
• docs/reference/metricbeat/exported-fields-beat.md
• metricbeat/module/beat/_meta/fields.yml
• metricbeat/module/beat/fields.go
• metricbeat/module/beat/stats/_meta/fields.yml
• metricbeat/module/beat/stats/_meta/test/stats.800.json
• metricbeat/module/beat/stats/data.go

---

📝 Walkthrough

Walkthrough

This pull request adds the failure_store metric to the Beat module's stats metricset, tracking the count of events written to the failure store. Changes include a changelog fragment, updates to exported field documentation, new field definitions in multiple _meta/fields.yml files, test data updates, and a restructured schema in data.go that expands the Beat metrics API with additional nested structures for runtime, handles, libbeat pipeline metrics, queue metrics, and new config and state sections. The fields.go asset digest is updated to reflect these changes.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

• 🛠️ Update Documentation: Commit on current branch
• 🛠️ Update Documentation: Create PR

📝 Coding Plan

• Generate coding plan for human review comments

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

CodeRabbit can approve the review once all CodeRabbit's comments are resolved.

Enable the reviews.request_changes_workflow setting to automatically approve the review once all CodeRabbit's comments are resolved.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[vishaangelova]

Docs LGTM

[leehinman]

@elastic/stack-monitoring any chance you could review?

[consulthys]

@leehinman it looks good to Stack Monitoring, but do you have the corresponding PR in the ES repo to add the new field to the beats index templates?

[leehinman]

@leehinman it looks good to Stack Monitoring, but do you have the corresponding PR in the ES repo to add the new field to the beats index templates?

Thank You. Just opened:

elastic/elasticsearch#144883

[leehinman]

@leehinman it looks good to Stack Monitoring, but do you have the corresponding PR in the ES repo to add the new field to the beats index templates?

Thank You. Just opened:

elastic/elasticsearch#144883

elastic/elasticsearch#144883 is merged now, anything else blocking this?

[consulthys]

LGT Stack Monitoring

[consulthys]

@leehinman Thanks, looks good, fire away!

[pierrehilbert]

@leehinman I think we should backport this to 9.4, WDYT?

[consulthys]

@leehinman I think we should backport this to 9.4, WDYT?

If you do, don't forget to also backport the PR in the elasticsearch repo

[leehinman]

@leehinman I think we should backport this to 9.4, WDYT?

If you do, don't forget to also backport the PR in the elasticsearch repo

@consulthys can you help me with the backport in the elasticsearch repo? I added "auto-backport" and "9.4.1" labels, but it looks like I'm missing a step.

[consulthys]

@leehinman I think we should backport this to 9.4, WDYT?

If you do, don't forget to also backport the PR in the elasticsearch repo

@consulthys can you help me with the backport in the elasticsearch repo? I added "auto-backport" and "9.4.1" labels, but it looks like I'm missing a step.

Oh I'm not sure if the backport labels work after the PR is merged and closed. I need to look closer into it

Also noting that

• this PR has backport-skip which means we're not backporting.
• the ES PR wasn't merged before the 9.4 FF, which means it's not going out until 9.4.1