metricbeat beat stats metricset fetch failure_store metric by leehinman · Pull Request #49452 · elastic/beats

leehinman · 2026-03-12T21:56:20Z

Proposed commit message

In Metricbeat beat modules, stats metricset collect the failure_store metric that was added in #48068

Having failure_store as a metric in elastic-agent internal monitoring is a use case for this.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

Will add failure_store field to libbeat.output.events

Author's Checklist

[ ]

How to test this PR locally

Configure metricbeat to monitor another beat
Check that target beat is exporting failure_store

curl http://localhost:5066/stats | jq '.libbeat.output.events'

Make sure documents in metricbeat have failure_store

Related issues

Relates [beats receivers] Ship the Elasticsearch exporter's failure store metrics to Fleet elastic-agent#10642

Use cases

Screenshots

Logs

github-actions · 2026-03-12T21:56:30Z

🤖 GitHub comments

Just comment with:

run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

mergify · 2026-03-12T21:57:27Z

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @leehinman? 🙏.
For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
backport-active-all is the label that automatically backports to all active branches.
backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

github-actions · 2026-03-12T21:57:43Z

✅ Vale Linting Results

No issues found on modified lines!

The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

github-actions · 2026-03-12T21:57:50Z

🔍 Preview links for changed docs

docs/reference/metricbeat/exported-fields-beat.md

coderabbitai · 2026-03-13T18:43:55Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9d5b7ffe-8d03-4749-845a-5f5621c53dbb

📥 Commits

Reviewing files that changed from the base of the PR and between af631bd and 87438f9.

📒 Files selected for processing (7)

changelog/fragments/1773425590-add-failure_store-metric-to-beats-stats-metricset.yaml
docs/reference/metricbeat/exported-fields-beat.md
metricbeat/module/beat/_meta/fields.yml
metricbeat/module/beat/fields.go
metricbeat/module/beat/stats/_meta/fields.yml
metricbeat/module/beat/stats/_meta/test/stats.800.json
metricbeat/module/beat/stats/data.go

📝 Walkthrough

Walkthrough

This pull request adds the failure_store metric to the Beat module's stats metricset, tracking the count of events written to the failure store. Changes include a changelog fragment, updates to exported field documentation, new field definitions in multiple _meta/fields.yml files, test data updates, and a restructured schema in data.go that expands the Beat metrics API with additional nested structures for runtime, handles, libbeat pipeline metrics, queue metrics, and new config and state sections. The fields.go asset digest is updated to reflect these changes.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

🛠️ Update Documentation: Commit on current branch
🛠️ Update Documentation: Create PR

📝 Coding Plan

Generate coding plan for human review comments

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

CodeRabbit can approve the review once all CodeRabbit's comments are resolved.

Enable the reviews.request_changes_workflow setting to automatically approve the review once all CodeRabbit's comments are resolved.

elasticmachine · 2026-03-13T19:55:01Z

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

vishaangelova

Docs LGTM

leehinman · 2026-03-23T18:40:35Z

@elastic/stack-monitoring any chance you could review?

consulthys · 2026-03-23T20:24:29Z

@leehinman it looks good to Stack Monitoring, but do you have the corresponding PR in the ES repo to add the new field to the beats index templates?

leehinman · 2026-03-24T18:15:15Z

@leehinman it looks good to Stack Monitoring, but do you have the corresponding PR in the ES repo to add the new field to the beats index templates?

Thank You. Just opened:

elastic/elasticsearch#144883

leehinman · 2026-04-15T01:28:21Z

@leehinman it looks good to Stack Monitoring, but do you have the corresponding PR in the ES repo to add the new field to the beats index templates?

Thank You. Just opened:

elastic/elasticsearch#144883

elastic/elasticsearch#144883 is merged now, anything else blocking this?

consulthys

LGT Stack Monitoring

consulthys · 2026-04-15T03:23:58Z

@leehinman Thanks, looks good, fire away!

pierrehilbert · 2026-04-15T06:39:36Z

@leehinman I think we should backport this to 9.4, WDYT?

consulthys · 2026-04-15T06:46:06Z

@leehinman I think we should backport this to 9.4, WDYT?

If you do, don't forget to also backport the PR in the elasticsearch repo

leehinman · 2026-04-15T18:12:03Z

@leehinman I think we should backport this to 9.4, WDYT?

If you do, don't forget to also backport the PR in the elasticsearch repo

@consulthys can you help me with the backport in the elasticsearch repo? I added "auto-backport" and "9.4.1" labels, but it looks like I'm missing a step.

consulthys · 2026-04-15T18:19:15Z

@leehinman I think we should backport this to 9.4, WDYT?

If you do, don't forget to also backport the PR in the elasticsearch repo

@consulthys can you help me with the backport in the elasticsearch repo? I added "auto-backport" and "9.4.1" labels, but it looks like I'm missing a step.

Oh I'm not sure if the backport labels work after the PR is merged and closed. I need to look closer into it

Also noting that

~~this PR has backport-skip which means we're not backporting.~~
the ES PR wasn't merged before the 9.4 FF, which means it's not going out until 9.4.1

(cherry picked from commit d1e97e9)

…50139) (cherry picked from commit d1e97e9) Co-authored-by: Lee E Hinman <57081003+leehinman@users.noreply.github.com>

botelastic Bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 12, 2026

mergify Bot assigned leehinman Mar 12, 2026

leehinman force-pushed the 10642_metricbeat_fetch_failure_store_metric branch from 26665b4 to fe95caa Compare March 13, 2026 14:31

metricbeat beat stats metricset fetch failure_store metric

87438f9

leehinman force-pushed the 10642_metricbeat_fetch_failure_store_metric branch from fe95caa to 87438f9 Compare March 13, 2026 18:39

leehinman marked this pull request as ready for review March 13, 2026 18:40

leehinman requested review from a team as code owners March 13, 2026 18:40

leehinman added backport-skip Skip notification from the automated backport with mergify Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team labels Mar 13, 2026

botelastic Bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Mar 13, 2026

vishaangelova approved these changes Mar 16, 2026

View reviewed changes

leehinman mentioned this pull request Mar 24, 2026

add failure_store to beats monitoring template elastic/elasticsearch#144883

Merged

pierrehilbert added the Team:Monitoring Stack Monitoring team label Mar 25, 2026

Merge branch 'main' into 10642_metricbeat_fetch_failure_store_metric

f672184

consulthys approved these changes Apr 15, 2026

View reviewed changes

leehinman merged commit d1e97e9 into elastic:main Apr 15, 2026
46 checks passed

leehinman added backport-9.4 and removed backport-skip Skip notification from the automated backport with mergify labels Apr 15, 2026

mergify Bot pushed a commit that referenced this pull request Apr 15, 2026

metricbeat beat stats metricset fetch failure_store metric (#49452)

a3d127b

(cherry picked from commit d1e97e9)

mergify Bot mentioned this pull request Apr 15, 2026

[9.4](backport #49452) metricbeat beat stats metricset fetch failure_store metric #50139

Merged

6 tasks

leehinman added a commit that referenced this pull request May 15, 2026

metricbeat beat stats metricset fetch failure_store metric (#49452) (#…

6747f3b

…50139) (cherry picked from commit d1e97e9) Co-authored-by: Lee E Hinman <57081003+leehinman@users.noreply.github.com>

Uh oh!

Conversation

leehinman commented Mar 12, 2026

Proposed commit message

Checklist

Disruptive User Impact

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

Uh oh!

github-actions Bot commented Mar 12, 2026

🤖 GitHub comments

Uh oh!

mergify Bot commented Mar 12, 2026

Uh oh!

github-actions Bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Vale Linting Results

Uh oh!

github-actions Bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Preview links for changed docs

Uh oh!

coderabbitai Bot commented Mar 13, 2026

Walkthrough

Uh oh!

elasticmachine commented Mar 13, 2026

Uh oh!

vishaangelova left a comment

Choose a reason for hiding this comment

Uh oh!

leehinman commented Mar 23, 2026

Uh oh!

consulthys commented Mar 23, 2026

Uh oh!

leehinman commented Mar 24, 2026

Uh oh!

leehinman commented Apr 15, 2026

Uh oh!

consulthys left a comment

Choose a reason for hiding this comment

Uh oh!

consulthys commented Apr 15, 2026

Uh oh!

pierrehilbert commented Apr 15, 2026

Uh oh!

consulthys commented Apr 15, 2026

Uh oh!

leehinman commented Apr 15, 2026

Uh oh!

Uh oh!

consulthys commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

github-actions Bot commented Mar 12, 2026 •

edited

Loading

github-actions Bot commented Mar 12, 2026 •

edited

Loading

consulthys commented Apr 15, 2026 •

edited

Loading