Autodiscover: pass paths through builders instead of global paths

[orestisfl]

Proposed commit message

Thread an explicit *paths.Path parameter through the autodiscover subsystem (NewAutodiscover → BuildProvider → AutodiscoverBuilder → NewBuilders → BuildBuilder → BuilderConstructor) instead of relying on the global paths.Paths variable.

Also makes NewLogHints unexported (newLogHints) since it is only used via the autodiscover registry.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

How to test this PR locally

Create a filebeat.yml with hints-based autodiscover:

filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/lib/docker/containers/${data.container.id}/*.log

output.console:
  pretty: true

Run filebeat and start a container with hint labels:

./filebeat -e -c filebeat.yml
# In another terminal:
docker run --label co.elastic.logs/module=nginx --rm nginx

Related issues

• Closes [beatreceiver] replace global paths in filebeat autodiscover hints builder #49460

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 859986c6-81d2-4a34-89e5-ba72f5324f48

📥 Commits

Reviewing files that changed from the base of the PR and between bde0009 and c9650af.

📒 Files selected for processing (1)

• filebeat/beater/filebeat.go

✅ Files skipped from review due to trivial changes (1)

• filebeat/beater/filebeat.go

---

📝 Walkthrough

Walkthrough

This change threads a *paths.Path parameter through the autodiscover initialization and builder/provider factory APIs. Constructor signatures for NewAutodiscover, builder and provider constructors, and registry Build functions were extended to accept a path *paths.Path and callers were updated to pass a paths instance or nil. Filebeat’s hints builder and related tests were changed to use an explicit paths object instead of the global paths singleton.

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	PR successfully threads *paths.Path through autodiscover subsystem and replaces global paths.Paths dependency in hints builder, addressing #49460 requirements.
Out of Scope Changes check	✅ Passed	All changes are scoped to autodiscover parameter threading and related builder signature updates; no unrelated modifications detected.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

• 🛠️ Update Documentation: Commit on current branch
• 🛠️ Update Documentation: Create PR

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[elasticmachine]

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

[gizas]

My only concern is if we have tested this in case of an upgrade from a previous version where the hints.paths dont exist.
What should happen in this case?

[gizas]

My only concern is if we have tested this in case of an upgrade from a previous version where the hints.paths dont exist. What should happen in this case?

We synced with Orestis, my comment was not accurate as this paths parameter refers to internal hints builder configuration.

[github-actions]

TL;DR

Both failed jobs are blocked by the same transient infrastructure error while provisioning the ECH test deployment: 502: Bad Gateway (POST /api/v1/deployments). Immediate action is to re-run the two FIPS ECH integration jobs.

Remediation

• Re-run:
  • x-pack/filebeat: FIPS ECH Integration Tests
  • x-pack/metricbeat: FIPS ECH Integration Tests
• If retries keep failing, investigate ECH API availability for deployment creation and use the logged request_id values for provider-side tracing.

Investigation details

Root Cause

The failures occur before beat test logic runs, during Terraform provisioning of the ECH deployment at testing/terraform-ech/ech.tf:54 (resource "ec_deployment" "default" {).

Evidence

• Build: https://buildkite.com/elastic/beats/builds/43802
• Job/step:
  • beats-xpack-filebeat build https://buildkite.com/elastic/beats-xpack-filebeat/builds/29751
  • beats-xpack-metricbeat build https://buildkite.com/elastic/beats-xpack-metricbeat/builds/30268
• Key log excerpts (both jobs):
  • Error: failed creating deployment
  • on ech.tf line 54, in resource "ec_deployment" "default":
  • api error: 1 error occurred:
  • * 502: Bad Gateway (POST /api/v1/deployments)
  • filebeat request_id: ojefzdtdyn1n9z6y0dlttri6frsc0m1p1d9ppnw8h1gqv5p9j6sa95dk7na6vz1t
  • metricbeat request_id: p6nnxn3m654ntjbjon8m5jpmhqr3bay3tqgz0o5g2y49u2k73djvrvmjg6xdsomv

Verification

• Not run locally; failure is in external ECH deployment provisioning (POST /api/v1/deployments) rather than repository test code paths.

Follow-up

If this reproduces after retries, correlate those request_id values with ECH service logs and check for regional/provider-side instability.

Note

🔒 Integrity filtering filtered 1 item

Integrity filtering activated and filtered the following item during workflow execution.
This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

• pr:Autodiscover: pass paths through builders instead of global paths #49537 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[orestisfl]

@Mergifyio backport 8.19

[mergify]

backport 8.19

✅ Backports have been created

Details

• #50030 [8.19](backport #49537) Autodiscover: pass paths through builders instead of global paths has been created for branch 8.19 but encountered conflicts

Cherry-pick of 31d1c30 has failed:

On branch mergify/bp/8.19/pr-49537
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit 31d1c309a.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   filebeat/autodiscover/builder/hints/logs.go
	modified:   filebeat/beater/filebeat.go
	modified:   heartbeat/autodiscover/builder/hints/monitors.go
	modified:   heartbeat/beater/heartbeat.go
	modified:   libbeat/autodiscover/autodiscover.go
	modified:   libbeat/autodiscover/builder.go
	modified:   libbeat/autodiscover/builder_test.go
	modified:   libbeat/autodiscover/provider.go
	modified:   libbeat/autodiscover/providers/docker/docker.go
	modified:   libbeat/autodiscover/providers/docker/docker_integration_test.go
	modified:   libbeat/autodiscover/providers/jolokia/jolokia.go
	modified:   libbeat/autodiscover/providers/kubernetes/config_test.go
	modified:   libbeat/autodiscover/providers/kubernetes/kubernetes.go
	modified:   metricbeat/autodiscover/builder/hints/metrics.go
	modified:   metricbeat/beater/metricbeat.go
	modified:   x-pack/libbeat/autodiscover/providers/aws/ec2/provider.go
	modified:   x-pack/libbeat/autodiscover/providers/aws/elb/provider.go
	modified:   x-pack/libbeat/autodiscover/providers/nomad/nomad.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   filebeat/autodiscover/builder/hints/logs_test.go
	both modified:   libbeat/autodiscover/autodiscover_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally