[8.19](backport #49537) Autodiscover: pass paths through builders instead of global paths

[mergify]

Proposed commit message

Thread an explicit *paths.Path parameter through the autodiscover subsystem (NewAutodiscover → BuildProvider → AutodiscoverBuilder → NewBuilders → BuildBuilder → BuilderConstructor) instead of relying on the global paths.Paths variable.

Also makes NewLogHints unexported (newLogHints) since it is only used via the autodiscover registry.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

How to test this PR locally

Create a filebeat.yml with hints-based autodiscover:

filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/lib/docker/containers/${data.container.id}/*.log

output.console:
  pretty: true

Run filebeat and start a container with hint labels:

./filebeat -e -c filebeat.yml
# In another terminal:
docker run --label co.elastic.logs/module=nginx --rm nginx

Related issues

• Closes [beatreceiver] replace global paths in filebeat autodiscover hints builder #49460

  ---

  This is an automatic backport of pull request Autodiscover: pass paths through builders instead of global paths #49537 done by Mergify.

[mergify]

Cherry-pick of 31d1c30 has failed:

On branch mergify/bp/8.19/pr-49537
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit 31d1c309a.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   filebeat/autodiscover/builder/hints/logs.go
	modified:   filebeat/beater/filebeat.go
	modified:   heartbeat/autodiscover/builder/hints/monitors.go
	modified:   heartbeat/beater/heartbeat.go
	modified:   libbeat/autodiscover/autodiscover.go
	modified:   libbeat/autodiscover/builder.go
	modified:   libbeat/autodiscover/builder_test.go
	modified:   libbeat/autodiscover/provider.go
	modified:   libbeat/autodiscover/providers/docker/docker.go
	modified:   libbeat/autodiscover/providers/docker/docker_integration_test.go
	modified:   libbeat/autodiscover/providers/jolokia/jolokia.go
	modified:   libbeat/autodiscover/providers/kubernetes/config_test.go
	modified:   libbeat/autodiscover/providers/kubernetes/kubernetes.go
	modified:   metricbeat/autodiscover/builder/hints/metrics.go
	modified:   metricbeat/beater/metricbeat.go
	modified:   x-pack/libbeat/autodiscover/providers/aws/ec2/provider.go
	modified:   x-pack/libbeat/autodiscover/providers/aws/elb/provider.go
	modified:   x-pack/libbeat/autodiscover/providers/nomad/nomad.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   filebeat/autodiscover/builder/hints/logs_test.go
	both modified:   libbeat/autodiscover/autodiscover_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[elasticmachine]

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[github-actions]

TL;DR

This Buildkite failure set is caused by unresolved merge-conflict markers committed in two test files. Remove the conflict blocks and keep one valid version in each file, then rerun CI.

Remediation

• Resolve merge markers in:
  • libbeat/autodiscover/autodiscover_test.go (conflict markers at lines 792/793/891)
  • filebeat/autodiscover/builder/hints/logs_test.go (conflict markers at lines 1109/1121/1135)
• Re-run the same checks (pre-commit run --all-files, make -C libbeat check update && make check-no-changes, make -C filebeat check update && make check-no-changes) to confirm all related jobs pass.

Investigation details

Root Cause

Classification: Code bug (invalid source from unresolved merge conflict).

All failing pre-commit jobs fail at check-merge-conflict, and both check/update jobs fail with Go parse errors in the same two files. The pip dependency warning is present, but it is not the terminating error.

In the PR commit content (f1c6f3611c912aae19031408af14e47ba7b97552), both files contain literal conflict markers:

• libbeat/autodiscover/autodiscover_test.go:792

  • <<<<<<< HEAD

• libbeat/autodiscover/autodiscover_test.go:793

  • =======

• libbeat/autodiscover/autodiscover_test.go:891

  • >>>>>>> 31d1c309a (Autodiscover: pass paths through builders instead of global paths (#49537))

• filebeat/autodiscover/builder/hints/logs_test.go:1109

  • <<<<<<< HEAD

• filebeat/autodiscover/builder/hints/logs_test.go:1121

  • =======

• filebeat/autodiscover/builder/hints/logs_test.go:1135

  • >>>>>>> 31d1c309a (Autodiscover: pass paths through builders instead of global paths (#49537))

Evidence

• Build: https://buildkite.com/elastic/beats/builds/43883
• Representative pre-commit failure (Filebeat: Run pre-commit):
  • hook id: check-merge-conflict
  • libbeat/autodiscover/autodiscover_test.go:792: Merge conflict string '<<<<<<<' found
  • filebeat/autodiscover/builder/hints/logs_test.go:1109: Merge conflict string '<<<<<<<' found
• Representative check/update failures:
  • beats-libbeat ... run-checkupdate:
    • autodiscover/autodiscover_test.go:792:1: expected declaration, found '<<'
    • `autodiscover/autodiscover_test.go:891:87: illegal character U+0023 '#'
  • filebeat ... run-checkupdate:
    • autodiscover/builder/hints/logs_test.go:1109:1: expected statement, found '<<'
    • `autodiscover/builder/hints/logs_test.go:1135:87: illegal character U+0023 '#'

Verification

• Not run locally in this workflow context; conclusions are based on Buildkite logs and the PR commit file contents.

Follow-up

After resolving the two conflict regions, CI should recover across all currently failing jobs since they all fail on the same marker detection/parse errors.

Note

🔒 Integrity filtering filtered 1 item

Integrity filtering activated and filtered the following item during workflow execution.
This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

• pr:[8.19](backport #49537) Autodiscover: pass paths through builders instead of global paths #50030 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[mergify]

This pull request has not been merged yet. Could you please review and merge it @orestisfl? 🙏