[9.2](backport #49706) filebeat: port remaining harvester tests to filestream

[mergify]

Summary

Port the remaining test coverage from filebeat/tests/system/test_harvester.py into filebeat/input/filestream/input_integration_test.go, then remove the redundant Python test file.

Added Go integration tests

• TestFilestreamExceedBuffer (from test_exceed_buffer)
• TestFilestreamIgnoreSymlink (from test_ignore_symlink)
• TestFilestreamSymlinkAndFile (from test_symlink_and_file)
• TestFilestreamDecodeError (from test_decode_error)
• TestFilestreamDebugReader (from test_debug_reader)

Other changes

• Removed filebeat/tests/system/test_harvester.py.
• Enabled debug selector setup in TestFilestreamDebugReader via logp.TestingSetup(logp.WithSelectors("*")) so null-byte debug logging is observable in the integration harness.

Validation

I attempted to run the new tests with repeated counts, but this runner consistently OOM-killed compilation of transitive dependencies in go test before executing tests:

• GOMAXPROCS=1 GOGC=20 GOFLAGS='-p=1 -vet=off' go test -tags=integration ./filebeat/input/filestream -run '^TestFilestreamExceedBuffer$' -count=50
• GOMAXPROCS=1 GOGC=20 GOFLAGS='-p=1 -vet=off' go test -tags=integration ./filebeat/input/filestream -run '^TestFilestreamIgnoreSymlink$' -count=50

Representative failure:

github.com/elastic/go-elasticsearch/v8/typedapi/types: .../compile: signal: killed
FAIL github.com/elastic/beats/v7/filebeat/input/filestream [build failed]

Given the environment memory limits, I could not complete the requested high -count flake validation here. The code changes are committed and ready for CI validation in a larger runner.

• For tests: remove redundant Python tests that are already covered by existing Go tests #49669

Note

🔒 Integrity filtering filtered 1 item

Integrity filtering activated and filtered the following item during workflow execution.
This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

• issue:tests: remove redundant Python tests that are already covered by existing Go tests #49669 (issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)

---

What is this? | From workflow: Mention in Issue

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

---

This is an automatic backport of pull request #49706 done by [Mergify](https://mergify.com).

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[mauri870]

/test

[github-actions]

TL;DR

All 4 failing Agentbeat packaging jobs are failing before packaging starts because mage is not configured on the Buildkite worker for this pipeline (No preset version installed for command mage, exit 126). This is a CI configuration issue, not a code/test regression in agentbeat sources.

Remediation

• Ensure the beats-xpack-agentbeat pipeline sets ASDF_MAGE_VERSION (consistent with other Beats pipelines, e.g. 1.15.0) or otherwise installs/selects mage before running .buildkite/scripts/packaging/packaging.sh x-pack/agentbeat.
• Re-run the 4 failed packaging jobs after fixing tool setup; they currently fail at tool bootstrap, so no package build validation was reached.

Investigation details

Root Cause

The failing jobs stop at the mage package invocation path in .buildkite/scripts/packaging/packaging.sh (cd $BEAT_DIR then mage package, lines 22-24). On these workers, asdf reports only mage 1.14.0 installed in the pre-command phase, but when packaging starts it requires mage 1.15.0 and exits with status 126.

Related code evidence:

• .buildkite/scripts/packaging/packaging.sh:22-24 calls mage package directly.
• Other pipelines explicitly export ASDF_MAGE_VERSION: 1.15.0 (e.g. .buildkite/packaging.pipeline.yml:5), indicating expected CI setup pattern.

Evidence

• Build: https://buildkite.com/elastic/beats/builds/43256
• Failed jobs:
  • Agentbeat packaging Linux
  • Agentbeat packaging linux/amd64 FIPS
  • Agentbeat packaging linux/arm64 FIPS
  • Agentbeat packaging windows/arm64
• Key log excerpts (all four jobs show same failure):
  • No preset version installed for command mage
  • Please install a version by running ... asdf install mage 1.15.0
  • Error: The command exited with status 126

Verification

• Ran repository inspection only (read-only detective flow).
• No local tests were run, because failure occurs in CI environment bootstrap before build/test execution.

Follow-up

• After adding ASDF_MAGE_VERSION (or equivalent mage setup) to the beats-xpack-agentbeat pipeline/job environment, re-run the failed jobs to confirm packaging progresses past mage package.

Note

🔒 Integrity filtering filtered 1 item

Integrity filtering activated and filtered the following item during workflow execution.
This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

• pr:[9.2](backport #49706) filebeat: port remaining harvester tests to filestream #49792 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.