[Filebeat] Fix flaky filestream delete retry integration subtest

[github-actions]

Summary

This updates TestFilestreamDeleteFileRemoveRetries/file_is_externally_removed to wait for the asynchronous retry counter deterministically.

• Replaced the immediate count.Load() != 2 assertion with require.Eventually(...)
• Wait window: 1s with 1ms polling
• Keeps the same assertion message while removing timing race on goroutine scheduling

Validation

• ./script/stresstest.sh --tags integration ./filebeat/input/filestream '^TestFilestreamDeleteFileRemoveRetries$' -p 1

Both commands passed in this environment.

• Fixes [Flaky Test] github.com/elastic/beats/v7/filebeat/input/filestream TestFilestreamDeleteFileRemoveRetries/file_is_externally_removed #49987

Note

🔒 Integrity filtering filtered 1 item

Integrity filtering activated and filtered the following item during workflow execution.
This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

• issue:[Flaky Test] github.com/elastic/beats/v7/filebeat/input/filestream TestFilestreamDeleteFileRemoveRetries/file_is_externally_removed #49987 (issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)

---

What is this? | From workflow: Mention in Issue

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @github-actions[bot]? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: cd269041-1987-48d7-98d2-265ea7d5fe07

📥 Commits

Reviewing files that changed from the base of the PR and between 83a9c69 and 3ef688d.

📒 Files selected for processing (1)

• filebeat/input/filestream/input_delete_integration_test.go

---

📝 Walkthrough

Walkthrough

The integration test TestFilestreamDeleteFileRemoveRetries (filebeat/input/filestream/input_delete_integration_test.go) was changed to synchronize on a channel that is closed when the second removeFn call occurs instead of performing an immediate synchronous assertion that removeFn had been called twice and checking deleteDone.Load(). The removeFn hook now closes the retried channel on the second invocation and reports an error if invoked more than twice; the prior immediate fatal checks were removed.

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	PR addresses the flaky test by replacing immediate assertion with deterministic polling (require.Eventually), eliminating timing-dependent goroutine scheduling race.
Out of Scope Changes check	✅ Passed	All changes are scoped to fixing the flaky test in TestFilestreamDeleteFileRemoveRetries/file_is_externally_removed; no unrelated modifications present.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix-filestream-delete-retries-flaky-49987-f00b308050c1171d

• 🛠️ Update Documentation: Commit on current branch
• 🛠️ Update Documentation: Create PR

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[orestisfl]

A suggestion about synchronization

[github-actions]

TL;DR

make -C filebeat check update failed because a test imports the Azure Event Hubs SDK without the /v2 module suffix, which introduces an undeclared module (github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs) and causes check-no-changes to fail after generated go.mod/go.sum edits.

Remediation

• Change the import in x-pack/filebeat/input/azureeventhub/v2_input_test.go:16 from github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs to github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs/v2.
• Re-run make -C filebeat check update (or minimally go test ./x-pack/filebeat/input/azureeventhub) and confirm go.mod/go.sum stay unchanged.

Investigation details

Root Cause

The new/updated test file x-pack/filebeat/input/azureeventhub/v2_input_test.go imports the non-versioned SDK package:

• x-pack/filebeat/input/azureeventhub/v2_input_test.go:16
  • "github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs"

But the codebase uses v2 everywhere else for this input (.../azeventhubs/v2). That mismatch makes Go resolve a new module (v1.4.0) and dirties module files during the check/update pipeline.

Evidence

• Build: https://buildkite.com/elastic/beats/builds/43934
• Job/step: Filebeat: Run check/update (make -C filebeat check update then make check-no-changes)
• Key log excerpt:
  • go: finding module for package github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs
  • go: found github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs in ... v1.4.0
  • diff --git a/go.mod b/go.mod (adds github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs v1.4.0)
  • go.mod: needs update
  • go.sum: needs update

Verification

• Ran: go test ./x-pack/filebeat/input/azureeventhub -run TestProcessReceivedEventsUpdatesProcessingTimeOnce -count=1
• Result: setup failed with no required module provides package github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs; to add it: go get ...

Follow-up

If the test intentionally requires the non-v2 module (unlikely given current package usage), then a deliberate go.mod/go.sum update is needed in the PR. Otherwise, switch the import to /v2 and the CI failure should clear.

Note

🔒 Integrity filtering filtered 1 item

Integrity filtering activated and filtered the following item during workflow execution.
This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

• pr:[Filebeat] Fix flaky filestream delete retry integration subtest #50037 (pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".)

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

@Mergifyio backport 8.19 9.3 9.4

[mergify]

backport 8.19 9.3 9.4

✅ Backports have been created

Details

• #50064 [8.19](backport #50037) [Filebeat] Fix flaky filestream delete retry integration subtest has been created for branch 8.19 but encountered conflicts

Cherry-pick of d27bf57 has failed:

On branch mergify/bp/8.19/pr-50037
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit d27bf5775.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	deleted by us:   filebeat/input/filestream/input_delete_integration_test.go

no changes added to commit (use "git add" and/or "git commit -a")

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

• #50065 [9.3](backport #50037) [Filebeat] Fix flaky filestream delete retry integration subtest has been created for branch 9.3
• #50066 [9.4](backport #50037) [Filebeat] Fix flaky filestream delete retry integration subtest has been created for branch 9.4