Skip to content

blog: your detector has a drop rate - #1577

Open
1991santhu wants to merge 2 commits into
falcosecurity:masterfrom
1991santhu:blog/detector-drop-rate
Open

blog: your detector has a drop rate#1577
1991santhu wants to merge 2 commits into
falcosecurity:masterfrom
1991santhu:blog/detector-drop-rate

Conversation

@1991santhu

Copy link
Copy Markdown

Adds a blog post on what Falco does when it cannot keep up, built entirely around the shipped defaults in falco.yaml at 0.44.1.

What it covers

  • syscall_event_drops.threshold is .1, so up to ten percent of syscall events in a second can be dropped before any action fires. The config comment already says what to do about it: set the threshold to zero to be alerted on any drops.
  • The resulting message is token-bucketed at rate: .03333 with max_burst: 1, i.e. one message every thirty seconds. The post treats that as sensible rather than a flaw, since a detector that floods its own alert channel during an event storm makes the storm worse, but notes the signal is deliberately coarse.
  • Why this lands differently for a detector than for a telemetry pipeline: a dropped syscall event produces silence that is indistinguishable from a quiet system, and the blind spot is correlated with load rather than randomly distributed.
  • Practical guidance: choose the threshold deliberately, treat a sustained drop rate as a capacity signal rather than something to tune away, and use metrics.output_rule with metrics.kernel_event_counters_enabled (and the per-CPU variant) for the actual counters instead of relying on the rate-limited notification.

Accuracy

Every value quoted is from falco.yaml at the 0.44.1 tag rather than the default branch, and the post says so in the second paragraph. The metrics recommendation follows Falco's own change notice, which points operators from the detailed drop-counter payload toward metrics.output_rule and metrics.kernel_event_counters_enabled.

No new claims about Falco internals beyond what the configuration and its comments state.

Housekeeping

Single file at content/en/blog/your-detector-has-a-drop-rate.md, front matter matching existing posts (title, date, author, slug, tags). No images. Commit is DCO signed off.

Happy to adjust the framing if the tone reads as criticism rather than documentation of a deliberate default, which is not the intent. I am also glad to trim it if it runs long for the blog.

Walks through the syscall_event_drops defaults and what they mean operationally:
up to ten percent of events in a second are lost before any action fires, and
the resulting message is token-bucketed to one every thirty seconds. Points
readers at metrics.kernel_event_counters_enabled for the actual numbers.

Signed-off-by: Santhosh Kumar Somarapu <somarapu.santhosh91@gmail.com>
@poiana

poiana commented Aug 10, 2026

Copy link
Copy Markdown

Welcome @1991santhu! It looks like this is your first PR to falcosecurity/falco-website 🎉

@poiana
poiana requested a review from mstemm August 10, 2026 15:15
@poiana

poiana commented Aug 10, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: 1991santhu
Once this PR has been reviewed and has the lgtm label, please assign vjjmiras for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@poiana
poiana requested a review from vjjmiras August 10, 2026 15:15
@poiana poiana added the size/L label Aug 10, 2026
Signed-off-by: Santhosh Kumar Somarapu <somarapu.santhosh91@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants