pkg/option: allow policy-filter-map-entries configurable via flag #4331

kyledong-suse · 2025-11-11T16:55:46Z

Description

This commit introduces a new flag to configure the number of entries in policy filter maps. This allows users to tune the map size based on workload scale and system resources, improving flexibility in policy handling.

Note: this commit only affects policies with k8s segmentation primitives (i.e., either podSelectors or namespaced policies).

Fixes: #4260

netlify · 2025-11-11T16:56:34Z

✅ Deploy Preview for tetragon ready!

Name	Link
🔨 Latest commit	`cc2b381`
🔍 Latest deploy log	https://app.netlify.com/projects/tetragon/deploys/694a31c6f40e0b0008c0e851
😎 Deploy Preview	https://deploy-preview-4331--tetragon.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

kyledong-suse · 2025-11-12T23:47:18Z

@mtardy and @kkourt, would you please review this PR when you get a chance. Thanks!

mtardy

Thanks for the patch, I think it's the right direction and a good idea, just need a few changes in how it's done :)

mtardy · 2025-11-20T18:13:28Z

bpf/process/policy_filter.h

 struct {
 	__uint(type, BPF_MAP_TYPE_HASH_OF_MAPS);
-	__uint(max_entries, POLICY_FILTER_MAX_POLICIES);
+	__uint(max_entries, 1); // will be resized by agent when needed


This is used by policy_filter_cgroup_maps just nearby. Maybe we would need to update both and remove that const?

Sure, I think it makes sense to me. I'll update both of them and remove that const.

@mtardy Given another think, the POLICY_FILTER_MAX_POLICIES in policy_filter_cgroup_maps represents how many policies can apply to a single cgroup (inner map size of policy_filter_cgroup_maps).
I’m fine to simplify and drive both policy_filter_maps and policy_filter_cgroup_maps from the same policy-filter-map-entries setting and remove that const. However, I just want you to aware that if we want to update both, it would be mixing a global capacity knob with a per-cgroup limit, which could waste memory if we config more policies globally.

@mtardy Please feel free to add you comment. Thanks

I’m fine to simplify and drive both policy_filter_maps and policy_filter_cgroup_maps from the same policy-filter-map-entries setting and remove that const. However, I just want you to aware that if we want to update both, it would be mixing a global capacity knob with a per-cgroup limit, which could waste memory if we config more policies globally.

My reading is that that policy_filter_cgroup_maps uses POLICY_FILTER_MAX_POLICIES since it's the most pessimistic approach. This means, that there will be no capacity issues (i.e., a failure to insert) in policy_filter_cgroup_maps without hitting a capacity issue in policy_filter_maps.

I think there are three options here:

Have the run-time size to determine the size of policy_filter_maps and policy_filter_cgroup_maps. That is, have the run-time size instead of POLICY_FILTER_MAX_POLICIES.

Have a different run-time size for both. In this case, we would need to check the error path of when the capacity of policy_filter_cgroup_maps is not enough and do something reasonable. This can be a follow-up PR.

Leave it as is. In this case, we would still need to check the error path since we might end up with a case where the capacity of policy_filter_cgroup_maps is not enough.

Note that option (2.) can be done in a followup PR after (1.).

CCing @tpapagian who I believe introduced this change.

My preference would be to leave it as-is for now, and keep policy_filter_cgroup_maps sized using POLICY_FILTER_MAX_POLICIES.

A few reasons for this:

Using POLICY_FILTER_MAX_POLICIES for policy_filter_cgroup_maps is intentionally pessimistic and gives us a safety guarantee.

While driving both maps from a single run-time knob is appealing, it mixes two different semantics: a global policy capacity, and a per-cgroup upper bound. This coupling can indeed lead to wasted memory when the global configuration is large, and I’m not fully convinced that trade-off is worth it yet.

Introducing separate run-time sizing (option 2) would require us to define and carefully handle a new failure mode (partial success where the global map accepts entries but the cgroup map does not). That feels like a larger behavioural change than this PR intends.

I agree that we should eventually harden the error path around policy_filter_cgroup_maps, but I think that can be done independently of changing sizing semantics, and safely as a follow-up.

pkg/sensors/program/loader_linux.go

pkg/policyfilter/map.go

pkg/option/flags.go

mtardy

Thanks, I think we can merge like this, I'd just do a small modification on the code part we had the loader before merging. :)

Sorry for the delay in the reviews

pkg/sensors/program/loader_linux.go

This commit introduces a new flag to configure the number of entries in policy filter maps. This allows users to tune the map size based on workload scale and system resources, improving flexibility in policy handling. Note: this commit only affects policies with k8s segmentation primitives (i.e., either podSelectors or namespaced policies). Fixes: cilium#4260 Signed-off-by: Kyle Dong <[email protected]>

Signed-off-by: Kyle Dong <[email protected]>

kyledong-suse · 2025-12-23T07:08:34Z

@mtardy Thank you so much for taking time to review and discuss this PR. I have edit the comment as you suggested. Please take another look when you have time. Thanks!

mtardy

Thanks! I'll just let @olsajiri ack this before merging. (this might take some time since people are on PTO for christmas, please ping back here beginning of Jan if no progress, thanks!!)

kkourt

Thanks! I have a small comment (please see below).

kkourt · 2026-01-07T14:48:01Z

pkg/option/flags.go


 	Config.RetprobesCacheSize = viper.GetInt(KeyRetprobesCacheSize)
+
+	Config.PolicyFilterMapEntries = viper.GetInt(KeyPolicyFilterMapEntries)


Given the discussion in 9dc9735#r2629371514, should we reject values that are larger than larger than POLICY_FILTER_MAX_POLICIES?

I don’t think we should reject values larger than POLICY_FILTER_MAX_POLICIES. The goal of this PR is to allow users to configure the number of entries in the policy filter maps. The original limit of 128 is sometimes too small, and users may legitimately have more policies in their maps.

OK, but given that we have:

// This map keeps exactly the same information as policy_filter_maps // but keeps the reverse mappings. i.e. // policy_filter_maps maps policy_id to cgroup_ids // policy_filter_cgroup_maps maps cgroup_id to policy_ids struct { __uint(type, BPF_MAP_TYPE_HASH_OF_MAPS); __uint(max_entries, POLICY_FILTER_MAX_CGROUP_IDS); __type(key, __u64); /* cgroup id */ __array( values, struct { __uint(type, BPF_MAP_TYPE_HASH); __uint(max_entries, POLICY_FILTER_MAX_POLICIES); __type(key, __u32); /* policy id */ __type(value, __u8); /* empty */ }); } policy_filter_cgroup_maps SEC(".maps");

Doesn't this mean that for a value larger than POLICY_FILTER_MAX_POLICIES, things will not work as expected when considering the policy_filter_cgoup_maps?

mtardy · 2026-01-16T11:16:18Z

please ping back here beginning of Jan if no progress

I sent a message to Jiri to progress on this

olsajiri · 2026-01-16T12:20:08Z

pkg/sensors/program/loader_linux.go

 		}
 	}

+	// TODO: remove this special case handling (see #4398)


this is bad.. would it help moving this all under program.Map ? there's draft PR for that already #4501 @sayboras

Yeah, it was discussed here for info #4331 (comment)

oh right, I was waiting for this to land in first, just to avoid any potential conflict here.

kyledong-suse requested a review from a team as a code owner November 11, 2025 16:55

kyledong-suse requested a review from FedeDP November 11, 2025 16:55

kyledong-suse force-pushed the pr/kyledong-suse/allow-configuring-policy-filter-map-size branch 2 times, most recently from 7475a97 to 444c976 Compare November 11, 2025 18:23

mtardy requested review from kkourt and mtardy November 20, 2025 18:13

mtardy requested changes Nov 20, 2025

View reviewed changes

kkourt added the release-note/minor This PR introduces a minor user-visible change label Nov 21, 2025

Andreagit97 mentioned this pull request Nov 25, 2025

Tetragon returns error when more than 128 policies are applied rancher-sandbox/runtime-enforcer#20

Closed

1 task

This was referenced Dec 3, 2025

tetragon/pkg: add user-configurable BPF_F_NO_PREALLOC flag support #4340

Open

Refactor policy_filter_maps to use program.Map #4398

Open

kyledong-suse force-pushed the pr/kyledong-suse/allow-configuring-policy-filter-map-size branch 2 times, most recently from 2f156b5 to 0453fe8 Compare December 16, 2025 06:13

kyledong-suse requested a review from mtardy December 18, 2025 23:25

Andreagit97 mentioned this pull request Dec 19, 2025

Scaling k8s workload aware tracing policies #4191

Open

mtardy reviewed Dec 22, 2025

View reviewed changes

pkg/sensors/program/loader_linux.go Outdated Show resolved Hide resolved

kyledong-suse force-pushed the pr/kyledong-suse/allow-configuring-policy-filter-map-size branch from 0453fe8 to cc2b381 Compare December 23, 2025 06:08

kyledong-suse added 2 commits December 23, 2025 01:24

fix: fix error message typo

53bbe52

Signed-off-by: Kyle Dong <[email protected]>

kyledong-suse force-pushed the pr/kyledong-suse/allow-configuring-policy-filter-map-size branch from cc2b381 to 53bbe52 Compare December 23, 2025 06:25

kyledong-suse requested a review from mtardy December 23, 2025 06:33

mtardy approved these changes Dec 23, 2025

View reviewed changes

mtardy requested a review from olsajiri January 5, 2026 09:41

kkourt reviewed Jan 7, 2026

View reviewed changes

olsajiri reviewed Jan 16, 2026

View reviewed changes


		Config.RetprobesCacheSize = viper.GetInt(KeyRetprobesCacheSize)

		Config.PolicyFilterMapEntries = viper.GetInt(KeyPolicyFilterMapEntries)

pkg/option: allow policy-filter-map-entries configurable via flag #4331

Are you sure you want to change the base?

pkg/option: allow policy-filter-map-entries configurable via flag #4331

Uh oh!

Conversation

kyledong-suse commented Nov 11, 2025

Description

Uh oh!

netlify bot commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for tetragon ready!

Uh oh!

kyledong-suse commented Nov 12, 2025

Uh oh!

mtardy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mtardy left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kyledong-suse commented Dec 23, 2025

Uh oh!

mtardy left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kkourt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mtardy commented Jan 16, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

netlify bot commented Nov 11, 2025 •

edited

Loading

mtardy left a comment •

edited

Loading

mtardy left a comment •

edited

Loading