chore: adding metrics for VAP integration #4317

JaydipGabani · 2025-12-20T01:35:34Z

What this PR does / why we need it:
Adds metrics for VAP integration
Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #4078

Special notes for your reviewer:

Signed-off-by: Jaydip Gabani <[email protected]>

JaydipGabani · 2025-12-20T01:38:56Z

pkg/controller/constrainttemplate/stats_reporter.go

-	ingestDuration = "constraint_template_ingestion_duration_seconds"
-	statusKey      = "status"
+	ctMetricName    = "constraint_templates"
+	celCTMetricName = "constraint_templates_with_cel"


Instead of a tag, I am adding new metrics tracking total CT with CEL, and this can be used with constraint_templates metrics to figure out rego only CT count.

The reason of adding a new metrics rather than a tag as discussed was to maintain backward compatibility. IMO this is better but if we want to do type=CEL,REGO I can revert to that.

Good call on using a separate constraint_templates_with_cel metric instead of adding a type=cel tag to constraint_templates for backward compatibility. Consider adding a brief code comment here explaining this design decision.

Copilot

Pull request overview

This PR adds comprehensive metrics support for ValidatingAdmissionPolicy (VAP) integration in Gatekeeper, enabling observability into the generation and status of VAP resources alongside constraint templates.

Introduces three new metrics: gatekeeper_constraint_templates_with_cel, gatekeeper_validating_admission_policies, and gatekeeper_validating_admission_policy_bindings
Implements thread-safe registry tracking for VAP and VAPB resources with active/error status reporting
Adds integration tests in BATS to verify metric reporting and removes redundant pod creation calls

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
website/docs/metrics.md	Documents the three new VAP-related metrics with descriptions, tags, and aggregation types
test/testutils/controller.go	Simplifies variable declaration by removing unnecessary parentheses
test/bats/test.bats	Adds metrics verification tests for VAP integration and optimizes pod creation to check for existing pods before creating new ones
pkg/controller/constrainttemplate/stats_reporter_test.go	Adds comprehensive test coverage for VAP, VAPB, and CEL template metrics including creation, deletion, and status updates
pkg/controller/constrainttemplate/stats_reporter.go	Implements metrics infrastructure for VAP and CEL constraint templates with thread-safe registries and observable gauges
pkg/controller/constrainttemplate/constrainttemplate_controller.go	Integrates metric reporting throughout the VAP lifecycle including creation, updates, errors, and deletion
pkg/controller/constraint/stats_reporter_test.go	Adds test coverage for VAPB metrics with status tracking and deletion scenarios
pkg/controller/constraint/stats_reporter.go	Implements VAPB metrics with registry-based tracking and status observation
pkg/controller/constraint/constraint_controller.go	Integrates VAPB metric reporting across all generation states including errors and successful binding creation

pkg/controller/constraint/stats_reporter.go

pkg/controller/constrainttemplate/constrainttemplate_controller.go

pkg/controller/constraint/constraint_controller.go

codecov-commenter · 2025-12-20T01:48:21Z

Codecov Report

❌ Patch coverage is 71.53846% with 37 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.33%. Comparing base (3350319) to head (27b710f).
⚠️ Report is 553 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/controller/constraint/constraint_controller.go	0.00%	22 Missing ⚠️
...onstrainttemplate/constrainttemplate_controller.go	52.63%	9 Missing ⚠️
...kg/controller/constrainttemplate/stats_reporter.go	91.11%	2 Missing and 2 partials ⚠️
pkg/controller/constraint/stats_reporter.go	88.23%	1 Missing and 1 partial ⚠️

❗ There is a different number of reports uploaded between BASE (3350319) and HEAD (27b710f). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (3350319) HEAD (27b710f)

unittests 2 1

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #4317       +/-   ##
===========================================
- Coverage   54.49%   41.33%   -13.16%     
===========================================
  Files         134      253      +119     
  Lines       12329    17923     +5594     
===========================================
+ Hits         6719     7409      +690     
- Misses       5116     9871     +4755     
- Partials      494      643      +149

Flag	Coverage Δ
unittests	`41.33% <71.53%> (-13.16%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

pkg/controller/constrainttemplate/stats_reporter.go

pkg/controller/constraint/stats_reporter.go

pkg/controller/constraint/constraint_controller.go

pkg/controller/constrainttemplate/constrainttemplate_controller.go

pkg/controller/constraint/constraint_controller.go

sozercan · 2025-12-22T18:10:59Z

pkg/controller/constraint/constraint_controller.go

can we improve unit test coverage here?

generally constraint tests are covered through CT tests so there is no reconcilation tests for constraint. For this change, I have added unit tests for stats_reporter and have added tests in bats file which should cover the code. Let me know if metrics tests through constraint lifecycle is still desired and I can add them.

sozercan · 2025-12-22T18:11:51Z

pkg/controller/constrainttemplate/stats_reporter.go

-	ingestDuration = "constraint_template_ingestion_duration_seconds"
-	statusKey      = "status"
+	ctMetricName    = "constraint_templates"
+	celCTMetricName = "constraint_templates_with_cel"


Good call on using a separate constraint_templates_with_cel metric instead of adding a type=cel tag to constraint_templates for backward compatibility. Consider adding a brief code comment here explaining this design decision.

sozercan

thanks! added a few comments

Signed-off-by: Jaydip Gabani <[email protected]>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

pkg/controller/constrainttemplate/constrainttemplate_controller.go

Signed-off-by: Jaydip Gabani <[email protected]>

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

pkg/controller/constraint/constraint_controller.go

Signed-off-by: Jaydip Gabani <[email protected]>

pkg/metrics/vapstatus_registry.go

pkg/controller/constrainttemplate/constrainttemplate_controller.go

test/bats/test.bats

pkg/controller/constrainttemplate/stats_reporter.go

pkg/controller/constraint/constraint_controller.go

sozercan · 2026-01-07T22:03:31Z

pkg/controller/constraint/constraint_controller.go

+						r.reporter.ReportVAPBStatus(vapBindingKey, metrics.VAPStatusError)
 						return noDelay, r.reportErrorOnConstraintStatus(ctx, status, err, "could not parse timestamp")
 					}
 					if t.After(time.Now()) {


do we want to emit a metric while waiting?

This may not be transient error if parsing timestamp is resulting in an error. That is why I think it may be a good idea to emit metrics. Eventually if VAPB is generated then this metric will be replaced with active status anyways.

Signed-off-by: Jaydip Gabani <[email protected]>

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

test/bats/test.bats

Signed-off-by: Jaydip Gabani <[email protected]>

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

adding metrics for VAP integration

7c16856

Signed-off-by: Jaydip Gabani <[email protected]>

JaydipGabani requested a review from a team as a code owner December 20, 2025 01:35

Copilot AI review requested due to automatic review settings December 20, 2025 01:35

Copilot started reviewing on behalf of JaydipGabani December 20, 2025 01:35 View session

JaydipGabani commented Dec 20, 2025

View reviewed changes

Copilot AI reviewed Dec 20, 2025

View reviewed changes

sozercan reviewed Dec 22, 2025

View reviewed changes

sozercan requested changes Dec 22, 2025

View reviewed changes

JaydipGabani added 3 commits January 2, 2026 20:01

refactor: sharing vapstatus consts among c and ct stats reporting

0834fed

Signed-off-by: Jaydip Gabani <[email protected]>

refactor: common vap stats registry for status

8059bd1

Signed-off-by: Jaydip Gabani <[email protected]>

fixing tests

04afd8e

Signed-off-by: Jaydip Gabani <[email protected]>

Copilot AI review requested due to automatic review settings January 5, 2026 21:11

Copilot started reviewing on behalf of JaydipGabani January 5, 2026 21:12 View session

Copilot AI reviewed Jan 5, 2026

View reviewed changes

pkg/controller/constrainttemplate/constrainttemplate_controller.go Outdated Show resolved Hide resolved

JaydipGabani added 2 commits January 5, 2026 22:38

improving CEL engine detemination logic

3e9f1af

Signed-off-by: Jaydip Gabani <[email protected]>

Merge branch 'master' into vap-metrics

63545e1

Copilot AI review requested due to automatic review settings January 6, 2026 00:04

Copilot started reviewing on behalf of JaydipGabani January 6, 2026 00:05 View session

JaydipGabani requested a review from sozercan January 6, 2026 00:05

Copilot AI reviewed Jan 6, 2026

View reviewed changes

pkg/controller/constraint/constraint_controller.go Show resolved Hide resolved

fixing bugs in reporting vapb status

b2fdabe

Signed-off-by: Jaydip Gabani <[email protected]>

sozercan reviewed Jan 7, 2026

View reviewed changes

pkg/metrics/vapstatus_registry.go Show resolved Hide resolved

sozercan reviewed Jan 7, 2026

View reviewed changes

pkg/controller/constrainttemplate/constrainttemplate_controller.go Show resolved Hide resolved

sozercan reviewed Jan 7, 2026

View reviewed changes

test/bats/test.bats Show resolved Hide resolved

sozercan reviewed Jan 7, 2026

View reviewed changes

pkg/controller/constrainttemplate/stats_reporter.go Show resolved Hide resolved

sozercan reviewed Jan 7, 2026

View reviewed changes

pkg/controller/constraint/constraint_controller.go Outdated Show resolved Hide resolved

sozercan reviewed Jan 7, 2026

View reviewed changes

pkg/controller/constraint/constraint_controller.go Outdated Show resolved Hide resolved

sozercan reviewed Jan 7, 2026

View reviewed changes

addressing feedback

34eb542

Signed-off-by: Jaydip Gabani <[email protected]>

Copilot AI review requested due to automatic review settings January 8, 2026 01:26

Copilot started reviewing on behalf of JaydipGabani January 8, 2026 01:26 View session

fixing lint

8df2deb

Signed-off-by: Jaydip Gabani <[email protected]>

Copilot AI reviewed Jan 8, 2026

View reviewed changes

test/bats/test.bats Outdated Show resolved Hide resolved

JaydipGabani added 2 commits January 8, 2026 01:40

fixing tests

cd44c7b

Signed-off-by: Jaydip Gabani <[email protected]>

fixing tests

42d8724

Signed-off-by: Jaydip Gabani <[email protected]>

Copilot AI review requested due to automatic review settings January 8, 2026 19:43

Copilot started reviewing on behalf of JaydipGabani January 8, 2026 19:43 View session

Copilot AI reviewed Jan 8, 2026

View reviewed changes

JaydipGabani requested a review from sozercan January 8, 2026 19:58

Merge branch 'master' into vap-metrics

27b710f

chore: adding metrics for VAP integration #4317

Are you sure you want to change the base?

chore: adding metrics for VAP integration #4317

Uh oh!

Conversation

JaydipGabani commented Dec 20, 2025

Uh oh!

JaydipGabani Dec 20, 2025

Choose a reason for hiding this comment

Uh oh!

sozercan Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Dec 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sozercan Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

JaydipGabani Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

sozercan Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

sozercan left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sozercan Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

JaydipGabani Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

codecov-commenter commented Dec 20, 2025 •

edited

Loading