Skip to content

OCPBUGS-73938: cabundleinjector: Add CA bundles watcher#329

Open
tchap wants to merge 1 commit intoopenshift:mainfrom
tchap:watch-ca-bundles
Open

OCPBUGS-73938: cabundleinjector: Add CA bundles watcher#329
tchap wants to merge 1 commit intoopenshift:mainfrom
tchap:watch-ca-bundles

Conversation

@tchap
Copy link
Copy Markdown
Contributor

@tchap tchap commented Mar 13, 2026

Add a watcher to watch all relevant CA bundle files for changes
so that they are automatically reloaded when modified.

There is also a periodic resync implemented and set to 10 minutes.

The current implementation does not update all affected ConfigMaps. This
means that only the ConfigMaps created after the bundles are reloaded
are affected and up-to-date with respect to the files present on the
disk.

Summary by CodeRabbit

Release Notes

  • New Features

    • Implemented dynamic CA bundle reloading from the file system with periodic polling capability
    • CA bundle updates now automatically propagate across webhook configurations, API services, ConfigMaps, and Custom Resource Definitions
  • Bug Fixes

    • Added error handling for unpopulated CA bundles to prevent invalid injections
    • Enhanced thread-safe access to CA bundle data for concurrent operations
  • Tests

    • Added test coverage for CA bundle observer behavior including file monitoring, updates, and data persistence

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 13, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@tchap: This pull request references Jira Issue OCPBUGS-73938, which is invalid:

  • expected the bug to target the "4.22.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

The CA bundle files are now being watched and reloaded on update. The last value is retained when the files are deleted.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Mar 13, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 13, 2026

Important

Review skipped

Auto reviews are limited based on label configuration.

🚫 Excluded labels (none allowed) (1)
  • do-not-merge/work-in-progress

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5f0c3c96-6569-4888-b157-ad4edd18653b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Walkthrough

This PR refactors CA bundle management across multiple injector types (webhook, API service, config map, CRD) from direct byte/string storage to atomic pointers for thread-safe concurrent access. It introduces a file observer mechanism in the starter that polls CA bundle files and updates the atomic pointers dynamically.

Changes

Cohort / File(s) Summary
Webhook CA Bundle Refactoring
pkg/controller/cabundleinjector/admissionwebhook.go, admissionwebhook_test.go
Changed caBundle field from []byte to *atomic.Pointer[[]byte] in webhook injector. Updated sync logic to load pointer, validate non-nil, and dereference before comparisons/updates. Test modified to initialize atomic pointer with ptr.To wrapper.
Injector CA Bundle Refactoring
pkg/controller/cabundleinjector/apiservice.go, configmap.go, crd.go
Converted caBundle storage from plain types to *atomic.Pointer[[]byte] across all injectors. Each sync method now loads, validates, and dereferences the pointer before use. Config map additionally converts dereferenced bytes to string for compatibility.
CA Observer and Polling Implementation
pkg/controller/cabundleinjector/starter.go
Added file-based CA bundle observer with polling mechanism. Introduced startCAObserver function that monitors ca-bundle.crt and sa.crt files, performs initial load via readBundles, and dynamically updates atomic pointers on file changes. Maintains last-known values after deletions.
Observer Test Coverage
pkg/controller/cabundleinjector/starter_test.go
New test suite validating CA observer behavior including initial file loading, dynamic reloading on updates, and retention of last-known values after file deletions using eventual consistency checks.

Sequence Diagram

sequenceDiagram
    participant FileSystem as File System<br/>(ca-bundle.crt, sa.crt)
    participant Observer as CA Observer<br/>(startCAObserver)
    participant AtomicPtr as Atomic Pointer<br/>Store
    participant Injectors as CA Bundle<br/>Injectors
    participant Sync as Sync Flow

    Observer->>FileSystem: Initial read via readBundles
    FileSystem-->>Observer: ca-bundle, sa-token-ca-bundle
    Observer->>AtomicPtr: Store bundles in atomic.Pointer
    Observer->>Observer: Start file polling loop
    
    Injectors->>AtomicPtr: Initialize with pointer refs
    
    FileSystem->>Observer: File change detected (poll)
    Observer->>FileSystem: Re-read bundles
    FileSystem-->>Observer: Updated content
    Observer->>AtomicPtr: Update atomic pointers
    
    Sync->>AtomicPtr: Load() pointer value
    AtomicPtr-->>Sync: Dereference CA bundle
    Sync->>Injectors: Apply CA bundle to resources
    
    FileSystem->>Observer: File deletion
    Observer->>AtomicPtr: Retain last-known value
    Sync->>AtomicPtr: Load() still returns previous value
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (2 warnings, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 10.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Test Structure And Quality ⚠️ Warning Test has critical coverage gap: only tests simultaneous file deletion but misses the saTokenCAFile deletion followed by caBundleFile modification scenario that triggers the retention bug. Add test coverage for deleting saTokenCAFile then modifying caBundleFile; replace hardcoded time.Sleep() with require.Eventually(); organize into subtests using t.Run().
Stable And Deterministic Test Names ❓ Inconclusive Shell commands cannot be executed in this environment; unable to analyze test file structure and naming patterns. Provide the test file content directly or use a compatible tool to extract and analyze test function names.
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The pull request title clearly summarizes the main change: introducing a CA bundles watcher mechanism for the cabundleinjector component.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests were added in this PR. The new starter_test.go uses standard Go testing framework only.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR adds standard Go unit tests using testing package and testify/require, not Ginkgo e2e tests. Tests are isolated with no cluster topology dependencies.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The new test added is a standard Go unit test, not a Ginkgo e2e test, so the IPv6 and disconnected network compatibility check is not applicable.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@tchap
Copy link
Copy Markdown
Contributor Author

tchap commented Mar 13, 2026

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Mar 13, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@tchap: This pull request references Jira Issue OCPBUGS-73938, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @xingxingxia

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Mar 13, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: tchap
Once this PR has been reviewed and has the lgtm label, please assign p0lyn0mial for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@tchap: This pull request references Jira Issue OCPBUGS-73938, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @xingxingxia

Details

In response to this:

The CA bundle files are now being watched and reloaded on update.
The last value is retained when the files are deleted.

Summary by CodeRabbit

Release Notes

  • New Features

  • Implemented dynamic CA bundle reloading from the file system with periodic polling capability

  • CA bundle updates now automatically propagate across webhook configurations, API services, ConfigMaps, and Custom Resource Definitions

  • Bug Fixes

  • Added error handling for unpopulated CA bundles to prevent invalid injections

  • Enhanced thread-safe access to CA bundle data for concurrent operations

  • Tests

  • Added test coverage for CA bundle observer behavior including file monitoring, updates, and data persistence

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/controller/cabundleinjector/starter.go`:
- Around line 132-143: The observer loses a retained saToken CA bundle when
saTokenCAFile is deleted but caBundleFile is later updated because
reloadCABundles() calls readBundles(caBundleFile, saTokenCAFile) which returns
nil for saTokenCABundleContent on missing file and that nil is written back to
saTokenCABundleData; fix by changing reloadCABundles()/readBundles call to pass
the current cached saTokenCABundleData as the fallback (e.g., pass
saTokenCABundleData into readBundles or add a parameter/current value) and
update readBundles to return the existing saToken bundle when the file is absent
so saTokenCABundleData is not overwritten; apply same pattern for the legacy
vulnerable CA path (vulnerableLegacyCABundleData) wherever readBundles is
called.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: fb29a04f-5862-4090-b895-beee77fc1cb5

📥 Commits

Reviewing files that changed from the base of the PR and between 4003a55 and 8d309e4.

📒 Files selected for processing (7)
  • pkg/controller/cabundleinjector/admissionwebhook.go
  • pkg/controller/cabundleinjector/admissionwebhook_test.go
  • pkg/controller/cabundleinjector/apiservice.go
  • pkg/controller/cabundleinjector/configmap.go
  • pkg/controller/cabundleinjector/crd.go
  • pkg/controller/cabundleinjector/starter.go
  • pkg/controller/cabundleinjector/starter_test.go

@tchap
Copy link
Copy Markdown
Contributor Author

tchap commented Mar 16, 2026

/retitle WIP: OCPBUGS-73938: cabundleinjector: Add CA bundles watcher

@openshift-ci openshift-ci bot changed the title OCPBUGS-73938: cabundleinjector: Add CA bundles watcher WIP: OCPBUGS-73938: cabundleinjector: Add CA bundles watcher Mar 16, 2026
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 16, 2026
@tchap tchap marked this pull request as draft March 16, 2026 17:11
@tchap tchap force-pushed the watch-ca-bundles branch 9 times, most recently from e4b41d8 to 83343d9 Compare March 20, 2026 13:42
@tchap tchap marked this pull request as ready for review March 20, 2026 13:43
@tchap tchap force-pushed the watch-ca-bundles branch 2 times, most recently from 778254d to 291cf41 Compare March 23, 2026 12:59
Add a watcher to watch all relevant CA bundle files for changes
so that they are automatically reloaded when modified.

There is also a periodic resync implemented and set to 10 minutes.

The current implementation does not update all affected ConfigMaps. This
means that only the ConfigMaps created after the bundles are reloaded
are affected and up-to-date with respect to the files present on the
disk.
@tchap tchap force-pushed the watch-ca-bundles branch from 291cf41 to c1e63f9 Compare March 23, 2026 13:00
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@tchap: This pull request references Jira Issue OCPBUGS-73938, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (ksiddiqu@redhat.com), skipping review request.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Add a watcher to watch all relevant CA bundle files for changes
so that they are automatically reloaded when modified.

There is also a periodic resync implemented and set to 10 minutes.

The current implementation does not update all affected ConfigMaps. This
means that only the ConfigMaps created after the bundles are reloaded
are affected and up-to-date with respect to the files present on the
disk.

Summary by CodeRabbit

Release Notes

  • New Features

  • Implemented dynamic CA bundle reloading from the file system with periodic polling capability

  • CA bundle updates now automatically propagate across webhook configurations, API services, ConfigMaps, and Custom Resource Definitions

  • Bug Fixes

  • Added error handling for unpopulated CA bundles to prevent invalid injections

  • Enhanced thread-safe access to CA bundle data for concurrent operations

  • Tests

  • Added test coverage for CA bundle observer behavior including file monitoring, updates, and data persistence

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@tchap
Copy link
Copy Markdown
Contributor Author

tchap commented Mar 23, 2026

/retitle OCPBUGS-73938: cabundleinjector: Add CA bundles watcher

@openshift-ci openshift-ci bot changed the title WIP: OCPBUGS-73938: cabundleinjector: Add CA bundles watcher OCPBUGS-73938: cabundleinjector: Add CA bundles watcher Mar 23, 2026
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 23, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Mar 23, 2026

@tchap: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants