Skip to content

Commit 118794f

Browse files
PillaiManishPillaiManish
committed
review changes
1 parent a6aa29d commit 118794f

File tree

1 file changed

+105
-81
lines changed

1 file changed

+105
-81
lines changed

‎enhancements/cert-manager/cert-manager-network-policies.md‎

Lines changed: 105 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ In a multi-tenant or security-conscious environment, it is crucial to enforce ne
4646
- Define baseline ingress and egress rules for the `istio-csr` component (deny-all with metrics access), with user-configurable network policies via the API for additional access requirements.
4747
- Ensure that metrics collection for all components remains functional.
4848
- Ensure the API server can communicate with the `cert-manager-webhook` for admission control.
49-
- Provide backward compatibility by making network policies opt-in via the `DefaultNetworkPolicy` field.
49+
- Provide backward compatibility by making network policies opt-in via the `defaultNetworkPolicy` field.
5050

5151
### Non-Goals
5252

@@ -56,12 +56,12 @@ In a multi-tenant or security-conscious environment, it is crucial to enforce ne
5656

5757
## Proposal
5858

59-
The proposal is to extend the `CertManager` custom resource with new API fields to enable and configure network policies. The `cert-manager-operator` will create and manage `NetworkPolicy` objects across all managed namespaces. For cert-manager, network policies are opt-in via the `DefaultNetworkPolicy` field (default "false") for backward compatibility with user-configurable additional rules. For istio-csr, network policies are automatically managed by the operator with no user configuration required. The strategy is to first apply a default-deny policy and then allow appropriate traffic based on component requirements.
59+
The proposal is to extend the `CertManager` custom resource with new API fields to enable and configure network policies. The `cert-manager-operator` will create and manage `NetworkPolicy` objects across all managed namespaces. For cert-manager, network policies are opt-in via the `defaultNetworkPolicy ` field (default "false") for backward compatibility with user-configurable additional rules. For istio-csr, network policies are automatically managed by the operator with no user configuration required. The strategy is to first apply a default-deny policy and then allow appropriate traffic based on component requirements.
6060

6161
### Workflow Description
6262

6363
1. **API-Driven Configuration:** Users configure network policies through the `CertManager` custom resource specification:
64-
- For cert-manager: Set `DefaultNetworkPolicy: "true"` and optionally provide custom `NetworkPolicies` rules
64+
- For cert-manager: Set `defaultNetworkPolicy: "true"` and optionally provide custom `networkPolicies[]` rules
6565
- For istio-csr: Network policies are automatically managed by the operator with no user configuration required
6666

6767
2. **Default Deny:** When network policies are enabled, the operator will create baseline `NetworkPolicy` objects that deny all traffic for the respective components. This ensures that no traffic is allowed unless explicitly permitted.
@@ -71,20 +71,22 @@ The proposal is to extend the `CertManager` custom resource with new API fields
7171
* **Allow Egress to API Server:** Permit outgoing traffic from the operator pod to the Kubernetes API server on port 6443/TCP.
7272
* **Allow Ingress for Metrics:** Permit incoming traffic to the operator pod on port 8443/TCP for Prometheus metrics scraping.
7373

74-
4. **Cert-Manager Operand Policies:** When `DefaultNetworkPolicy` is "true", the operator will create baseline policies for each component:
74+
4. **Cert-Manager Operand Policies:** When `defaultNetworkPolicy ` is "true", the operator will create baseline policies for each component:
7575

7676
* **Default policies include:**
77-
* **API Server Egress:** For all components to communicate with the Kubernetes API server
78-
* **Metrics Ingress:** For all components to expose metrics on port 9402/TCP
79-
* **Webhook Ingress:** For the webhook component to receive admission requests on port 10250/TCP
77+
* **Egress to API Server:** For all components to communicate with the Kubernetes API server
78+
* **Egress to DNS:** For cert-manager controller to communicate with the cluster DNS service
79+
* **Ingress to Metrics Endpoint:** For OpenShift Monitoring components to access metrics endpoint on port 9402/TCP
80+
* **Ingress to Webhook:** For the Kubernetes API server to establish HTTPS connections to the webhook component for admission requests on port 10250/TCP
8081

81-
* **User-configurable egress policies:** Users can specify additional egress rules via the `NetworkPolicies` field in the `CertManager` spec. If no egress policies are provided, cert-manager components will have deny-all egress policies (which will prevent proper operation without user configuration).
82+
* **User-configurable egress policies:** Users can specify additional egress rules via the `networkPolicies` field in the `CertManager` spec. If no egress policies are provided, cert-manager components will have deny-all egress policies (which will prevent proper operation without user configuration).
8283

8384
5. **Istio-CSR Policies:** The operator will create baseline policies for istio-csr:
8485

8586
* **Default policies include:**
86-
* **API Server Egress:** For communication with the Kubernetes API server on port 6443/TCP
87-
* **Metrics Ingress:** For exposing metrics on port 9402/TCP
87+
* **Egress to API Server:** For communication with the Kubernetes API server on port 6443/TCP
88+
* **Ingress to gRPC Endpoint:** For incoming gRPC connections to the Istio-CSR on port 6443/TCP
89+
* **Ingress to Metrics Endpoint:** For OpenShift Monitoring components to access metrics endpoint on port 9402/TCP
8890

8991
* **Automatic policy management:** All istio-csr network policies (both ingress and egress) are automatically managed by the operator. No user configuration is required or supported for istio-csr network policies.
9092

@@ -105,7 +107,9 @@ The implementation will involve extending the existing APIs and creating `Networ
105107
name: deny-all-traffic
106108
namespace: cert-manager-operator
107109
spec:
108-
podSelector: {}
110+
podSelector:
111+
matchLabels:
112+
name: cert-manager-operator
109113
policyTypes:
110114
- Ingress
111115
- Egress
@@ -135,16 +139,16 @@ The implementation will involve extending the existing APIs and creating `Networ
135139
from:
136140
- namespaceSelector:
137141
matchLabels:
138-
name: openshift-monitoring
142+
name: openshift-user-workload-monitoring
139143
- protocol: TCP
140144
port: 8443
141145
```
142146
143147
#### Cert-Manager Operand Namespace Policies
144148
145-
When `DefaultNetworkPolicy` is set to "true", the operator will create baseline policies for cert-manager components. Users can provide additional or custom policies via the `NetworkPolicies` field in the `CertManager` spec.
149+
When `defaultNetworkPolicy` is set to "true", the operator will create baseline policies for cert-manager components. Users can provide additional or custom policies via the `networkPolicies` field in the `CertManager` spec.
146150

147-
1. **Baseline Deny-All Policy:** Applied when `DefaultNetworkPolicy` is "true".
151+
1. **Baseline Deny-All Policy:** Applied when `defaultNetworkPolicy` is "true".
148152

149153
```yaml
150154
apiVersion: networking.k8s.io/v1
@@ -153,13 +157,15 @@ When `DefaultNetworkPolicy` is set to "true", the operator will create baseline
153157
name: deny-all-traffic
154158
namespace: cert-manager
155159
spec:
156-
podSelector: {}
160+
podSelector:
161+
matchLabels:
162+
app.kubernetes.io/instance: cert-manager
157163
policyTypes:
158164
- Ingress
159165
- Egress
160166
```
161167

162-
2. **Default Allow Policies:** The operator creates these baseline policies when `DefaultNetworkPolicy` is "true":
168+
2. **Default Allow Policies:** The operator creates these baseline policies when `defaultNetworkPolicy` is "true":
163169

164170
```yaml
165171
# API Server egress for all components
@@ -171,52 +177,79 @@ When `DefaultNetworkPolicy` is set to "true", the operator will create baseline
171177
spec:
172178
podSelector:
173179
matchLabels:
174-
app.kubernetes.io/name: cert-manager
180+
app.kubernetes.io/instance: cert-manager
175181
policyTypes:
176182
- Egress
177183
egress:
178184
- ports:
179185
- protocol: TCP
180186
port: 6443
181187
---
182-
# Metrics ingress for all components
188+
# DNS egress for cert-manager controller component
183189
apiVersion: networking.k8s.io/v1
184190
kind: NetworkPolicy
185191
metadata:
186-
name: allow-metrics-ingress
187-
namespace: cert-manager
188-
spec:
189-
podSelector:
190-
matchLabels:
191-
app.kubernetes.io/name: cert-manager
192-
policyTypes:
193-
- Ingress
194-
ingress:
195-
- ports:
196-
from:
197-
- namespaceSelector:
198-
matchLabels:
199-
name: openshift-monitoring
200-
- protocol: TCP
201-
port: 9402
202-
---
203-
# Webhook ingress for admission control
204-
apiVersion: networking.k8s.io/v1
205-
kind: NetworkPolicy
206-
metadata:
207-
name: allow-webhook-ingress
192+
name: allow-dns-egress
208193
namespace: cert-manager
209194
spec:
195+
egress:
196+
- ports:
197+
- port: 5353
198+
protocol: TCP
199+
- port: 5353
200+
protocol: UDP
201+
to:
202+
- namespaceSelector:
203+
matchLabels:
204+
kubernetes.io/metadata.name: openshift-dns
205+
podSelector:
206+
matchLabels:
207+
dns.operator.openshift.io/daemonset-dns: default
210208
podSelector:
211209
matchLabels:
212-
app: webhook
210+
app: cert-manager
213211
policyTypes:
214-
- Ingress
215-
ingress:
216-
- ports:
217-
- protocol: TCP
218-
port: 10250
219-
```
212+
- Egress
213+
214+
---
215+
# Metrics ingress for all components
216+
apiVersion: networking.k8s.io/v1
217+
kind: NetworkPolicy
218+
metadata:
219+
name: allow-metrics-ingress
220+
namespace: cert-manager
221+
spec:
222+
podSelector:
223+
matchLabels:
224+
app.kubernetes.io/instance: cert-manager
225+
policyTypes:
226+
- Ingress
227+
ingress:
228+
- ports:
229+
from:
230+
- namespaceSelector:
231+
matchLabels:
232+
name: openshift-user-workload-monitoring
233+
- protocol: TCP
234+
port: 9402
235+
---
236+
# Webhook ingress for admission control
237+
apiVersion: networking.k8s.io/v1
238+
kind: NetworkPolicy
239+
metadata:
240+
name: allow-webhook-ingress
241+
namespace: cert-manager
242+
spec:
243+
podSelector:
244+
matchLabels:
245+
app: webhook
246+
policyTypes:
247+
- Ingress
248+
ingress:
249+
- ports:
250+
- protocol: TCP
251+
port: 10250
252+
```
220253

221254
3. **User-Configurable Policies:** Users must configure additional policies via the API for cert-manager controller egress (to communicate with external issuers). Example user configuration:
222255

@@ -228,10 +261,22 @@ When `DefaultNetworkPolicy` is set to "true", the operator will create baseline
228261
spec:
229262
defaultNetworkPolicy: "true"
230263
networkPolicies:
231-
- name: allow-cert-manager-controller-egress
232-
componentName: CoreController
233-
egress:
234-
- {} # Allow all egress for external issuers communication
264+
- componentName: CoreController
265+
egress:
266+
- ports:
267+
- port: 80
268+
protocol: TCP
269+
- port: 443
270+
protocol: TCP
271+
name: allow-egress-to-acme-server
272+
- componentName: CoreController
273+
egress:
274+
- ports:
275+
- port: 53
276+
protocol: UDP
277+
- port: 53
278+
protocol: TCP
279+
name: allow-egress-to-dns-service
235280
```
236281

237282
#### Istio-CSR Namespace Policies
@@ -298,30 +343,13 @@ The `istio-csr` component requires specific network policies to function correct
298343
from:
299344
- namespaceSelector:
300345
matchLabels:
301-
name: openshift-monitoring
346+
name: openshift-user-workload-monitoring
302347
- protocol: TCP
303348
port: 9402 # Metrics port
304349
```
305350

306351
3. **Automatic Policy Management:** All istio-csr network policies are automatically managed by the operator based on the component's requirements. No user configuration is needed.
307352

308-
```yaml
309-
apiVersion: operator.openshift.io/v1alpha1
310-
kind: IstioCSR
311-
metadata:
312-
name: cluster
313-
spec:
314-
networkPolicies:
315-
- name: allow-istio-csr-grpc-service
316-
componentName: IstioCSR
317-
policyTypes:
318-
- Ingress
319-
ingress:
320-
- ports:
321-
- protocol: TCP
322-
port: 6443 # Replace with actual configured gRPC service port
323-
```
324-
325353
### API Extensions
326354

327355
This enhancement introduces new fields to the existing `CertManager` custom resource to support network policy configuration.
@@ -391,9 +419,6 @@ const (
391419

392420
// Webhook represents the cert-manager webhook component
393421
Webhook ComponentName = "Webhook"
394-
395-
// IstioCSR represents the cert-manager Istio CSR component
396-
IstioCSR ComponentName = "IstioCSR"
397422
)
398423

399424

@@ -409,7 +434,7 @@ type NetworkPolicy struct {
409434
Name string `json:"name"`
410435

411436
// ComponentName represents the different cert-manager components that can have network policies applied.
412-
// +kubebuilder:validation:Enum:=CAInjector;CoreController;WebHook;IstioCSR
437+
// +kubebuilder:validation:Enum:=CAInjector;CoreController;WebHook
413438
// +kubebuilder:validation:Required
414439
ComponentName ComponentName `json:"componentName"`
415440

@@ -461,15 +486,15 @@ The main drawback is the added complexity of managing multiple `NetworkPolicy` o
461486
## Test Plan
462487

463488
* **Integration Tests:**
464-
1. Test with `DefaultNetworkPolicy: "false"` (default): Verify no NetworkPolicy objects are created and cert-manager functions normally.
465-
2. Test with `DefaultNetworkPolicy: "true"` but no custom `NetworkPolicies` specified: Verify baseline policies are created and cert-manager controller cannot communicate with external issuers (expected behavior).
466-
3. Test with `DefaultNetworkPolicy: "true"` and custom `NetworkPolicies` for cert-manager controller egress: Verify cert-manager can communicate with external issuers.
467-
4. Test istio-csr with no `NetworkPolicies` specified: Verify baseline policies are created and external egress is blocked while ingress (gRPC service) is automatically allowed (expected behavior).
489+
1. Test with `defaultNetworkPolicy: "false"` (default): Verify no NetworkPolicy objects are created and cert-manager functions normally.
490+
2. Test with `defaultNetworkPolicy: "true"` but no custom `networkPolicies` specified: Verify baseline policies are created and cert-manager controller cannot communicate with external issuers (expected behavior).
491+
3. Test with `defaultNetworkPolicy: "true"` and custom `networkPolicies` for cert-manager controller egress: Verify cert-manager can communicate with external issuers.
492+
4. Test istio-csr with no `networkPolicies` specified: Verify baseline policies are created and external egress is blocked while ingress (gRPC service) is automatically allowed (expected behavior).
468493
5. Test istio-csr with automatic network policies: Verify istio-csr can communicate with required services with operator-managed policies.
469494
6. Create a `curl` pod and confirm it **can** access the metrics endpoints (`:8443` for operator, `:9402` for operands) when policies are enabled.
470495
7. Confirm the `curl` pod **cannot** access pods on non-allowed ports when policies are enabled.
471496
* **End-to-End (E2E) Tests:**
472-
1. Run the existing `cert-manager` E2E test suite with `DefaultNetworkPolicy: "true"`. Configure proper NetworkPolicy settings via the API and run the cert-manager E2E test suite with network policies enabled.
497+
1. Run the existing `cert-manager` E2E test suite with `defaultNetworkPolicy: "true"`. Configure proper NetworkPolicy settings via the API and run the cert-manager E2E test suite with network policies enabled.
473498
2. Run the istio-csr E2E test suite with automatic operator-managed network policies enabled.
474499

475500
## Graduation Criteria
@@ -495,7 +520,7 @@ Not applicable.
495520

496521
## Upgrade / Downgrade Strategy
497522

498-
* **Upgrade:** On upgrade, the new API fields will be available but network policies remain disabled by default (`DefaultNetworkPolicy: "false"`). This ensures backward compatibility. Users must explicitly enable network policies and configure them appropriately.
523+
* **Upgrade:** On upgrade, the new API fields will be available but network policies remain disabled by default (`defaultNetworkPolicy: "false"`). This ensures backward compatibility. Users must explicitly enable network policies and configure them appropriately.
499524
* **Downgrade:** If a user downgrades to a version of the operator that is not aware of the new API fields:
500525
- The API fields will be ignored by the older operator version
501526
- Any existing `NetworkPolicy` objects created by the newer operator will be orphaned
@@ -518,8 +543,7 @@ Not applicable, as this enhancement does not introduce any API extensions.
518543
Support personnel debugging potential network policy issues should follow these steps:
519544

520545
1. **Check API Configuration:**
521-
- Verify `CertManager` resource: `oc get certmanager cluster -o yaml` and check `defaultNetworkPolicy` and `NetworkPolicies` fields
522-
- Verify `IstioCSR` resource: `oc get istiocsr cluster -o yaml` and check `NetworkPolicies` field
546+
- Verify `CertManager` resource: `oc get certmanager cluster -o yaml` and check `defaultNetworkPolicy` and `networkPolicies` fields
523547

524548
2. **Verify NetworkPolicy Objects:**
525549
- Check if NetworkPolicy objects exist: `oc get networkpolicy -n <namespace>`

0 commit comments

Comments
 (0)