Skip to content

add nginx content cache as cache policy #8005

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 41 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
f4a1ae5
initial cache-policy commit
vepatel Jul 2, 2025
c0f6d66
add example files
vepatel Jul 2, 2025
7acfa85
update CRD validation and add sessionAffinity
vepatel Jul 2, 2025
a82a790
add values.yaml
vepatel Jul 2, 2025
d90c2d2
add overrideUpstreamCache option
vepatel Jul 3, 2025
d7604cf
add unique purge var per VS, VSR
vepatel Jul 4, 2025
1563406
remove redundant enable flag
vepatel Jul 4, 2025
028f7b1
add levels and shared cache config
vepatel Jul 9, 2025
56ca632
Merge branch 'main' into feat/cache-policy
vepatel Jul 9, 2025
4c96532
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 9, 2025
e63b045
Merge branch 'main' into feat/cache-policy
vepatel Aug 6, 2025
3d8f457
Merge branch 'main' into feat/cache-policy
vepatel Aug 7, 2025
0857c5b
add unit tests
vepatel Aug 7, 2025
1b635a2
fix kubebuilder issues anbd add more validation to policy.go
vepatel Aug 7, 2025
948c816
use CEL rules
vepatel Aug 7, 2025
4aa91e1
reduce cyclomatic complexity and update codegen
vepatel Aug 7, 2025
020b0b9
Merge branch 'main' into feat/cache-policy
vepatel Aug 7, 2025
c1ccf4c
add more CEL based validation in CRDs
vepatel Aug 8, 2025
f47a8b2
add maxItems for allowedMethods
vepatel Aug 8, 2025
ce72be2
Merge branch 'main' into feat/cache-policy
vepatel Aug 8, 2025
1eb7f1e
Merge branch 'main' into feat/cache-policy
vepatel Aug 11, 2025
3b643ba
move more validations from go to CRDs
vepatel Aug 11, 2025
29efd58
Merge branch 'main' into feat/cache-policy
vepatel Aug 11, 2025
9301c63
add better descriptions and remove costly CEL expressions
vepatel Aug 11, 2025
9974453
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 11, 2025
9d0c651
replace CEL with go logic
vepatel Aug 11, 2025
877e699
add functional tests and fix plus vs template
vepatel Aug 12, 2025
886369f
Merge branch 'main' into feat/cache-policy
vepatel Aug 12, 2025
23602f6
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 12, 2025
54fa49d
update pytest marker
vepatel Aug 12, 2025
733485b
update pytest marker in ci and fix oss template
vepatel Aug 13, 2025
62182ac
add template unit tests
vepatel Aug 13, 2025
0b6abaa
consolidate policy unit tests
vepatel Aug 13, 2025
a52d7a2
add example and readme
vepatel Aug 13, 2025
8337f42
Merge branch 'main' into feat/cache-policy
vepatel Aug 13, 2025
09c56c3
remove empty test files
vepatel Aug 13, 2025
7b4926f
update example readme with suggestion
vepatel Aug 13, 2025
eb94bd8
remove pv-pvc yaml
vepatel Aug 13, 2025
5173313
fix test marker in workflow data
vepatel Aug 13, 2025
7207b5b
Merge branch 'main' into feat/cache-policy
vepatel Aug 14, 2025
29412e9
Merge branch 'main' into feat/cache-policy
vepatel Aug 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/data/matrix-smoke-oss.json
Original file line number Diff line number Diff line change
Expand Up @@ -39,14 +39,14 @@
"label": "policies 1/2",
"image": "alpine",
"type": "oss",
"marker": "'policies and not policies_rl and not policies_ac and not policies_jwt and not policies_mtls'",
"marker": "'policies and not policies_rl and not policies_ac and not policies_jwt and not policies_mtls and not policies_cache'",
"platforms": "linux/arm64, linux/amd64"
},
{
"label": "policies 2/2",
"image": "alpine",
"type": "oss",
"marker": "'policies_rl or policies_ac or policies_jwt or policies_mtls or otel'",
"marker": "'policies_rl or policies_ac or policies_jwt or policies_mtls or policies_cache or otel'",
"platforms": "linux/arm64, linux/amd64"
},
{
Expand Down
4 changes: 2 additions & 2 deletions .github/data/matrix-smoke-plus.json
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@
"label": "policies 1/3",
"image": "ubi-9-plus",
"type": "plus",
"marker": "'policies and not policies_ac and not policies_jwt and not policies_mtls and not policies_rl'",
"marker": "'policies and not policies_ac and not policies_jwt and not policies_mtls and not policies_rl and not policies_cache'",
"platforms": "linux/arm64, linux/amd64"
},
{
Expand All @@ -81,7 +81,7 @@
"label": "policies 3/3",
"image": "ubi-9-plus",
"type": "plus",
"marker": "policies_rl",
"marker": "'policies_rl or policies_cache'",
"platforms": "linux/arm64, linux/amd64"
},
{
Expand Down
13 changes: 13 additions & 0 deletions charts/nginx-ingress/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -352,14 +352,24 @@ List of volumes for controller.
{{- if eq (include "nginx-ingress.readOnlyRootFilesystem" .) "true" }}
- name: nginx-etc
emptyDir: {}
{{- if .Values.controller.cache.enableShared }}
- name: nginx-cache
persistentVolumeClaim:
claimName: {{ .Values.controller.cache.sharedPVCName }}
{{- else }}
- name: nginx-cache
emptyDir: {}
{{- end }}
- name: nginx-lib
emptyDir: {}
- name: nginx-state
emptyDir: {}
- name: nginx-log
emptyDir: {}
{{- else if .Values.controller.cache.enableShared }}
- name: nginx-cache
persistentVolumeClaim:
claimName: {{ .Values.controller.cache.sharedPVCName }}
{{- end }}
{{- if .Values.controller.appprotect.v5 }}
{{ toYaml .Values.controller.appprotect.volumes }}
Expand Down Expand Up @@ -419,6 +429,9 @@ volumeMounts:
name: nginx-state
- mountPath: /var/log/nginx
name: nginx-log
{{- else if .Values.controller.cache.enableShared }}
- mountPath: /var/cache/nginx
name: nginx-cache
{{- end }}
{{- if .Values.controller.appprotect.v5 }}
- name: app-protect-bd-config
Expand Down
8 changes: 8 additions & 0 deletions charts/nginx-ingress/templates/controller-service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,14 @@ spec:
{{- end }}
selector:
{{- include "nginx-ingress.selectorLabels" . | nindent 4 }}
{{- if .Values.controller.service.sessionAffinity.enable }}
sessionAffinity: {{ .Values.controller.service.sessionAffinity.type }}
{{- if eq .Values.controller.service.sessionAffinity.type "ClientIP" }}
sessionAffinityConfig:
clientIP:
timeoutSeconds: {{ .Values.controller.service.sessionAffinity.timeoutSeconds }}
{{- end }}
{{- end }}
{{- if .Values.controller.service.externalIPs }}
externalIPs:
{{ toYaml .Values.controller.service.externalIPs | indent 4 }}
Expand Down
51 changes: 50 additions & 1 deletion charts/nginx-ingress/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -1455,6 +1455,50 @@
"type": "object",
"ref": "https://raw.githubusercontent.com/nginxinc/kubernetes-json-schema/master/v1.33.1/_definitions.json#/definitions/io.k8s.api.core.v1.ServicePort"
}
},
"sessionAffinity": {
"type": "object",
"default": {},
"title": "The sessionAffinity Schema",
"required": [],
"properties": {
"enable": {
"type": "boolean",
"default": false,
"title": "Enable session affinity",
"examples": [
false
]
},
"type": {
"type": "string",
"default": "ClientIP",
"title": "Session affinity type",
"enum": [
"ClientIP"
],
"examples": [
"ClientIP"
]
},
"timeoutSeconds": {
"type": "integer",
"default": 3600,
"title": "Session affinity timeout in seconds",
"minimum": 1,
"maximum": 86400,
"examples": [
3600
]
}
},
"examples": [
{
"enable": false,
"type": "ClientIP",
"timeoutSeconds": 3600
}
]
}
},
"examples": [
Expand Down Expand Up @@ -1483,7 +1527,12 @@
"targetPort": 443,
"name": "https"
},
"customPorts": []
"customPorts": [],
"sessionAffinity": {
"enable": false,
"type": "ClientIP",
"timeoutSeconds": 3600
}
}
]
},
Expand Down
19 changes: 19 additions & 0 deletions charts/nginx-ingress/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,16 @@ controller:
## Sets the log format of Ingress Controller. Options include: glog, json, text
logFormat: glog

## Cache configuration options
cache:
## Enables shared cache across multiple pods using an external persistent volume
## When enabled, the /var/cache/nginx directory will be mounted from a PVC instead of using emptyDir
## User must create and configure a PVC with appropriate access mode
enableShared: false

## The name of the PersistentVolumeClaim to use for shared cache, should match the name of the PVC created by the user
sharedPVCName: "nginx-shared-cache"

## A list of custom ports to expose on the NGINX Ingress Controller pod. Follows the conventional Kubernetes yaml syntax for container ports.
customPorts: []

Expand Down Expand Up @@ -502,6 +512,15 @@ controller:
## A list of custom ports to expose through the Ingress Controller service. Follows the conventional Kubernetes yaml syntax for service ports.
customPorts: []

## Session affinity configuration for the Ingress Controller service, ensures requests from the same client IP go to the same pod
sessionAffinity:
## Enable session affinity. Valid values: None, ClientIP
enable: false
## Session affinity type. Currently only ClientIP is supported.
type: ClientIP
## Session affinity timeout in seconds (default: 3600 = 1 hour)
timeoutSeconds: 3600

serviceAccount:
## The annotations of the service account of the Ingress Controller pods.
annotations: {}
Expand Down
86 changes: 86 additions & 0 deletions config/crd/bases/k8s.nginx.org_policies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,92 @@ spec:
otherwise the secret will be rejected as invalid.
type: string
type: object
cache:
description: The Cache Key defines a cache policy for proxy caching
properties:
allowedCodes:
description: |-
AllowedCodes defines which HTTP response codes should be cached.
Accepts either:
- The string "any" to cache all response codes (must be the only element)
- A list of HTTP status codes as integers (100-599)
Examples: ["any"], [200, 301, 404], [200].
Invalid: ["any", 200] (cannot mix "any" with specific codes).
items:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
type: array
allowedMethods:
description: |-
AllowedMethods defines which HTTP methods should be cached.
Only "GET", "HEAD", and "POST" are supported by NGINX proxy_cache_methods directive.
GET and HEAD are always cached by default even if not specified.
Maximum of 3 items allowed. Examples: ["GET"], ["GET", "HEAD", "POST"].
Invalid methods: PUT, DELETE, PATCH, etc.
items:
type: string
maxItems: 3
type: array
x-kubernetes-validations:
- message: 'allowed methods must be one of: GET, HEAD, POST'
rule: self.all(method, method in ['GET', 'HEAD', 'POST'])
cachePurgeAllow:
description: |-
CachePurgeAllow defines IP addresses or CIDR blocks allowed to purge cache.
This feature is only available in NGINX Plus.
Examples: ["192.168.1.100", "10.0.0.0/8", "::1"].
Invalid in NGINX OSS (will be ignored).
items:
type: string
type: array
cacheZoneName:
description: |-
CacheZoneName defines the name of the cache zone. Must start with a lowercase letter,
followed by alphanumeric characters or underscores, and end with an alphanumeric character.
Single lowercase letters are also allowed. Examples: "cache", "my_cache", "cache1".
pattern: ^[a-z][a-zA-Z0-9_]*[a-zA-Z0-9]$|^[a-z]$
type: string
cacheZoneSize:
description: |-
CacheZoneSize defines the size of the cache zone. Must be a number followed by a size unit:
'k' for kilobytes, 'm' for megabytes, or 'g' for gigabytes.
Examples: "10m", "1g", "512k".
pattern: ^[0-9]+[kmg]$
type: string
levels:
description: |-
Levels defines the cache directory hierarchy levels for storing cached files.
Must be in format "X:Y" or "X:Y:Z" where X, Y, Z are either 1 or 2.
This controls the number of subdirectory levels and their name lengths.
Examples: "1:2", "2:2", "1:2:2".
Invalid: "3:1", "1:3", "1:2:3".
pattern: ^[12](?::[12]){0,2}$
type: string
overrideUpstreamCache:
default: false
description: |-
OverrideUpstreamCache controls whether to override upstream cache headers
(using proxy_ignore_headers directive). When true, NGINX will ignore
cache-related headers from upstream servers like Cache-Control, Expires, etc.
Default: false.
type: boolean
time:
description: |-
Time defines the default cache time. Required when allowedCodes is specified.
Must be a number followed by a time unit:
's' for seconds, 'm' for minutes, 'h' for hours, 'd' for days.
Examples: "30s", "5m", "1h", "2d".
pattern: ^[0-9]+[smhd]$
type: string
required:
- cacheZoneName
- cacheZoneSize
type: object
x-kubernetes-validations:
- message: time is required when allowedCodes is specified
rule: '!has(self.allowedCodes) || (has(self.allowedCodes) && has(self.time))'
egressMTLS:
description: The EgressMTLS policy configures upstreams authentication
and certificate verification.
Expand Down
86 changes: 86 additions & 0 deletions deploy/crds.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,92 @@ spec:
otherwise the secret will be rejected as invalid.
type: string
type: object
cache:
description: The Cache Key defines a cache policy for proxy caching
properties:
allowedCodes:
description: |-
AllowedCodes defines which HTTP response codes should be cached.
Accepts either:
- The string "any" to cache all response codes (must be the only element)
- A list of HTTP status codes as integers (100-599)
Examples: ["any"], [200, 301, 404], [200].
Invalid: ["any", 200] (cannot mix "any" with specific codes).
items:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
type: array
allowedMethods:
description: |-
AllowedMethods defines which HTTP methods should be cached.
Only "GET", "HEAD", and "POST" are supported by NGINX proxy_cache_methods directive.
GET and HEAD are always cached by default even if not specified.
Maximum of 3 items allowed. Examples: ["GET"], ["GET", "HEAD", "POST"].
Invalid methods: PUT, DELETE, PATCH, etc.
items:
type: string
maxItems: 3
type: array
x-kubernetes-validations:
- message: 'allowed methods must be one of: GET, HEAD, POST'
rule: self.all(method, method in ['GET', 'HEAD', 'POST'])
cachePurgeAllow:
description: |-
CachePurgeAllow defines IP addresses or CIDR blocks allowed to purge cache.
This feature is only available in NGINX Plus.
Examples: ["192.168.1.100", "10.0.0.0/8", "::1"].
Invalid in NGINX OSS (will be ignored).
items:
type: string
type: array
cacheZoneName:
description: |-
CacheZoneName defines the name of the cache zone. Must start with a lowercase letter,
followed by alphanumeric characters or underscores, and end with an alphanumeric character.
Single lowercase letters are also allowed. Examples: "cache", "my_cache", "cache1".
pattern: ^[a-z][a-zA-Z0-9_]*[a-zA-Z0-9]$|^[a-z]$
type: string
cacheZoneSize:
description: |-
CacheZoneSize defines the size of the cache zone. Must be a number followed by a size unit:
'k' for kilobytes, 'm' for megabytes, or 'g' for gigabytes.
Examples: "10m", "1g", "512k".
pattern: ^[0-9]+[kmg]$
type: string
levels:
description: |-
Levels defines the cache directory hierarchy levels for storing cached files.
Must be in format "X:Y" or "X:Y:Z" where X, Y, Z are either 1 or 2.
This controls the number of subdirectory levels and their name lengths.
Examples: "1:2", "2:2", "1:2:2".
Invalid: "3:1", "1:3", "1:2:3".
pattern: ^[12](?::[12]){0,2}$
type: string
overrideUpstreamCache:
default: false
description: |-
OverrideUpstreamCache controls whether to override upstream cache headers
(using proxy_ignore_headers directive). When true, NGINX will ignore
cache-related headers from upstream servers like Cache-Control, Expires, etc.
Default: false.
type: boolean
time:
description: |-
Time defines the default cache time. Required when allowedCodes is specified.
Must be a number followed by a time unit:
's' for seconds, 'm' for minutes, 'h' for hours, 'd' for days.
Examples: "30s", "5m", "1h", "2d".
pattern: ^[0-9]+[smhd]$
type: string
required:
- cacheZoneName
- cacheZoneSize
type: object
x-kubernetes-validations:
- message: time is required when allowedCodes is specified
rule: '!has(self.allowedCodes) || (has(self.allowedCodes) && has(self.time))'
egressMTLS:
description: The EgressMTLS policy configures upstreams authentication
and certificate verification.
Expand Down
Loading
Loading