e2e-tests.md

e2e test suite for Ingress NGINX Controller

[Admission] admission controller

• reject ingress with global-rate-limit annotations when memcached is not configured
• should not allow overlaps of host and paths without canary annotations
• should allow overlaps of host and paths with canary annotation
• should block ingress with invalid path
• should return an error if there is an error validating the ingress definition
• should return an error if there is an invalid value in some annotation
• should return an error if there is a forbidden value in some annotation
• should return an error if there is an invalid path and wrong pathType is set
• should not return an error if the Ingress V1 definition is valid with Ingress Class
• should not return an error if the Ingress V1 definition is valid with IngressClass annotation
• should return an error if the Ingress V1 definition contains invalid annotations
• should not return an error for an invalid Ingress when it has unknown class

affinity session-cookie-name

• should set sticky cookie SERVERID
• should change cookie name on ingress definition change
• should set the path to /something on the generated cookie
• does not set the path to / on the generated cookie if there's more than one rule referring to the same backend
• should set cookie with expires
• should set cookie with domain
• should not set cookie without domain annotation
• should work with use-regex annotation and session-cookie-path
• should warn user when use-regex is true and session-cookie-path is not set
• should not set affinity across all server locations when using separate ingresses
• should set sticky cookie without host
• should work with server-alias annotation
• should set secure in cookie with provided true annotation on http
• should not set secure in cookie with provided false annotation on http
• should set secure in cookie with provided false annotation on https

affinitymode

• Balanced affinity mode should balance
• Check persistent affinity mode

server-alias

• should return status code 200 for host 'foo' and 404 for 'bar'
• should return status code 200 for host 'foo' and 'bar'
• should return status code 200 for hosts defined in two ingresses, different path with one alias

app-root

• should redirect to /foo

auth-*

• should return status code 200 when no authentication is configured
• should return status code 503 when authentication is configured with an invalid secret
• should return status code 401 when authentication is configured but Authorization header is not configured
• should return status code 401 when authentication is configured and Authorization header is sent with invalid credentials
• should return status code 401 and cors headers when authentication and cors is configured but Authorization header is not configured
• should return status code 200 when authentication is configured and Authorization header is sent
• should return status code 200 when authentication is configured with a map and Authorization header is sent
• should return status code 401 when authentication is configured with invalid content and Authorization header is sent
• when external auth is configured
• when external auth is not configured
• when auth-headers are set
• should set cache_key when external auth cache is configured
• user retains cookie by default
• user does not retain cookie if upstream returns error status code
• user with annotated ingress retains cookie if upstream returns error status code
• should return status code 200 when signed in
• should redirect to signin url when not signed in
• keeps processing new ingresses even if one of the existing ingresses is misconfigured
• should overwrite Foo header with auth response
• should not create additional upstream block when auth-keepalive is not set
• should not create additional upstream block when host part of auth-url contains a variable
• should not create additional upstream block when auth-keepalive is negative
• should not create additional upstream block when auth-keepalive is set with HTTP/2
• should create additional upstream block when auth-keepalive is set with HTTP/1.x
• should return status code 200 when signed in
• should redirect to signin url when not signed in
• keeps processing new ingresses even if one of the existing ingresses is misconfigured
• should return status code 200 when signed in after auth backend is deleted
• should deny login for different location on same server
• should deny login for different servers
• should redirect to signin url when not signed in
• should return 503 (location was denied)
• should add error to the config

auth-tls-*

• should set sslClientCertificate, sslVerifyClient and sslVerifyDepth with auth-tls-secret
• should set valid auth-tls-secret, sslVerify to off, and sslVerifyDepth to 2
• should 302 redirect to error page instead of 400 when auth-tls-error-page is set
• should pass URL-encoded certificate to upstream
• should validate auth-tls-verify-client
• should return 403 using auth-tls-match-cn with no matching CN from client
• should return 200 using auth-tls-match-cn with matching CN from client
• should return 200 using auth-tls-match-cn where atleast one of the regex options matches CN from client

backend-protocol

• should set backend protocol to https:// and use proxy_pass
• should set backend protocol to $scheme:// and use proxy_pass
• should set backend protocol to grpc:// and use grpc_pass
• should set backend protocol to grpcs:// and use grpc_pass
• should set backend protocol to '' and use fastcgi_pass
• should set backend protocol to '' and use ajp_pass

canary-*

• should response with a 200 status from the mainline upstream when requests are made to the mainline ingress
• should return 404 status for requests to the canary if no matching ingress is found
• should return the correct status codes when endpoints are unavailable
• should route requests to the correct upstream if mainline ingress is created before the canary ingress
• should route requests to the correct upstream if mainline ingress is created after the canary ingress
• should route requests to the correct upstream if the mainline ingress is modified
• should route requests to the correct upstream if the canary ingress is modified
• should route requests to the correct upstream
• should route requests to the correct upstream
• should route requests to the correct upstream
• should route requests to the correct upstream
• should routes to mainline upstream when the given Regex causes error
• should route requests to the correct upstream
• respects always and never values
• should route requests only to mainline if canary weight is 0
• should route requests only to canary if canary weight is 100
• should route requests only to canary if canary weight is equal to canary weight total
• should route requests split between mainline and canary if canary weight is 50
• should route requests split between mainline and canary if canary weight is 100 and weight total is 200
• should not use canary as a catch-all server
• should not use canary with domain as a server
• does not crash when canary ingress has multiple paths to the same non-matching backend
• always routes traffic to canary if first request was affinitized to canary (default behavior)
• always routes traffic to canary if first request was affinitized to canary (explicit sticky behavior)
• routes traffic to either mainline or canary backend (legacy behavior)

client-body-buffer-size

• should set client_body_buffer_size to 1000
• should set client_body_buffer_size to 1K
• should set client_body_buffer_size to 1k
• should set client_body_buffer_size to 1m
• should set client_body_buffer_size to 1M
• should not set client_body_buffer_size to invalid 1b

connection-proxy-header

• set connection header to keep-alive

cors-*

• should enable cors
• should set cors methods to only allow POST, GET
• should set cors max-age
• should disable cors allow credentials
• should allow origin for cors
• should allow headers for cors
• should expose headers for cors
• should allow - single origin for multiple cors values
• should not allow - single origin for multiple cors values
• should allow correct origins - single origin for multiple cors values
• should not break functionality
• should not break functionality with extra domain
• should not match
• should allow - single origin with required port
• should not allow - single origin with port and origin without port
• should not allow - single origin without port and origin with required port
• should allow - matching origin with wildcard origin (2 subdomains)
• should not allow - unmatching origin with wildcard origin (2 subdomains)
• should allow - matching origin+port with wildcard origin
• should not allow - portless origin with wildcard origin
• should allow correct origins - missing subdomain + origin with wildcard origin and correct origin
• should allow - missing origins (should allow all origins)

custom-http-errors

• configures Nginx correctly

default-backend

• should use a custom default backend as upstream

disable-access-log disable-http-access-log disable-stream-access-log

• disable-access-log set access_log off
• disable-http-access-log set access_log off
• disable-stream-access-log set access_log off

backend-protocol - FastCGI

• should use fastcgi_pass in the configuration file
• should add fastcgi_index in the configuration file
• should add fastcgi_param in the configuration file
• should return OK for service with backend protocol FastCGI

force-ssl-redirect

• should redirect to https

from-to-www-redirect

• should redirect from www HTTP to HTTP
• should redirect from www HTTPS to HTTPS

annotation-global-rate-limit

• generates correct configuration

backend-protocol - GRPC

• should use grpc_pass in the configuration file
• should return OK for service with backend protocol GRPC
• authorization metadata should be overwritten by external auth response headers
• should return OK for service with backend protocol GRPCS

http2-push-preload

• enable the http2-push-preload directive

denylist-source-range

• only deny explicitly denied IPs, allow all others
• only allow explicitly allowed IPs, deny all others

whitelist-source-range

• should set valid ip whitelist range

Annotation - limit-connections

• should limit-connections

limit-rate

• Check limit-rate annotation

enable-access-log enable-rewrite-log

• set access_log off
• set rewrite_log on

mirror-*

• should set mirror-target to http://localhost/mirror
• should set mirror-target to https://test.env.com/$request_uri
• should disable mirror-request-body

modsecurity owasp

• should enable modsecurity
• should enable modsecurity with transaction ID and OWASP rules
• should disable modsecurity
• should enable modsecurity with snippet
• should enable modsecurity without using 'modsecurity on;'
• should disable modsecurity using 'modsecurity off;'
• should enable modsecurity with snippet and block requests
• should enable modsecurity globally and with modsecurity-snippet block requests
• should enable modsecurity when enable-owasp-modsecurity-crs is set to true
• should enable modsecurity through the config map
• should enable modsecurity through the config map but ignore snippet as disabled by admin
• should disable default modsecurity conf setting when modsecurity-snippet is specified

preserve-trailing-slash

• should allow preservation of trailing slashes

proxy-*

• should set proxy_redirect to off
• should set proxy_redirect to default
• should set proxy_redirect to hello.com goodbye.com
• should set proxy client-max-body-size to 8m
• should not set proxy client-max-body-size to incorrect value
• should set valid proxy timeouts
• should not set invalid proxy timeouts
• should turn on proxy-buffering
• should turn off proxy-request-buffering
• should build proxy next upstream
• should setup proxy cookies
• should change the default proxy HTTP version

proxy-ssl-*

• should set valid proxy-ssl-secret
• should set valid proxy-ssl-secret, proxy-ssl-verify to on, proxy-ssl-verify-depth to 2, and proxy-ssl-server-name to on
• should set valid proxy-ssl-secret, proxy-ssl-ciphers to HIGH:!AES
• should set valid proxy-ssl-secret, proxy-ssl-protocols
• proxy-ssl-location-only flag should change the nginx config server part

permanent-redirect permanent-redirect-code

• should respond with a standard redirect code
• should respond with a custom redirect code

rewrite-target use-regex enable-rewrite-log

• should write rewrite logs
• should use correct longest path match
• should use ~* location modifier if regex annotation is present
• should fail to use longest match for documented warning
• should allow for custom rewrite parameters

satisfy

• should configure satisfy directive correctly
• should allow multiple auth with satisfy any

server-snippet

• add valid directives to server via server snippet
• drops server snippet if disabled by the administrator

service-upstream

• should use the Service Cluster IP and Port
• should use the Service Cluster IP and Port
• should not use the Service Cluster IP and Port

configuration-snippet

ssl-ciphers

• should change ssl ciphers

stream-snippet

• should add value of stream-snippet to nginx config
• should add stream-snippet and drop annotations per admin config

upstream-hash-by-*

• should connect to the same pod
• should connect to the same subset of pods

upstream-vhost

• set host to upstreamvhost.bar.com

x-forwarded-prefix

• should set the X-Forwarded-Prefix to the annotation value
• should not add X-Forwarded-Prefix if the annotation value is empty

Debug CLI

• should list the backend servers
• should get information for a specific backend server
• should produce valid JSON for /dbg general

[Default Backend] custom service

• uses custom default backend that returns 200 as status code

[Default Backend]

• should return 404 sending requests when only a default backend is running
• enables access logging for default backend
• disables access logging for default backend

[Default Backend] SSL

• should return a self generated SSL certificate

[Default Backend] change default settings

• should apply the annotation to the default backend

[Endpointslices] long service name

• should return 200 when service name has max allowed number of characters 63

[TopologyHints] topology aware routing

• should return 200 when service has topology hints

[Setting]

[Shutdown] Grace period shutdown

• /healthz should return status code 500 during shutdown grace period

[Shutdown] ingress controller

• should shutdown in less than 60 secons without pending connections
• should shutdown after waiting 60 seconds for pending connections to be closed
• should shutdown after waiting 150 seconds for pending connections to be closed

[Shutdown] Graceful shutdown with pending request

• should let slow requests finish before shutting down

[Ingress] DeepInspection

• should drop whole ingress if one path matches invalid regex

single ingress - multiple hosts

• should set the correct $service_name NGINX variable

[Ingress] [PathType] exact

• should choose exact location for /exact

[Ingress] [PathType] mix Exact and Prefix paths

• should choose the correct location

[Ingress] [PathType] prefix checks

• should return 404 when prefix /aaa does not match request /aaaccc

[Ingress] definition without host

• should set ingress details variables for ingresses without a host
• should set ingress details variables for ingresses with host without IngressRuleValue, only Backend

[Memory Leak] Dynamic Certificates

• should not leak memory from ingress SSL certificates or configuration updates

[Load Balancer] load-balance

• should apply the configmap load-balance setting

[Load Balancer] EWMA

• does not fail requests

[Load Balancer] round-robin

• should evenly distribute requests with round-robin (default algorithm)

[Lua] dynamic certificates

• picks up the certificate when we add TLS spec to existing ingress
• picks up the previously missing secret for a given ingress without reloading
• supports requests with domain with trailing dot
• picks up the updated certificate without reloading
• falls back to using default certificate when secret gets deleted without reloading
• picks up a non-certificate only change
• removes HTTPS configuration when we delete TLS spec

[Lua] dynamic configuration

• configures balancer Lua middleware correctly
• handles endpoints only changes
• handles endpoints only changes (down scaling of replicas)
• handles endpoints only changes consistently (down scaling of replicas vs. empty service)
• handles an annotation change

[metrics] exported prometheus metrics

• exclude socket request metrics are absent
• exclude socket request metrics are present

nginx-configuration

• start nginx with default configuration
• fails when using alias directive
• fails when using root directive

[Security] request smuggling

• should not return body content from error_page

[Service] backend status code 503

• should return 503 when backend service does not exist
• should return 503 when all backend service endpoints are unavailable

[Service] Type ExternalName

• works with external name set to incomplete fqdn
• should return 200 for service type=ExternalName without a port defined
• should return 200 for service type=ExternalName with a port defined
• should return status 502 for service type=ExternalName with an invalid host
• should return 200 for service type=ExternalName using a port name
• should return 200 for service type=ExternalName using FQDN with trailing dot
• should update the external name after a service update
• should sync ingress on external name service addition/deletion

[Service] Nil Service Backend

• should return 404 when backend service is nil

access-log

• use the default configuration
• use the specified configuration
• use the specified configuration
• use the specified configuration
• use the specified configuration

Bad annotation values

• [BAD_ANNOTATIONS] should drop an ingress if there is an invalid character in some annotation
• [BAD_ANNOTATIONS] should drop an ingress if there is a forbidden word in some annotation
• [BAD_ANNOTATIONS] should allow an ingress if there is a default blocklist config in place
• [BAD_ANNOTATIONS] should drop an ingress if there is a custom blocklist config in place and allow others to pass

brotli

• condition

Configmap change

• should reload after an update in the configuration

add-headers

• Add a custom header
• Add multiple custom headers

[SSL] [Flag] default-ssl-certificate

• uses default ssl certificate for catch-all ingress
• uses default ssl certificate for host based ingress when configured certificate does not match host

[Flag] disable-catch-all

• should ignore catch all Ingress with backend
• should ignore catch all Ingress with backend and rules
• should delete Ingress updated to catch-all
• should allow Ingress with rules

[Flag] disable-service-external-name

• should ignore services of external-name type

[Flag] disable-sync-events

• should create sync events (default)
• should create sync events
• should not create sync events

enable-real-ip

• trusts X-Forwarded-For header only when setting is true
• should not trust X-Forwarded-For header when setting is false

use-forwarded-headers

• should trust X-Forwarded headers when setting is true
• should not trust X-Forwarded headers when setting is false

Geoip2

• should include geoip2 line in config when enabled and db file exists
• should only allow requests from specific countries

[Security] block-*

• should block CIDRs defined in the ConfigMap
• should block User-Agents defined in the ConfigMap
• should block Referers defined in the ConfigMap

[Security] global-auth-url

• should return status code 401 when request any protected service
• should return status code 200 when request whitelisted (via no-auth-locations) service and 401 when request protected service
• should return status code 200 when request whitelisted (via ingress annotation) service and 401 when request protected service
• should still return status code 200 after auth backend is deleted using cache
• should proxy_method method when global-auth-method is configured
• should add custom error page when global-auth-signin url is configured
• should add auth headers when global-auth-response-headers is configured
• should set request-redirect when global-auth-request-redirect is configured
• should set snippet when global external auth is configured
• user retains cookie by default
• user does not retain cookie if upstream returns error status code
• user with global-auth-always-set-cookie key in configmap retains cookie if upstream returns error status code

global-options

• should have worker_rlimit_nofile option
• should have worker_rlimit_nofile option and be independent on amount of worker processes

settings-global-rate-limit

• generates correct NGINX configuration

gzip

• should be disabled by default
• should be enabled with default settings
• should set gzip_comp_level to 4
• should set gzip_disable to msie6
• should set gzip_min_length to 100
• should set gzip_types to application/javascript

hash size

• should set server_names_hash_bucket_size
• should set server_names_hash_max_size
• should set proxy-headers-hash-bucket-size
• should set proxy-headers-hash-max-size
• should set variables-hash-bucket-size
• should set variables-hash-max-size
• should set vmap-hash-bucket-size

[Flag] ingress-class

• should ignore Ingress with a different class annotation
• should ignore Ingress with different controller class
• should accept both Ingresses with default IngressClassName and IngressClass annotation
• should ignore Ingress without IngressClass configuration
• should delete Ingress when class is removed
• should serve Ingress when class is added
• should serve Ingress when class is updated between annotation and ingressClassName
• should ignore Ingress with no class and accept the correctly configured Ingresses
• should watch Ingress with no class and ignore ingress with a different class
• should watch Ingress that uses the class name even if spec is different
• should watch Ingress with correct annotation
• should ignore Ingress with only IngressClassName

keep-alive keep-alive-requests

• should set keepalive_timeout
• should set keepalive_requests
• should set keepalive connection to upstream server
• should set keep alive connection timeout to upstream server
• should set keepalive time to upstream server
• should set the request count to upstream server through one keep alive connection

Configmap - limit-rate

• Check limit-rate config

[Flag] custom HTTP and HTTPS ports

• should set X-Forwarded-Port headers accordingly when listening on a non-default HTTP port
• should set X-Forwarded-Port header to 443
• should set the X-Forwarded-Port header to 443

log-format-*

• should not configure log-format escape by default
• should enable the log-format-escape-json
• should disable the log-format-escape-json
• should enable the log-format-escape-none
• should disable the log-format-escape-none
• log-format-escape-json enabled
• log-format default escape
• log-format-escape-none enabled

[Lua] lua-shared-dicts

• configures lua shared dicts

main-snippet

• should add value of main-snippet setting to nginx config

[Security] modsecurity-snippet

• should add value of modsecurity-snippet setting to nginx config

enable-multi-accept

• should be enabled by default
• should be enabled when set to true
• should be disabled when set to false

[Flag] watch namespace selector

• should ingore Ingress of namespace without label foo=bar and accept those of namespace with label foo=bar

[Security] no-auth-locations

• should return status code 401 when accessing '/' unauthentication
• should return status code 200 when accessing '/' authentication
• should return status code 200 when accessing '/noauth' unauthenticated

Add no tls redirect locations

• Check no tls redirect locations config

OCSP

• should enable OCSP and contain stapling information in the connection

Configure Opentelemetry

• should not exists opentelemetry directive
• should exists opentelemetry directive when is enabled
• should include opentelemetry_trust_incoming_spans on directive when enabled
• should not exists opentelemetry_operation_name directive when is empty
• should exists opentelemetry_operation_name directive when is configured

Configure OpenTracing

• should not exists opentracing directive
• should exists opentracing directive when is enabled
• should include opentracing_trust_incoming_span off directive when disabled
• should not exists opentracing_operation_name directive when is empty
• should exists opentracing_operation_name directive when is configured
• should not exists opentracing_location_operation_name directive when is empty
• should exists opentracing_location_operation_name directive when is configured
• should enable opentracing using zipkin
• should enable opentracing using jaeger
• should enable opentracing using jaeger with sampler host
• should propagate the w3c header when configured with jaeger
• should enable opentracing using jaeger with an HTTP endpoint
• should enable opentracing using datadog

plugins

• should exist a x-hello-world header

[Security] Pod Security Policies

• should be running with a Pod Security Policy

[Security] Pod Security Policies with volumes

• should be running with a Pod Security Policy

proxy-connect-timeout

• should set valid proxy timeouts using configmap values
• should not set invalid proxy timeouts using configmap values

Dynamic $proxy_host

• should exist a proxy_host
• should exist a proxy_host using the upstream-vhost annotation value

proxy-next-upstream

• should build proxy next upstream using configmap values

use-proxy-protocol

• should respect port passed by the PROXY Protocol
• should respect proto passed by the PROXY Protocol server port
• should enable PROXY Protocol for HTTPS
• should enable PROXY Protocol for TCP

proxy-read-timeout

• should set valid proxy read timeouts using configmap values
• should not set invalid proxy read timeouts using configmap values

proxy-send-timeout

• should set valid proxy send timeouts using configmap values
• should not set invalid proxy send timeouts using configmap values

reuse-port

• reuse port should be enabled by default
• reuse port should be disabled
• reuse port should be enabled

configmap server-snippet

• should add value of server-snippet setting to all ingress config
• should add global server-snippet and drop annotations per admin config

server-tokens

• should not exists Server header in the response
• should exists Server header in the response when is enabled

ssl-ciphers

• Add ssl ciphers

With enable-ssl-passthrough enabled

• should enable ssl-passthrough-proxy-port on a different port
• should pass unknown traffic to default backend and handle known traffic

configmap stream-snippet

• should add value of stream-snippet via config map to nginx config

[SSL] TLS protocols, ciphers and headers)

• setting cipher suite
• setting max-age parameter
• setting includeSubDomains parameter
• setting preload parameter
• overriding what's set from the upstream
• should not use ports during the HTTP to HTTPS redirection
• should not use ports or X-Forwarded-Host during the HTTP to HTTPS redirection

[SSL] redirect to HTTPS

• should redirect from HTTP to HTTPS when secret is missing

[SSL] secret update

• should not appear references to secret updates not used in ingress rules
• should return the fake SSL certificate if the secret is invalid

[Status] status update

• should update status field after client-go reconnection

[TCP] tcp-services

• should expose a TCP service
• should expose an ExternalName TCP service
• should reload after an update in the configuration