Skip to content

Research Gateway API as replacement for rolling-updates #260

@heruan

Description

@heruan

Recent changes in Kubernetes networking APIs require us to re-evaluate how Kubernetes Kit supports application rolling updates and in-app notifications for new versions.

Our current documentation describes how to use NGINX Ingress annotations and rules to implement:

  1. Rolling updates with smooth traffic migration
    https://vaadin.com/docs/latest/tools/kubernetes/update-version

  2. In-app notifications when a new application version is available
    Implemented by enabling NGINX configuration snippets (deprecated since NGINX 1.9) to set a cookie on responses.

However:

1. The NGINX Ingress reference implementation of the Ingress API is being discontinued

Kubernetes announced formal deprecation and retirement of the NGINX Ingress Controller. Support ends March 2026, and Kubernetes recommends adopting the Gateway API as a modern, extensible replacement. This affects Kubernetes Kit because our documented rollout and notification mechanisms rely on Ingress API semantics, annotations, and NGINX-specific behavior.

2. The current mechanism for in-app version notifications does not work

Beginning with NGINX Ingress Controller 1.9.0, the cookie-based routing trick used in our documentation stopped working as NGINX removed non-secure annotation-based configuration snippets.

We must research and design a Gateway API approach that supports:

  • Rolling update routing (directing portions of traffic to the new revision, observing readiness, switching over)
  • In-app notifications (the application needs awareness that a newer deployment exists)
  • Secure configuration without path-routing hacks or controller-specific annotations
  • Compatibility with common Gateway implementations

Challenges

Migrating from Ingress to Gateway API is not a drop-in replacement. While both expose HTTP routing for Kubernetes workloads, they differ significantly in architecture and capabilities:

  • Different resource model:
    Ingress bundles routing and controller behavior into a single resource, whereas Gateway API splits responsibilities across GatewayClass, Gateway, and HTTPRoute. This requires generating and managing multiple coordinated objects instead of one.

  • No annotation-based behavior:
    Ingress relies heavily on controller-specific annotations (e.g., NGINX canary annotations). Gateway API uses structured fields with explicit semantics. Existing rolling-update and version-notification logic cannot be ported as-is because those annotations simply do not exist in Gateway API.

  • Traffic splitting semantics differ:
    Ingress canary behavior is controller-specific and implemented by NGINX. Gateway API defines weight-based backend refs, but implementations may vary in feature completeness or require specific filters. This affects how Kubernetes Kit must implement progressive rollouts.

  • No built-in cookie hacks:
    The cookie-based technique we used to detect “older version still running” is tightly coupled to NGINX behavior and no longer works in NGINX Ingress ≥ 1.9.0. Gateway API treats security, conformance, and routing rules differently, so the same trick cannot be replicated.

  • Controller diversity:
    Gateway is a specification with multiple conformant implementations (Envoy, Istio, Kong, HAProxy, NGF). Behavior must be portable across them, meaning Kubernetes Kit would preferably not rely on implementation-specific extensions (although we can start with a reference implementation).

Overall, Gateway API requires rethinking how rolling updates and in-app version notifications are expressed. The API is more powerful and structured, but it replaces many annotation-driven shortcuts with explicit, portable configuration—so a direct mechanical migration is not feasible.

Metadata

Metadata

Assignees

Type

Projects

Status

Done

Status

Done / Pending Release

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions