Skip to content

Commit 2861c1b

Browse files
sukumargaonkaraabchooyuzisun
authored
feat: add route-level HeaderMutation support for request header manipulation (envoyproxy#1414)
**Description** This commit implements route-level HeaderMutation support for AIGatewayRouteRuleBackendRef. The feature allows users to define header mutations at the route level (per backend reference on AIGatewayRoute) in addition to the existing backend-level HeaderMutation support. **Key Features:** - **Route-level HeaderMutation**: New `headerMutation` field in `AIGatewayRouteRuleBackendRef` for fine-grained header control per route - **Smart merge logic**: When both route-level and backend-level HeaderMutation are defined, they are intelligently combined with route-level taking precedence for conflicts - **Case-insensitive handling**: Proper header name normalization and conflict resolution (e.g., `X-Custom` vs `x-custom` are treated as the same header) - **Full backward compatibility**: Existing backend-level only configurations continue to work unchanged **Use Cases:** **GCP Provisioned Throughput** [docs [1]]: - Unlike bedrock where provisioned-throughput and pay-as-you-go (on-demand) models get different endpoints, in GCP both provisioned-throughput and pay-as-you-go get the same endpoint. - whether a request should be counted under provisioned-throughput or pay-as-you-go needs to be controlled via headers. - This change will allow configuring custom headers at route-level ```yaml apiVersion: aigateway.envoyproxy.io/v1alpha1 kind: AIGatewayRoute metadata: annotations: name: gemini-2.5-flash namespace: gateway spec: # ... other spec fields ... rules: - backendRefs: # Provisioned Throughput on Region-1 - modelNameOverride: gemini-2.5-flash name: gcp-<region-1> priority: 0 headerMutation: # <-- New set: - name: X-Vertex-AI-LLM-Request-Type value: "dedicated" # Provisioned Throughput on Region-2 - modelNameOverride: gemini-2.5-flash name: gcp-<region-2> priority: 0 headerMutation: # <-- New set: - name: X-Vertex-AI-LLM-Request-Type value: "dedicated" # Pay-As-You-GO on Region-1 - modelNameOverride: gemini-2.5-flash name: gcp-<region-1> priority: 1 headerMutation: # <-- New set: - name: X-Vertex-AI-LLM-Request-Type value: "shared" # Pay-As-You-GO on Region-2 - modelNameOverride: gemini-2.5-flash name: gcp-<region-2> priority: 1 headerMutation: # <-- New set: - name: X-Vertex-AI-LLM-Request-Type value: "shared" # ... other spec fields ... ``` 1: https://cloud.google.com/vertex-ai/generative-ai/docs/provisioned-throughput/use-provisioned-throughput#limitations-of-the-dashboard --------- Signed-off-by: Sukumar Gaonkar <[email protected]> Signed-off-by: Dan Sun <[email protected]> Co-authored-by: Aaron Choo <[email protected]> Co-authored-by: Dan Sun <[email protected]>
1 parent 3df0a93 commit 2861c1b

File tree

6 files changed

+315
-1
lines changed

6 files changed

+315
-1
lines changed

‎api/v1alpha1/ai_gateway_route.go‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -317,6 +317,14 @@ type AIGatewayRouteRuleBackendRef struct {
317317
// +optional
318318
ModelNameOverride string `json:"modelNameOverride,omitempty"`
319319

320+
// HeaderMutation defines the request header mutation to be applied to this backend.
321+
// When both route-level and backend-level HeaderMutation are defined,
322+
// route-level takes precedence over backend-level for conflicting operations.
323+
// This field is ignored when referencing InferencePool resources.
324+
//
325+
// +optional
326+
HeaderMutation *HTTPHeaderMutation `json:"headerMutation,omitempty"`
327+
320328
// Weight is the weight of the backend. This is exactly the same as the weight in
321329
// the BackendRef in the Gateway API. See for the details:
322330
// https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io%2fv1.BackendRef

‎api/v1alpha1/zz_generated.deepcopy.go‎

Lines changed: 5 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎internal/controller/gateway.go‎

Lines changed: 57 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,53 @@ func headerMutationToFilterAPI(m *aigv1a1.HTTPHeaderMutation) *filterapi.HTTPHea
178178
return ret
179179
}
180180

181+
// mergeHeaderMutations merges route-level and backend-level HeaderMutation with route-level taking precedence.
182+
// Returns the merged HeaderMutation where route-level operations override backend-level operations for conflicting headers.
183+
func mergeHeaderMutations(routeLevel, backendLevel *aigv1a1.HTTPHeaderMutation) *aigv1a1.HTTPHeaderMutation {
184+
if routeLevel == nil {
185+
return backendLevel
186+
}
187+
if backendLevel == nil {
188+
return routeLevel
189+
}
190+
191+
result := &aigv1a1.HTTPHeaderMutation{}
192+
193+
// Merge Set operations (route-level wins conflicts)
194+
headerMap := make(map[string]gwapiv1.HTTPHeader)
195+
196+
// Add backend-level headers first
197+
for _, h := range backendLevel.Set {
198+
headerMap[strings.ToLower(string(h.Name))] = h
199+
}
200+
201+
// Override with route-level headers (route-level wins)
202+
for _, h := range routeLevel.Set {
203+
headerMap[strings.ToLower(string(h.Name))] = h
204+
}
205+
206+
// Convert back to slice
207+
for _, h := range headerMap {
208+
result.Set = append(result.Set, h)
209+
}
210+
211+
// Merge Remove operations (combine and deduplicate)
212+
removeMap := make(map[string]struct{})
213+
214+
for _, h := range backendLevel.Remove {
215+
removeMap[strings.ToLower(h)] = struct{}{}
216+
}
217+
for _, h := range routeLevel.Remove {
218+
removeMap[strings.ToLower(h)] = struct{}{}
219+
}
220+
221+
for h := range removeMap {
222+
result.Remove = append(result.Remove, h)
223+
}
224+
225+
return result
226+
}
227+
181228
// reconcileFilterConfigSecret updates the filter config secret for the external processor.
182229
func (c *GatewayController) reconcileFilterConfigSecret(
183230
ctx context.Context,
@@ -238,7 +285,16 @@ func (c *GatewayController) reconcileFilterConfigSecret(
238285
"namespace", backendNamespace)
239286
continue
240287
}
241-
b.HeaderMutation = headerMutationToFilterAPI(backendObj.Spec.HeaderMutation)
288+
289+
// Extract HeaderMutation from both route and backend levels
290+
routeHeaderMutation := backendRef.HeaderMutation
291+
backendHeaderMutation := backendObj.Spec.HeaderMutation
292+
293+
// Merge with route-level taking precedence over backend-level
294+
mergedHeaderMutation := mergeHeaderMutations(routeHeaderMutation, backendHeaderMutation)
295+
296+
// Convert to FilterAPI format
297+
b.HeaderMutation = headerMutationToFilterAPI(mergedHeaderMutation)
242298
b.Schema = schemaToFilterAPI(backendObj.Spec.APISchema)
243299
if bsp != nil {
244300
b.Auth, err = c.bspToFilterAPIBackendAuth(ctx, bsp)

‎internal/controller/gateway_test.go‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,8 @@ import (
1111
"testing"
1212
"time"
1313

14+
"github.com/google/go-cmp/cmp"
15+
"github.com/google/go-cmp/cmp/cmpopts"
1416
"github.com/stretchr/testify/require"
1517
"go.uber.org/zap/zapcore"
1618
appsv1 "k8s.io/api/apps/v1"
@@ -1013,3 +1015,158 @@ func TestGatewayController_reconcileFilterMCPConfigSecret(t *testing.T) {
10131015
require.NotNil(t, fc.MCPConfig)
10141016
require.Equal(t, "http://127.0.0.1:"+strconv.Itoa(internalapi.MCPBackendListenerPort), fc.MCPConfig.BackendListenerAddr)
10151017
}
1018+
1019+
func Test_mergeHeaderMutations(t *testing.T) {
1020+
tests := []struct {
1021+
name string
1022+
routeLevel *aigv1a1.HTTPHeaderMutation
1023+
backendLevel *aigv1a1.HTTPHeaderMutation
1024+
expected *aigv1a1.HTTPHeaderMutation
1025+
}{
1026+
{
1027+
name: "both nil",
1028+
routeLevel: nil,
1029+
backendLevel: nil,
1030+
expected: nil,
1031+
},
1032+
{
1033+
name: "route nil, backend has values",
1034+
routeLevel: nil,
1035+
backendLevel: &aigv1a1.HTTPHeaderMutation{
1036+
Set: []gwapiv1.HTTPHeader{{Name: "Backend-Header", Value: "backend-value"}},
1037+
Remove: []string{"Backend-Remove"},
1038+
},
1039+
expected: &aigv1a1.HTTPHeaderMutation{
1040+
Set: []gwapiv1.HTTPHeader{{Name: "Backend-Header", Value: "backend-value"}},
1041+
Remove: []string{"Backend-Remove"},
1042+
},
1043+
},
1044+
{
1045+
name: "route has values, backend nil",
1046+
routeLevel: &aigv1a1.HTTPHeaderMutation{
1047+
Set: []gwapiv1.HTTPHeader{{Name: "Route-Header", Value: "route-value"}},
1048+
Remove: []string{"Route-Remove"},
1049+
},
1050+
backendLevel: nil,
1051+
expected: &aigv1a1.HTTPHeaderMutation{
1052+
Set: []gwapiv1.HTTPHeader{{Name: "Route-Header", Value: "route-value"}},
1053+
Remove: []string{"Route-Remove"},
1054+
},
1055+
},
1056+
{
1057+
name: "no conflicts - different headers",
1058+
routeLevel: &aigv1a1.HTTPHeaderMutation{
1059+
Set: []gwapiv1.HTTPHeader{{Name: "Route-Header", Value: "route-value"}},
1060+
Remove: []string{"Route-Remove"},
1061+
},
1062+
backendLevel: &aigv1a1.HTTPHeaderMutation{
1063+
Set: []gwapiv1.HTTPHeader{{Name: "Backend-Header", Value: "backend-value"}},
1064+
Remove: []string{"Backend-Remove"},
1065+
},
1066+
expected: &aigv1a1.HTTPHeaderMutation{
1067+
Set: []gwapiv1.HTTPHeader{
1068+
{Name: "Backend-Header", Value: "backend-value"},
1069+
{Name: "Route-Header", Value: "route-value"},
1070+
},
1071+
Remove: []string{"backend-remove", "route-remove"},
1072+
},
1073+
},
1074+
{
1075+
name: "route overrides backend for same header name",
1076+
routeLevel: &aigv1a1.HTTPHeaderMutation{
1077+
Set: []gwapiv1.HTTPHeader{{Name: "X-Custom", Value: "route-value"}},
1078+
},
1079+
backendLevel: &aigv1a1.HTTPHeaderMutation{
1080+
Set: []gwapiv1.HTTPHeader{{Name: "X-Custom", Value: "backend-value"}},
1081+
},
1082+
expected: &aigv1a1.HTTPHeaderMutation{
1083+
Set: []gwapiv1.HTTPHeader{{Name: "X-Custom", Value: "route-value"}},
1084+
},
1085+
},
1086+
{
1087+
name: "case insensitive header name conflicts",
1088+
routeLevel: &aigv1a1.HTTPHeaderMutation{
1089+
Set: []gwapiv1.HTTPHeader{{Name: "x-custom", Value: "route-value"}},
1090+
},
1091+
backendLevel: &aigv1a1.HTTPHeaderMutation{
1092+
Set: []gwapiv1.HTTPHeader{{Name: "X-CUSTOM", Value: "backend-value"}},
1093+
},
1094+
expected: &aigv1a1.HTTPHeaderMutation{
1095+
Set: []gwapiv1.HTTPHeader{{Name: "x-custom", Value: "route-value"}},
1096+
},
1097+
},
1098+
{
1099+
name: "remove operations are combined and deduplicated",
1100+
routeLevel: &aigv1a1.HTTPHeaderMutation{
1101+
Remove: []string{"X-Remove", "x-shared"},
1102+
},
1103+
backendLevel: &aigv1a1.HTTPHeaderMutation{
1104+
Remove: []string{"X-Backend-Remove", "X-SHARED"},
1105+
},
1106+
expected: &aigv1a1.HTTPHeaderMutation{
1107+
Remove: []string{"x-backend-remove", "x-remove", "x-shared"},
1108+
},
1109+
},
1110+
{
1111+
name: "complex merge scenario",
1112+
routeLevel: &aigv1a1.HTTPHeaderMutation{
1113+
Set: []gwapiv1.HTTPHeader{
1114+
{Name: "X-Route-Only", Value: "route-only"},
1115+
{Name: "X-Override", Value: "route-wins"},
1116+
},
1117+
Remove: []string{"X-Route-Remove", "x-shared-remove"},
1118+
},
1119+
backendLevel: &aigv1a1.HTTPHeaderMutation{
1120+
Set: []gwapiv1.HTTPHeader{
1121+
{Name: "X-Backend-Only", Value: "backend-only"},
1122+
{Name: "x-override", Value: "backend-loses"},
1123+
},
1124+
Remove: []string{"X-Backend-Remove", "X-SHARED-REMOVE"},
1125+
},
1126+
expected: &aigv1a1.HTTPHeaderMutation{
1127+
Set: []gwapiv1.HTTPHeader{
1128+
{Name: "X-Backend-Only", Value: "backend-only"},
1129+
{Name: "X-Override", Value: "route-wins"},
1130+
{Name: "X-Route-Only", Value: "route-only"},
1131+
},
1132+
Remove: []string{"x-backend-remove", "x-route-remove", "x-shared-remove"},
1133+
},
1134+
},
1135+
{
1136+
name: "empty mutations",
1137+
routeLevel: &aigv1a1.HTTPHeaderMutation{
1138+
Set: []gwapiv1.HTTPHeader{},
1139+
Remove: []string{},
1140+
},
1141+
backendLevel: &aigv1a1.HTTPHeaderMutation{
1142+
Set: []gwapiv1.HTTPHeader{},
1143+
Remove: []string{},
1144+
},
1145+
expected: &aigv1a1.HTTPHeaderMutation{
1146+
Set: nil,
1147+
Remove: nil,
1148+
},
1149+
},
1150+
}
1151+
1152+
for _, tt := range tests {
1153+
t.Run(tt.name, func(t *testing.T) {
1154+
result := mergeHeaderMutations(tt.routeLevel, tt.backendLevel)
1155+
1156+
if tt.expected == nil {
1157+
require.Nil(t, result)
1158+
return
1159+
}
1160+
1161+
require.NotNil(t, result)
1162+
1163+
if d := cmp.Diff(tt.expected, result, cmpopts.SortSlices(func(a, b gwapiv1.HTTPHeader) bool {
1164+
return a.Name < b.Name
1165+
}), cmpopts.SortSlices(func(a, b string) bool {
1166+
return a < b
1167+
})); d != "" {
1168+
t.Errorf("mergeHeaderMutations() mismatch (-expected +got):\n%s", d)
1169+
}
1170+
})
1171+
}
1172+
}

‎manifests/charts/ai-gateway-crds-helm/templates/aigateway.envoyproxy.io_aigatewayroutes.yaml‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -464,6 +464,88 @@ spec:
464464
maxLength: 253
465465
pattern: ^$|^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$
466466
type: string
467+
headerMutation:
468+
description: |-
469+
HeaderMutation defines the request header mutation to be applied to this backend.
470+
When both route-level and backend-level HeaderMutation are defined,
471+
route-level takes precedence over backend-level for conflicting operations.
472+
This field is ignored when referencing InferencePool resources.
473+
properties:
474+
remove:
475+
description: |-
476+
Remove the given header(s) from the HTTP request before the action. The
477+
value of Remove is a list of HTTP header names. Note that the header
478+
names are case-insensitive (see
479+
https://datatracker.ietf.org/doc/html/rfc2616#section-4.2).
480+
481+
Input:
482+
GET /foo HTTP/1.1
483+
my-header1: foo
484+
my-header2: bar
485+
my-header3: baz
486+
487+
Config:
488+
remove: ["my-header1", "my-header3"]
489+
490+
Output:
491+
GET /foo HTTP/1.1
492+
my-header2: bar
493+
items:
494+
type: string
495+
maxItems: 16
496+
type: array
497+
x-kubernetes-list-type: set
498+
set:
499+
description: |-
500+
Set overwrites/adds the request with the given header (name, value)
501+
before the action.
502+
503+
Input:
504+
GET /foo HTTP/1.1
505+
my-header: foo
506+
507+
Config:
508+
set:
509+
- name: "my-header"
510+
value: "bar"
511+
512+
Output:
513+
GET /foo HTTP/1.1
514+
my-header: bar
515+
items:
516+
description: HTTPHeader represents an HTTP Header
517+
name and value as defined by RFC 7230.
518+
properties:
519+
name:
520+
description: |-
521+
Name is the name of the HTTP Header to be matched. Name matching MUST be
522+
case-insensitive. (See https://tools.ietf.org/html/rfc7230#section-3.2).
523+
524+
If multiple entries specify equivalent header names, the first entry with
525+
an equivalent name MUST be considered for a match. Subsequent entries
526+
with an equivalent header name MUST be ignored. Due to the
527+
case-insensitivity of header names, "foo" and "Foo" are considered
528+
equivalent.
529+
maxLength: 256
530+
minLength: 1
531+
pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$
532+
type: string
533+
value:
534+
description: Value is the value of HTTP Header
535+
to be matched.
536+
maxLength: 4096
537+
minLength: 1
538+
type: string
539+
required:
540+
- name
541+
- value
542+
type: object
543+
maxItems: 16
544+
type: array
545+
x-kubernetes-list-map-keys:
546+
- name
547+
x-kubernetes-list-type: map
548+
type: object
467549
kind:
468550
description: |-
469551
Kind is the kind of the backend resource.

‎site/docs/api/api.mdx‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -569,6 +569,11 @@ It can reference either an AIServiceBackend or an InferencePool resource.
569569
type="string"
570570
required="false"
571571
description="Name of the model in the backend. If provided this will override the name provided in the request.<br />This field is ignored when referencing InferencePool resources."
572+
/><ApiField
573+
name="headerMutation"
574+
type="[HTTPHeaderMutation](#httpheadermutation)"
575+
required="false"
576+
description="HeaderMutation defines the request header mutation to be applied to this backend.<br />When both route-level and backend-level HeaderMutation are defined,<br />route-level takes precedence over backend-level for conflicting operations.<br />This field is ignored when referencing InferencePool resources."
572577
/><ApiField
573578
name="weight"
574579
type="integer"
@@ -1315,6 +1320,7 @@ GCPCredentialsFile specifies the service account key json file to authenticate w
13151320

13161321

13171322
**Appears in:**
1323+
- [AIGatewayRouteRuleBackendRef](#aigatewayrouterulebackendref)
13181324
- [AIServiceBackendSpec](#aiservicebackendspec)
13191325

13201326
HTTPHeaderMutation defines the mutation of HTTP headers that will be applied to the request

0 commit comments

Comments
 (0)