Skip to content

[InferenceModel update] Created fairness/identity header flag #1245

@kfswain

Description

@kfswain

This is related to: https://github.com/kubernetes-sigs/gateway-api-inference-extension/tree/main/docs/proposals/1199-inferencemodel-api-evolution#structure-change

Specifically: The EPP will expose a flag to define the header key (default: x-gateway-inference-fairness-id) that will be used in tracking Request Usage (which will act as the identifier for simple fairness implementation)

This will flag into the Flow Controller to be consumed there.

Metadata

Metadata

Assignees

Labels

triage/acceptedIndicates an issue or PR is ready to be actively worked on.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions