Skip to content

Add Pod Port or DP Rank Header #230

@danehans

Description

@danehans

Currently, sim adds the "x-inference-pod" header to responses with the pod name as the value. Since sim supports Data Parallelism (DP), it would be beneficial to included a header in responses that indicates the port number that the request was received on or the rank number that served the request. This would allow tests to check for this header to assert DP.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions