fix: restore legacy X-Ray-Serve-Request-Id header for sticky routing#224
Conversation
The ModelScope gateway (www.modelscope.cn/twinkle) requires the `X-Ray-Serve-Request-Id` header name for sticky session routing. Commit dff3143 replaced it with `x-request-id`, which Ray Serve 2.55+ reads natively but the gateway does not forward correctly. Add `H_REQUEST_ID_LEGACY` alongside `H_REQUEST_ID` so both header names are emitted, fixing `BadRequestError: Missing X-Ray-Serve-Request-Id header` when connecting through the ModelScope gateway.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a legacy request ID header (X-Ray-Serve-Request-Id) to the client routing headers. The feedback suggests ensuring server-side compatibility by updating ray_serve_patch.py to recognize this legacy header and modifying the gateway proxy to fall back to it if the primary request ID is missing.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
There was a problem hiding this comment.
Pull request overview
This PR restores compatibility with the ModelScope Twinkle gateway’s sticky-session routing by sending Ray Serve’s legacy request-id header name (X-Ray-Serve-Request-Id) alongside the newer x-request-id.
Changes:
- Add
H_REQUEST_ID_LEGACY = 'X-Ray-Serve-Request-Id'header constant. - Include the legacy request-id header in
_ROUTING_HEADERSso it is emitted bybuild_routing_headers().
- `gateway/proxy.py`: fallback to `H_REQUEST_ID_LEGACY` when extracting request_id from incoming headers, so sticky routing works even if the gateway only forwards `X-Ray-Serve-Request-Id`. - `ray_serve_patch.py`: also match lowercased `x-ray-serve-request-id` when populating Ray Serve's request context, mirroring the client-side dual-header approach.
Summary
H_REQUEST_ID_LEGACY = 'X-Ray-Serve-Request-Id'to routing headerswww.modelscope.cn/twinkle) requires this legacy header name for sticky session routing; the newerx-request-id(Ray Serve 2.55+) is not forwarded correctlyBadRequestError: Missing X-Ray-Serve-Request-Id header, required for sticky sessionwhen connecting tinker client through the gatewayTest plan
ServiceClientcreates successfully againstwww.modelscope.cn/twinkleshort_math_grpo.py) runs end-to-end without the 400 error