Skip to content

[0.34.0-dlc][python] Support session based sticky routing in async mode#2886

Merged
xyang16 merged 1 commit intodeepjavalibrary:0.34.0-dlcfrom
xyang16:0.34.0-dlc
Sep 29, 2025
Merged

[0.34.0-dlc][python] Support session based sticky routing in async mode#2886
xyang16 merged 1 commit intodeepjavalibrary:0.34.0-dlcfrom
xyang16:0.34.0-dlc

Conversation

@xyang16
Copy link
Copy Markdown
Contributor

@xyang16 xyang16 commented Sep 25, 2025

Description

Brief description of what this PR is about

This PR add support for session based sticky routing, all requests from the same session are routed to the same instance.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Checklist:

  • Please add the link of Integration Tests Executor run with related tests.
  • Have you manually built the docker image and verify the change?
  • Have you run related tests? Check how to set up the test environment here; One example would be pytest tests.py -k "TestCorrectnessLmiDist" -m "lmi_dist"
  • Have you added tests that prove your fix is effective or that this feature works?
  • Has code been commented, particularly in hard-to-understand areas?
  • Have you made corresponding changes to the documentation?

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

  • Test A
    Logs for Test A

  • Test B
    Logs for Test B

@Lokiiiiii
Copy link
Copy Markdown
Member

We also need to add documentation and examples for this new feature. Let's address that in a follow up PR.

@xyang16 xyang16 force-pushed the 0.34.0-dlc branch 9 times, most recently from ec4fa1d to 6aae33e Compare September 26, 2025 00:55
@xyang16 xyang16 force-pushed the 0.34.0-dlc branch 5 times, most recently from 210ef95 to 4609dfa Compare September 28, 2025 00:41
@xyang16
Copy link
Copy Markdown
Contributor Author

xyang16 commented Sep 28, 2025

We also need to add documentation and examples for this new feature. Let's address that in a follow up PR.

Added documentation stateful_sessions.md in this PR and sample notebook deepjavalibrary/djl-demo#573

@xyang16 xyang16 force-pushed the 0.34.0-dlc branch 8 times, most recently from 567407a to a8673a5 Compare September 29, 2025 06:50
@xyang16 xyang16 merged commit 18ac531 into deepjavalibrary:0.34.0-dlc Sep 29, 2025
5 of 6 checks passed
@xyang16 xyang16 deleted the 0.34.0-dlc branch September 29, 2025 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants