Skip to content

Conversation

@christ-tt
Copy link
Collaborator

@christ-tt christ-tt commented Dec 25, 2025

  • Add a check that there's at least one ready pipeline, defined by all nodes in it are is_active, before dispatching requests.
  • Decoupled router and scheduler; so we don't conditioning on strategy; and we let router handles the routing related tasks.

TODO

  • Maybe rename node.is_active , as the active here means even after we have assigned layers / pipeline formation, node will need time for parameters loading, engine setup, cuda graph init, etc. The naming here may causing confusion for how we define active and standby for our nodes.

@christ-tt christ-tt changed the title feat(scheduler): Router Ready Check feat(scheduler): Decoupled Scheduler with Router and Adding Router Readiness Check Dec 25, 2025
@christ-tt christ-tt marked this pull request as ready for review December 25, 2025 02:24
@christ-tt christ-tt requested review from a team and gufengc December 25, 2025 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants