[Debugging] Push PolicyWorker into Router and leverage Services #61
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
meta-cla
bot
added
the
CLA Signed
joecummings
reviewed
Aug 20, 2025
src/forge/actors/policy.py
Outdated
| log_stats=None, | ||
| ) | ||
|
|
||
| async def setupWorker(self, config, guided_decoding, num_samples): |
allenwang28
reviewed
Aug 20, 2025
| "MASTER_ADDR": str(get_loopback_ip()), | ||
| "MASTER_PORT": str(get_open_port()), | ||
| }, | ||
| ) |
There was a problem hiding this comment.
hmm, we might want to keep the worker mesh's procs distinctly separate...
The reason is because downstream, we will want something that can monitor the proc health and re-spawn it if necessary: https://github.com/meta-pytorch/forge/blob/main/src/forge/controller/service.py#L948-L977
I think having vLLM router handling this will be unnecessarily complex
There was a problem hiding this comment.
Will wait to tackled on top of the planned Service change
Jack-Khuu
force-pushed
the
refactor-policy-router
branch
from
4df9f8e to
d4c83d3
Compare
Jack-Khuu
changed the title
* make dataset configurable * add validation loop * update config * fix infinite loop, current_num_tokens * add single pass to the parameters * add single pass to param * fix the hang issue * minor: update error message * move batch_to_device to utils and add support to blockmask * remove comment * clean * fix validation backward thing for pp * remove self.model * add max_steps for validation to avoid hang * remove infinite
* Add reward interface, math reward, unit tests * move test files to rl folder * add thinking reward
* initial commit for replica * clean up * phase out service for service v2 * remove v2 * remove v2 from spawn * more minor cleanups * remove comment * remove comment * simplify and unify replica initialization * address comments * address comments * add capacity semaphore * f-strings * remove redundant health set --------- Co-authored-by: Allen Wang <[email protected]>
* Add reward interface, math reward, unit tests * refactor rewards: merge into one file * remove file accidentally had
… files (#69) * initial commit for replica * clean up * phase out service for service v2 * remove v2 * remove v2 from spawn * more minor cleanups * remove comment * remove comment * initial commit of ServiceEndpoint * tests work * simplify and unify replica initialization * stop the underlying service proc * split out components into their own files * address comments * address comments * add capacity semaphore * rebasing changes * fix test * logger changes * fix sess_id kwarg * makes _call its own implementation * docstring fix * add comment on serviceinterface --------- Co-authored-by: Allen Wang <[email protected]>
…e into refactor-policy-router
Jack-Khuu
changed the title
|
History on this PR is borked See #70 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refactors
PolicyRouter->PolicyPolicy->PolicyWorkerPushed Mesh and Worker set up into Policy (used to be Router)
Note: Will need follow up PR's to refactor to use @allenwang28 updated Service when ready