Skip to content

Create vllm policy app #75

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Aug 27, 2025
Merged

Create vllm policy app #75

merged 2 commits into from
Aug 27, 2025

Conversation

Jack-Khuu
Copy link
Contributor

@Jack-Khuu Jack-Khuu commented Aug 26, 2025 •
edited
Loading

Refactor the policy execution code into an apps/vllm

python apps/vllm/main.py --guided-decoding --num-samples 3

Starting background processing...
Requesting generation...

Generation Results:
================================================================================
Sample 1:
User: What is 3+5?
Assistant: Positive
--------------------------------------------------------------------------------
Sample 2:
User: What is 3+5?
Assistant: Negative
--------------------------------------------------------------------------------
Sample 3:
User: What is 3+5?
Assistant: Negative
--------------------------------------------------------------------------------

@Jack-Khuu Jack-Khuu requested a review from pbontrager August 26, 2025 23:21
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Aug 26, 2025
ebsmothers
ebsmothers approved these changes Aug 26, 2025
View reviewed changes
Copy link
Contributor

@ebsmothers ebsmothers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice. Super nit: would be nice to have something like this (I know argparse will kinda give you this, it's just nice to have something you can easily copy-paste right at the top of the file)

@Jack-Khuu Jack-Khuu merged commit bae98b2 into main Aug 27, 2025
6 of 7 checks passed
@Jack-Khuu Jack-Khuu deleted the vllm-app branch August 30, 2025 22:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ebsmothers ebsmothers ebsmothers approved these changes

@pbontrager pbontrager Awaiting requested review from pbontrager

@allenwang28 allenwang28 Awaiting requested review from allenwang28

@joecummings joecummings Awaiting requested review from joecummings

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Jack-Khuu @ebsmothers