Skip to content

feat: Support gpt-oss class of models with flash attention 3 support #603

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ashokponkumar merged 12 commits into from
Sep 3, 2025
Merged

feat: Support gpt-oss class of models with flash attention 3 support #603

ashokponkumar merged 12 commits into from
Sep 3, 2025

Conversation

@dushyantbehl
Copy link
Collaborator

@dushyantbehl dushyantbehl commented Sep 1, 2025 •
edited
Loading

Description of the change

  1. Adds code changes required to support the new GPT-OSS models.
  2. Adds a new dockerfile which is based on nvcr for dev mode and supports flash attention 3.
  3. Updates most of the required package versions.
  4. Adds a new quantization class for Mxfp4 which dequantizes the model before training.

Related issue number

How to verify the PR

Was the PR tested

  • I have added >=1 unit test(s) for every new method I have added.
  • I have ensured all unit tests pass

@github-actions
Copy link

github-actions bot commented Sep 1, 2025

Thanks for making a pull request! 😃
One of the maintainers will review and advise on the next steps.

@github-actions github-actions bot added the feat label Sep 1, 2025
@dushyantbehl
Enable tuning of GPT OSS models
889b8f3 
enable cuda 12.8 in dockerfile and add triton kernels in pyproject

Signed-off-by: Dushyant Behl <[email protected]>
dushyantbehl
dushyantbehl commented Sep 2, 2025
View reviewed changes
@dushyantbehl dushyantbehl changed the title feat: Support gpt-oss class of models with flash attention support feat: Support gpt-oss class of models with flash attention 3 support Sep 2, 2025
ashokponkumar
ashokponkumar reviewed Sep 2, 2025
View reviewed changes
Copy link
Collaborator

@ashokponkumar ashokponkumar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Some suggestions.

@dushyantbehl
Fix linter complaining about quantization config.
42ae4b6 
Signed-off-by: Harikrishnan Balagopal <[email protected]>
Signed-off-by: Dushyant Behl <[email protected]>
HarikrishnanBalagopal
HarikrishnanBalagopal reviewed Sep 2, 2025
View reviewed changes
ashokponkumar
ashokponkumar previously approved these changes Sep 2, 2025
View reviewed changes
@dushyantbehl
enable mamba in nvcr image
130dcc8 
Signed-off-by: Dushyant Behl <[email protected]>
@ashokponkumar ashokponkumar merged commit 7e261d2 into main Sep 3, 2025
12 checks passed
@dushyantbehl dushyantbehl deleted the gpt-oss branch November 7, 2025 03:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ashokponkumar ashokponkumar ashokponkumar approved these changes

@anhuong anhuong Awaiting requested review from anhuong anhuong is a code owner

@aluu317 aluu317 Awaiting requested review from aluu317 aluu317 is a code owner

@fabianlim fabianlim Awaiting requested review from fabianlim fabianlim is a code owner

@kmehant kmehant Awaiting requested review from kmehant kmehant is a code owner

+1 more reviewer

@HarikrishnanBalagopal HarikrishnanBalagopal HarikrishnanBalagopal left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

feat

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@dushyantbehl @ashokponkumar @HarikrishnanBalagopal