Skip to content

[Tests] Add tests for "quantization_w4a4_fp4" examples #1715

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

dbarbuzzi
Copy link
Collaborator

SUMMARY:
Adds tests for the example scripts in the examples/quantization_w4a4_fp4 folder.

TEST PLAN:
Internal examples test run with just the new tests: https://github.com/neuralmagic/llm-compressor-testing/actions/runs/16831501387/job/47709542536

The failure appears to be a legitimate issue, not a test issue, and has been reported.

derekk-nm and others added 3 commits August 5, 2025 14:56
Signed-off-by: Derek Kozikowski <[email protected]>
Signed-off-by: Domenic Barbuzzi <[email protected]>
Copy link

github-actions bot commented Aug 8, 2025

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @dbarbuzzi, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

I've implemented new tests to ensure the example scripts located in the examples/quantization_w4a4_fp4 directory are functioning as expected. These tests validate that the scripts run without errors and produce output in the correct format, thereby improving the reliability and maintainability of these examples.

Highlights

  • New Test File: I've added a new test file, tests/examples/test_quantization_w4a4_fp4.py, specifically designed to validate the example scripts in the examples/quantization_w4a4_fp4 directory.
  • Parameterized Example Testing: The new tests utilize pytest.mark.parametrize to run llama3_example.py, llama4_example.py, and qwen_30b_a3b.py examples.
  • Output Validation: Each test verifies that the example script executes successfully, generates the expected *-NVFP4 output directory, and that the config.json within this directory correctly specifies the nvfp4-pack-quantized format.
  • GPU Requirement: These tests are configured to run only when at least one GPU is available, ensuring they are executed in the appropriate environment.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds tests for the quantization_w4a4_fp4 examples. The new test file tests/examples/test_quantization_w4a4_fp4.py correctly parameterizes tests for three example scripts and verifies their output. My review includes a couple of suggestions to improve the robustness and clarity of the test code, such as making a file search more specific and simplifying the use of pytest.mark.parametrize.

@dbarbuzzi dbarbuzzi force-pushed the derekk/nvfp4-example-tests branch from df7461c to 7a82b9f Compare August 8, 2025 23:25
Signed-off-by: Domenic Barbuzzi <[email protected]>
Signed-off-by: Domenic Barbuzzi <[email protected]>
@dbarbuzzi dbarbuzzi force-pushed the derekk/nvfp4-example-tests branch from 7a82b9f to bd3cfaf Compare August 8, 2025 23:34
@dbarbuzzi dbarbuzzi added the ready When a PR is ready for review label Aug 8, 2025
@dbarbuzzi dbarbuzzi marked this pull request as ready for review August 8, 2025 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready When a PR is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants