Skip to content

Conversation

@cjluo-nv
Copy link
Collaborator

@cjluo-nv cjluo-nv commented Dec 4, 2025

Add GPU cache clearing for 3D weight tensors to prevent OOM issues.

What does this PR do?

Type of change: ? Bug fix

Testing

python hf_ptq.py --dataset cnn_dailymail --pyt_ckpt_path Llama-4-Maverick-17B-128E-Instruct --qformat nvfp4 --kv_cache_qformat fp8 --batch_size 1 --calib_size 32 --use_seq_device_map --gpu_max_mem_percentage 0.6 --trust_remote_code

Before your PR is "Ready for review"

  • Make sure you read and follow Contributor guidelines and your commits are signed.
  • Is this change backward compatible?: Yes/No
  • Did you write any new necessary tests?: Yes/No
  • Did you add or update any necessary documentation?: Yes/No
  • Did you update Changelog?: Yes/No

Additional Information

Add GPU cache clearing for 3D weight tensors to prevent OOM issues.

Signed-off-by: Chenjie Luo <[email protected]>
@cjluo-nv cjluo-nv requested a review from a team as a code owner December 4, 2025 22:53
@codecov
Copy link

codecov bot commented Dec 4, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.68%. Comparing base (097037d) to head (2031d8e).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #649   +/-   ##
=======================================
  Coverage   74.68%   74.68%           
=======================================
  Files         183      183           
  Lines       18552    18552           
=======================================
  Hits        13856    13856           
  Misses       4696     4696           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants