Fix issues of attention.core_attention.softmax_offset is None #330
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WalkthroughReplaces a hasattr check with a getattr(..., None) is not None guard when reading Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Importer as MegatronImporter
participant Attn as attention.core_attention
participant Rules as self.rules["softmax_offset"]
Importer->>Attn: Read softmax_offset
alt softmax_offset exists AND != None
Importer->>Rules: Invoke with softmax_offset
Rules-->>Importer: Return transformed offset
else softmax_offset missing or None
Note over Importer: Skip invoking rule
end
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
Tip 👮 Agentic pre-merge checks are now available in preview!Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.
Please see the documentation for more information. Example: reviews:
pre_merge_checks:
custom_checks:
- name: "Undocumented Breaking Changes"
mode: "warning"
instructions: |
Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).Please share your feedback with us on this Discord post. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
yueshen2016
changed the title
ChenhanYu
reviewed
Sep 17, 2025
| self.rules["linear_qkv"](attention.linear_qkv, layer_id) | ||
| self.rules["linear_proj"](attention.linear_proj, layer_id) | ||
| if hasattr(attention.core_attention, "softmax_offset"): | ||
| if ( |
There was a problem hiding this comment.
if getattr(attention.core_attention, "softmax_offset", None) is not None:
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
modelopt/torch/export/plugins/megatron_importer.py (1)
515-521: Harden the guard to avoid AttributeError whencore_attentionis absent.Accessing
attention.core_attentioninsidehasattr(...)can still raise ifcore_attentionitself is missing or None on some attention variants. Usegetattrchaining and reuse the computedoffset.- if ( - hasattr(attention.core_attention, "softmax_offset") - and attention.core_attention.softmax_offset is not None - ): - self.rules["softmax_offset"]( - attention.core_attention.softmax_offset, layer_id - ) + core_attn = getattr(attention, "core_attention", None) + offset = getattr(core_attn, "softmax_offset", None) if core_attn is not None else None + if offset is not None: + self.rules["softmax_offset"](offset, layer_id)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
modelopt/torch/export/plugins/megatron_importer.py(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: linux
- GitHub Check: code-quality
- GitHub Check: build-docs
…atron importer Signed-off-by: Yue <[email protected]>
yueshen2016
force-pushed
the
yue/fix-megatron-importer
branch
from
da6dc1b to
8e8e77d
Compare
ChenhanYu
approved these changes
Sep 17, 2025
|
/ok to test 8e8e77d |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #330 +/- ##
==========================================
- Coverage 73.82% 73.82% -0.01%
==========================================
Files 172 172
Lines 17438 17438
==========================================
- Hits 12874 12873 -1
- Misses 4564 4565 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
yeyu-nvidia
pushed a commit
that referenced
this pull request
Sep 18, 2025
Signed-off-by: Yue <[email protected]> Signed-off-by: Ye Yu <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Type of change: ?
Bug fix
Overview: ?
Fix issues of attention.core_attention.softmax_offset is None for megatron import
Usage
# Add a code snippet demonstrating how to use thisTesting
Before your PR is "Ready for review"
Additional Information
Summary by CodeRabbit