Skip to content

Yeyu/set block #480

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 53 commits into
base: main
Choose a base branch
from
Draft

Yeyu/set block #480

wants to merge 53 commits into from
+526 −524

Conversation

@yeyu-nvidia
Copy link
Contributor

@yeyu-nvidia yeyu-nvidia commented Oct 28, 2025

What does this PR do?

Type of change: ?

Overview: ?

Usage

# Add a code snippet demonstrating how to use this

Testing

Before your PR is "Ready for review"

  • Make sure you read and follow Contributor guidelines and your commits are signed.
  • Is this change backward compatible?: Yes/No
  • Did you write any new necessary tests?: Yes/No
  • Did you add or update any necessary documentation?: Yes/No
  • Did you update Changelog?: Yes/No

Additional Information

@yeyu-nvidia
refactor_multi_step_attn_mask_for_arbitrary_step
3e53bb8 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
minor
0b531c4 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
make new mask the same dtype and device as attn_mask
a123371 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
minor
c5a2e36 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
revert
c6731f8 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
integrate parallel draft to eagle auto regression
8aa0e12 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
ff04f43 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
fix pseudo spec generate for parallel draft
e5f2403 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
implement kv cache (inference_context) for eagle training
e032a52 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
allow groundtruth mismatch in AcceptanceRateValidation
5328b55 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
set fail_when_mismatch to False
be260f6 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
035d5ef 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
72b3aa0 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
460bc4e 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
72067dd 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
remove redundant index as now eagle_logits_ has a length of seq-len
890a718 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
minor edit based on coderabbit's suggestions
9195c18 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
make ttt_step configurable in forward
32919ff 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
fix the bug in pseudo_speculative_generate
4abfb52 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
change variable name to make it clear
cb9282e 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug: reduce kv cache size from ttt*parallel to ttt+parallel-1; in e…
b47928e 
…ach ttt step, only the non_parallel tokens from previous ttt are used as context

Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
gitnore type for precommit
b528ccc 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
consolidate acc printout
44e0eb1 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
fix the bug in pseudo_speculative_generate
13f218c 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
use embedding for mask tokens as hidden_states
8212265 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
aa328ed 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
remove mask tokens and use learnable embeddings and hidden_states ins…
d18dc94 
…tead

Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
50fec9a 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
fcc38fe 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
d3e1e0a 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug: move learnable parallel draft embedding and hidden_states afte…
753c8a1 
…r base model param freeze

Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
bfdd637 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
Merge branch 'main' into yeyu/debug_paralllel_draft
ab472f0
@yeyu-nvidia
only compute acc on tokens whose label is not IGNORE_TOKEN_ID
fc4551a 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
revert acc compute
a663526 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
pre debug
8b04e3e
@yeyu-nvidia
debug
dcb595a 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
a05e719 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
df6ccb0 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug: move eagle_embedding out of eagle_module so ckpt can be saved
b3bd50d 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
0c20495 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
6ffbf12 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug: when eagle_embedding is extra, its replica_id for pp should be 0
282c897 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
343255b 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
separate mask generation for ttt and diffusion
998800c 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
748cec3 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
c4dc6dc 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
1b36843 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
revert
dae905b 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
4b01c2a 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
d1374bb 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia yeyu-nvidia requested a review from a team as a code owner October 28, 2025 20:48
@yeyu-nvidia yeyu-nvidia requested a review from ChenhanYu October 28, 2025 20:48
@yeyu-nvidia yeyu-nvidia marked this pull request as draft October 28, 2025 20:48
@copy-pr-bot
Copy link

copy-pr-bot bot commented Oct 28, 2025

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@yeyu-nvidia
remove garbage
4d020a1 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
5e0b561 
Signed-off-by: Ye Yu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ChenhanYu ChenhanYu Awaiting requested review from ChenhanYu ChenhanYu is a code owner automatically assigned from NVIDIA/modelopt-torch-speculative-codeowners

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@yeyu-nvidia