Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
3e53bb8
refactor_multi_step_attn_mask_for_arbitrary_step
yeyu-nvidia Sep 19, 2025
0b531c4
minor
yeyu-nvidia Sep 19, 2025
a123371
make new mask the same dtype and device as attn_mask
yeyu-nvidia Sep 19, 2025
c5a2e36
minor
yeyu-nvidia Sep 22, 2025
c6731f8
revert
yeyu-nvidia Sep 22, 2025
8aa0e12
integrate parallel draft to eagle auto regression
yeyu-nvidia Sep 23, 2025
ff04f43
debug
yeyu-nvidia Sep 23, 2025
e5f2403
fix pseudo spec generate for parallel draft
yeyu-nvidia Sep 23, 2025
e032a52
implement kv cache (inference_context) for eagle training
yeyu-nvidia Sep 24, 2025
5328b55
allow groundtruth mismatch in AcceptanceRateValidation
yeyu-nvidia Sep 25, 2025
be260f6
set fail_when_mismatch to False
yeyu-nvidia Sep 25, 2025
035d5ef
debug
yeyu-nvidia Sep 26, 2025
72b3aa0
debug
yeyu-nvidia Sep 26, 2025
460bc4e
debug
yeyu-nvidia Sep 26, 2025
72067dd
debug
yeyu-nvidia Sep 26, 2025
890a718
remove redundant index as now eagle_logits_ has a length of seq-len
yeyu-nvidia Sep 29, 2025
9195c18
minor edit based on coderabbit's suggestions
yeyu-nvidia Sep 29, 2025
32919ff
make ttt_step configurable in forward
yeyu-nvidia Oct 1, 2025
4abfb52
fix the bug in pseudo_speculative_generate
yeyu-nvidia Oct 1, 2025
cb9282e
change variable name to make it clear
yeyu-nvidia Oct 1, 2025
b47928e
debug: reduce kv cache size from ttt*parallel to ttt+parallel-1; in e…
yeyu-nvidia Oct 1, 2025
b528ccc
gitnore type for precommit
yeyu-nvidia Oct 1, 2025
44e0eb1
consolidate acc printout
yeyu-nvidia Oct 1, 2025
13f218c
fix the bug in pseudo_speculative_generate
yeyu-nvidia Oct 2, 2025
8212265
use embedding for mask tokens as hidden_states
yeyu-nvidia Oct 3, 2025
aa328ed
debug
yeyu-nvidia Oct 3, 2025
d18dc94
remove mask tokens and use learnable embeddings and hidden_states ins…
yeyu-nvidia Oct 6, 2025
50fec9a
debug
yeyu-nvidia Oct 6, 2025
fcc38fe
debug
yeyu-nvidia Oct 6, 2025
d3e1e0a
debug
yeyu-nvidia Oct 6, 2025
753c8a1
debug: move learnable parallel draft embedding and hidden_states afte…
yeyu-nvidia Oct 9, 2025
bfdd637
debug
yeyu-nvidia Oct 10, 2025
ab472f0
Merge branch 'main' into yeyu/debug_paralllel_draft
yeyu-nvidia Oct 13, 2025
fc4551a
only compute acc on tokens whose label is not IGNORE_TOKEN_ID
yeyu-nvidia Oct 17, 2025
a663526
revert acc compute
yeyu-nvidia Oct 17, 2025
8b04e3e
pre debug
yeyu-nvidia Oct 28, 2025
dcb595a
debug
yeyu-nvidia Oct 28, 2025
a05e719
debug
yeyu-nvidia Oct 28, 2025
df6ccb0
debug
yeyu-nvidia Oct 28, 2025
b3bd50d
debug: move eagle_embedding out of eagle_module so ckpt can be saved
yeyu-nvidia Oct 28, 2025
0c20495
debug
yeyu-nvidia Oct 28, 2025
6ffbf12
debug
yeyu-nvidia Oct 28, 2025
282c897
debug: when eagle_embedding is extra, its replica_id for pp should be 0
yeyu-nvidia Oct 28, 2025
343255b
debug
yeyu-nvidia Oct 28, 2025
998800c
separate mask generation for ttt and diffusion
yeyu-nvidia Oct 28, 2025
748cec3
debug
yeyu-nvidia Oct 28, 2025
c4dc6dc
debug
yeyu-nvidia Oct 28, 2025
1b36843
debug
yeyu-nvidia Oct 28, 2025
dae905b
revert
yeyu-nvidia Oct 28, 2025
4b01c2a
debug
yeyu-nvidia Oct 28, 2025
d1374bb
debug
yeyu-nvidia Oct 28, 2025
4d020a1
remove garbage
yeyu-nvidia Oct 28, 2025
5e0b561
debug
yeyu-nvidia Oct 28, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions modelopt/torch/speculative/eagle/eagle_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,5 +47,4 @@ def modify(
self.eagle_loss_decay_factor = eagle_loss_decay_factor

if eagle_architecture_config.get("parallel_draft_step", 1) > 1:
for i in range(eagle_architecture_config.get("parallel_draft_step") - 1):
self.register_buffer(f"mask_token_{i}", torch.tensor(-1))
self.register_buffer("mask_token", torch.tensor(-1))
Loading
Loading