Skip to content

feat(speculative): add DFlash validation metrics#3072

Open
khazic wants to merge 2 commits into
NVIDIA-NeMo:mainfrom
khazic:khazic/feat/dflash-validation-metrics
Open

feat(speculative): add DFlash validation metrics#3072
khazic wants to merge 2 commits into
NVIDIA-NeMo:mainfrom
khazic:khazic/feat/dflash-validation-metrics

Conversation

@khazic

@khazic khazic commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

What does this PR do ?

Unify DFlash, Domino, and JetSpec validation metrics across distributed ranks and add optional W&B logging for training and validation.

Changelog

  • Add token-weighted validation loss and accuracy using globally reduced additive statistics.
  • Add block-weighted acceptance length for DFlash, Domino, and JetSpec.
  • Add Domino final-head and base-head validation diagnostics.
  • Add optional W&B configuration and train/val metric logging.
  • Document the metric keys and example W&B configuration.
  • Add unit coverage for weighted aggregation, short batches, W&B logging, and rank-symmetric multi-process reduction.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?

Additional Information

Signed-off-by: khazic <khazzz1c@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 15, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Signed-off-by: khazic <khazzz1c@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants