Introducing CFG (classifier free guidance)#6

Open

vklimkov-nvidia wants to merge 9 commits intovklimkov/modeldeffrom

Commits on Jul 4, 2025

[DRAFT] Initial implementation of CFG (classifier free guidance) for LLMs
vklimkov-nvidia
committed
cpp/tensorrt_llm/batch_manager/handleContextLogits.cpp: apply CFG during context phase too (first frame of logits)
vklimkov-nvidia
committed
CFG in T5TTS: introduce acoustic token which expands to zeros for unconditional generation
vklimkov-nvidia
committed
runtimeBuffers: remove print, replace hardcode with vocab size from config
vklimkov-nvidia
committed
switch to using cublas to compute logits after cfg
vklimkov-nvidia
committed
Fixes to freeing cache and slots for case when llm request contains multiple sequences
vklimkov-nvidia
committed
Adjust gptManagerBenchmark to work with decoder models with cross attention, where encoder features are provided explicitely
vklimkov-nvidia
committed
gptManagerBenchmark.cpp: pre-copy encoder features to GPU in benchmark
vklimkov-nvidia
committed

Commits on Jul 8, 2025

Enable latest grpo checkpoint, apply attention prior in leader thread
vklimkov-nvidia
committed