speculative decoding in gemma3

[tonyay163]

On 0.20.0, I got an error trying to use speculative decoding because of an unsupported param. But even after plumbing it in, it's still giving me an error:

terminate called after throwing an instance of 'tensorrt_llm::common::TllmException'
  what():  [TensorRT-LLM][ERROR] Assertion failed: No available XQA kernels are found for speculative decoding mode. (/home/jenkins/agent/workspace/LLM/main/L0_Test-x86_64/tensorrt_llm/cpp/tensorrt_llm/common/attentionOp.cpp:2027)

How can the kernels be added for gemma3 models?

[karljang]

@tonyay163 ,
Apologies for the very delayed response.
Since it was reported on an relatively old version, could you try the latest version to to see if the problem still persists?
And, to help us investigate more efficiently, be sure to include reproduction steps such as the command line and any other relevant information.
Thank you!

[github-actions]

Issue has not received an update in over 14 days. Adding stale label.

[github-actions]

This issue was closed because it has been 14 days without activity since it has been marked as stale.