Add Eagle3 config support for auxiliary hidden state layer IDs

Support configuring eagle_aux_hidden_state_layer_ids and inference_type
in the Eagle3 speculator configuration. This allows users to specify
which verifier layers should output auxiliary hidden states for the
drafter to consume during speculative decoding.

Signed-off-by: rahul-tuli <[email protected]>
Signed-off-by: Rahul Tuli <[email protected]>

Author: rahul-tuli

vllm/transformers_utils/configs/speculators/algos.py

@@ -30,3 +30,8 @@ def update_eagle3(config_dict: dict, vllm_config: dict) -> None:
     vllm_config["norm_before_residual"] = config_dict.get(
         "norm_before_residual", True)
     vllm_config["architectures"] = ["Eagle3LlamaForCausalLM"]
+    if config_dict.get("eagle_aux_hidden_state_layer_ids"):
+        vllm_config["eagle_aux_hidden_state_layer_ids"] = config_dict[
+            "eagle_aux_hidden_state_layer_ids"]
+    if config_dict.get("inference_type"):
+        vllm_config["inference_type"] = config_dict["inference_type"]