Skip to content

Commit 6248ce2

Browse files
hsliuustc0106Dilute-l
authored andcommitted
[Doc] Added unmentioned required option "method" in the usage of EAGLE-3 based models (vllm-project#21737)
Signed-off-by: Dilute-l <[email protected]> Co-authored-by: Dilute-l <[email protected]>
1 parent 4fba7b1 commit 6248ce2

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

docs/features/spec_decode.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,7 @@ an [EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency)](https
203203
"model": "yuhuili/EAGLE-LLaMA3-Instruct-8B",
204204
"draft_tensor_parallel_size": 1,
205205
"num_speculative_tokens": 2,
206+
"method": "eagle",
206207
},
207208
)
208209

@@ -231,6 +232,9 @@ A few important things to consider when using the EAGLE based draft models:
231232
reported in the reference implementation [here](https://github.com/SafeAILab/EAGLE). This issue is under
232233
investigation and tracked here: <gh-issue:9565>.
233234

235+
4. When using EAGLE-3 based draft model, option "method" must be set to "eagle3".
236+
That is, to specify `"method": "eagle3"` in `speculative_config`.
237+
234238
A variety of EAGLE draft models are available on the Hugging Face hub:
235239

236240
| Base Model | EAGLE on Hugging Face | # EAGLE Parameters |

0 commit comments

Comments
 (0)