File tree Expand file tree Collapse file tree 1 file changed +4
-0
lines changed Expand file tree Collapse file tree 1 file changed +4
-0
lines changed Original file line number Diff line number Diff line change @@ -203,6 +203,7 @@ an [EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency)](https
203
203
"model": "yuhuili/EAGLE-LLaMA3-Instruct-8B",
204
204
"draft_tensor_parallel_size": 1,
205
205
"num_speculative_tokens": 2,
206
+ "method": "eagle",
206
207
},
207
208
)
208
209
@@ -231,6 +232,9 @@ A few important things to consider when using the EAGLE based draft models:
231
232
reported in the reference implementation [ here] ( https://github.com/SafeAILab/EAGLE ) . This issue is under
232
233
investigation and tracked here: < gh-issue:9565 > .
233
234
235
+ 4 . When using EAGLE-3 based draft model, option "method" must be set to "eagle3".
236
+ That is, to specify ` "method": "eagle3" ` in ` speculative_config ` .
237
+
234
238
A variety of EAGLE draft models are available on the Hugging Face hub:
235
239
236
240
| Base Model | EAGLE on Hugging Face | # EAGLE Parameters |
You can’t perform that action at this time.
0 commit comments