SUBTASK 1 P-eagle model definition and Configuration #328
Description
Originally a task in Add P-eagle support in training.
Task
Create PEagleDraftModel and PEagleSpeculatorConfig based on Eagle3 model definition and configuration
- P-EAGLE model registration with SpeculatorModel registry
- P-EAGLE model definition with mask_hidden parameter (shape:
$[1, 1, 3\times hidden size]$ ) - Configuration class supporting para_depths, down_sample_ratio, down_sample_ratio_min, ptd_token_id parameters
- Reuse of existing EAGLE-3 decoder layers and architecture