Commit 949087f
committed
refactor: implement SOLID-compliant strategy architecture with interfaces and concrete classes
Replace enum-based strategy switching with proper Open/Closed Principle architecture.
This allows extending strategies without modifying existing code.
## New Architecture
### Adaptive Strategies
**New Interface**: IAdaptiveDistillationStrategy<T>
- Defines contract for adaptive temperature adjustment
- Methods: UpdatePerformance, ComputeAdaptiveTemperature, GetPerformance
**New Base Class**: AdaptiveDistillationStrategyBase<T>
- Extends DistillationStrategyBase<T, Vector<T>>
- Implements IAdaptiveDistillationStrategy<T>
- Provides shared logic: EMA performance tracking, temperature clamping, helper methods
- Abstract method: ComputeAdaptiveTemperature (strategy-specific logic)
**New Concrete Implementations**:
1. ConfidenceBasedAdaptiveStrategy<T>
- Adapts based on max probability (confidence)
- Low confidence → higher temperature (softer targets)
- High confidence → lower temperature (sharper targets)
- Best for: General-purpose, no labels needed
2. AccuracyBasedAdaptiveStrategy<T>
- Adapts based on prediction correctness
- Incorrect → higher temperature (help learn)
- Correct → lower temperature (reinforce)
- Best for: Supervised learning with labels
3. EntropyBasedAdaptiveStrategy<T>
- Adapts based on prediction uncertainty (entropy)
- High entropy → lower temperature (focus learning)
- Low entropy → higher temperature (explore)
- Best for: Holistic uncertainty measurement
### Curriculum Strategies
**New Interface**: ICurriculumDistillationStrategy<T>
- Defines contract for progressive difficulty adjustment
- Methods: UpdateProgress, SetSampleDifficulty, ShouldIncludeSample, ComputeCurriculumTemperature
**New Base Class**: CurriculumDistillationStrategyBase<T>
- Extends DistillationStrategyBase<T, Vector<T>>
- Implements ICurriculumDistillationStrategy<T>
- Provides shared logic: Progress tracking, difficulty management, temperature range
- Abstract methods: ShouldIncludeSample, ComputeCurriculumTemperature
**New Concrete Implementations**:
1. EasyToHardCurriculumStrategy<T>
- Progresses from easy to hard samples
- Temperature: High (soft) → Low (sharp)
- Sample filter: Include if difficulty ≤ progress
- Best for: Training from scratch
2. HardToEasyCurriculumStrategy<T>
- Progresses from hard to easy samples (inverted)
- Temperature: Low (sharp) → High (soft)
- Sample filter: Include if difficulty ≥ (1 - progress)
- Best for: Fine-tuning, transfer learning
## Deleted Files
- AdaptiveDistillationStrategy.cs (enum-based, replaced by 3 concrete classes)
- CurriculumDistillationStrategy.cs (enum-based, replaced by 2 concrete classes)
## Updated Documentation
- MIGRATION_GUIDE.md: Comprehensive guide with:
- Architecture diagrams
- Before/after code examples for all 5 strategies
- Custom strategy creation examples
- Interface reference
- Benefits of Open/Closed architecture
- Common migration issues and solutions
## Benefits
### Open/Closed Principle
**Before**: Adding new strategy required modifying enum + switch
**After**: Just create new class extending base - no existing code modified
### Testability
**Before**: Mock entire class, test specific enum branch
**After**: Test each strategy in isolation
### Composition
**Before**: Can't combine strategies easily
**After**: Compose through interfaces (hybrid strategies possible)
### Dependency Injection
**Before**: Tightly coupled to concrete enum
**After**: Inject through IAdaptiveDistillationStrategy<T> or ICurriculumDistillationStrategy<T>
## Migration Path
Replace enum-based construction with specific strategy class:
```csharp
// OLD:
new AdaptiveDistillationStrategy<double>(strategy: AdaptiveStrategy.ConfidenceBased)
// NEW:
new ConfidenceBasedAdaptiveStrategy<double>()
```
Resolves #408 - Implements production-ready SOLID architecture for distillation strategies1 parent 9c284a1 commit 949087f
File tree
12 files changed
+1848
-941
lines changed- src/KnowledgeDistillation
- Strategies
12 files changed
+1848
-941
lines changedLarge diffs are not rendered by default.
Lines changed: 139 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
0 commit comments