Release v0.2.2: Selective layer width pruning & optimized hybrid importance

Author: peremartra

.DS_Store

@@ -1,3 +1,82 @@
+## [0.2.2] - 2025-11-26
+
+### 🎉 New Features
+
+#### Selective Layer Width Pruning
+- **layer_indices for MLP_GLU**: Extended `layer_indices` parameter to support selective neuron pruning in specific layers
+- **Contextual Usage**: For DEPTH pruning, specifies layers to remove; for MLP_GLU, specifies layers to prune
+- **Preservation Strategy**: Allows preserving critical layers (e.g., first/last) at full capacity while pruning others
+- **Full Compatibility**: Works seamlessly with all MLP_GLU features (expansion_rate, expansion_divisor, dataloader, all methods)
+
+#### Simplified Hybrid Importance Calculation
+- **Optimized MAW Hybrid**: Simplified `compute_neuron_pair_importance_maw_hybrid()` to use simple MAW for gate_proj and up_proj
+- **Focused Complexity**: Maintains complex activation-weighted calculation only for down_proj where it has most impact
+- **Better Performance**: Faster execution by reducing unnecessary calculations
+- **Consistent Formula**: Uses same MAW method (max + |min|) as static pruning for gate/up components
+
+### ✨ Enhancements
+
+- **Extended API**: `layer_indices` parameter now works for both DEPTH and MLP_GLU pruning types
+- **Smart Validation**: Comprehensive error checking for layer indices (range, duplicates, empty lists, types)
+- **Enhanced Statistics**: `get_pruning_statistics()` now reports selective pruning info (pruned_layers, total_layers)
+- **Selective Calibration**: Hooks only registered on selected layers when using data-driven pruning with layer_indices
+- **CLI Support**: Updated `--layer-indices` help text to mention both pruning types
+- **Backward Compatible**: `layer_indices=None` maintains default behavior (prunes all layers)
+
+### 🔧 Technical Details
+
+#### Modified Functions
+- `prune_model()`: Updated docstring and passes `layer_indices` to `prune_model_mlp_glu()`
+- `prune_model_mlp_glu()`: Added `layer_indices` parameter with full validation and filtering logic
+- `setup_mlp_hooks_for_importance()`: Now accepts `layer_indices` to register hooks only on selected layers
+- `compute_neuron_pair_importance_maw_hybrid()`: Simplified to use MAW for gate/up, complex calculation only for down
+- `get_pruning_statistics()`: Detects and reports selective pruning information
+- CLI `commands.py`: Removed restriction blocking `layer_indices` for MLP_GLU, added parsing logic
+
+### 📚 Documentation
+
+- **README.md**: New "Selective Layer Width Pruning" section with examples and use cases
+- **Reference Manual**: Comprehensive section with 4+ usage examples and best practices
+- **New Example File**: `examples/selective_layer_width_pruning.py` with 5 complete examples
+- **Updated Roadmap**: Marked selective pruning as completed in v0.2.2
+- **API Documentation**: Updated parameter descriptions for contextual meaning
+
+### 🧪 Testing
+
+- Complete test suite in `tests/test_selective_layer_pruning.py`
+- 12 comprehensive test cases covering:
+  - Basic selective pruning (single and multiple layers)
+  - All neuron selection methods (MAW, VOW, PON)
+  - Compatibility with expansion_rate and expansion_divisor
+  - Data-driven pruning with layer_indices
+  - Invalid input handling and validation
+  - Statistics reporting
+  - Weight preservation in unpruned layers
+  - Result consistency and reproducibility
+
+### 💡 Use Cases
+
+1. **Preserve Critical Layers**: Keep first and last layers at full capacity
+2. **Importance-Based**: Target least important layers identified by analysis
+3. **Domain Adaptation**: Implement asymmetric pruning strategies
+4. **Experimental**: Test different layer-wise pruning patterns
+
+### 🔒 Compatibility
+
+- Fully backward compatible with v0.2.1
+- Works with all neuron selection methods (MAW, VOW, PON)
+- Compatible with both static and data-driven pruning
+- Integrates with expansion_rate and expansion_divisor
+
+### ⚠️ Important Notes
+
+- `layer_indices` validation ensures indices are valid, unique integers within model range
+- Empty lists raise `ValueError`
+- Selective pruning with dataloader only calibrates on specified layers (more efficient)
+- Statistics include `pruned_layers` and `total_layers` when selective pruning is detected
+
+---
+
 ## [0.2.1] - 2025-11-24
 
 ### 🎉 New Features

CHANGELOG.md

@@ -184,6 +184,48 @@ pruned_model.save_pretrained("./pruned-datadriven-model")
 
 **Note:** Data-driven pruning is currently only available with `neuron_selection_method="MAW"`. Using a dataloader with "VOW" or "PON" will raise a `ValueError`.
 
+### Selective Layer Width Pruning (NEW in v0.2.0)
+
+Prune neurons only in specific layers while leaving others unchanged. Perfect for preserving critical layers or implementing layer-specific optimization strategies.
+
+```python
+from transformers import AutoModelForCausalLM
+from optipfair import prune_model
+
+# Load a pre-trained model
+model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B")
+
+# Prune neurons only in specific layers (e.g., middle layers)
+pruned_model, stats = prune_model(
+    model=model,
+    pruning_type="MLP_GLU",
+    neuron_selection_method="MAW",
+    pruning_percentage=30,
+    layer_indices=[5, 10, 15, 20, 25],  # Only prune these layers
+    show_progress=True,
+    return_stats=True
+)
+
+# Print pruning statistics
+print(f"Pruned {stats['pruned_layers']} of {stats['total_layers']} layers")
+print(f"Total reduction: {stats['reduction']:,} parameters ({stats['percentage_reduction']:.2f}%)")
+
+# Save the pruned model
+pruned_model.save_pretrained("./selective-pruned-llama")
+```
+
+**Key Benefits:**
+- 🎯 **Precision Control**: Choose exactly which layers to optimize
+- 🛡️ **Preserve Critical Layers**: Keep first and last layers at full capacity
+- 🔬 **Data-Driven Selection**: Combine with layer importance analysis
+- ⚡ **Full Compatibility**: Works with all MLP_GLU features (expansion_rate, expansion_divisor, dataloader)
+
+**Use Cases:**
+- Preserve embedding and output layers while pruning middle layers
+- Target specific layer ranges based on importance analysis
+- Implement asymmetric pruning strategies for domain adaptation
+- Experiment with different layer-wise pruning patterns
+
 ### Hardware-Optimized Pruning with expansion_divisor (NEW in v0.2.0)
 
 The `expansion_divisor` parameter ensures that intermediate layer sizes are divisible by specific values (32, 64, 128, or 256), optimizing performance on modern GPUs and TPUs.
@@ -219,6 +261,48 @@ pruned_model.save_pretrained("./pruned-optimized-model")
 
 **Note:** Cannot be used alone—requires either `pruning_percentage` or `expansion_rate`.
 
+### Selective Layer Width Pruning (NEW in v0.2.0)
+
+Prune neurons only in specific layers while leaving others unchanged. Perfect for preserving critical layers or implementing layer-specific optimization strategies.
+
+```python
+from transformers import AutoModelForCausalLM
+from optipfair import prune_model
+
+# Load a pre-trained model
+model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B")
+
+# Prune neurons only in specific layers (e.g., middle layers)
+pruned_model, stats = prune_model(
+    model=model,
+    pruning_type="MLP_GLU",
+    neuron_selection_method="MAW",
+    pruning_percentage=30,
+    layer_indices=[5, 10, 15, 20, 25],  # Only prune these layers
+    show_progress=True,
+    return_stats=True
+)
+
+# Print pruning statistics
+print(f"Pruned {stats['pruned_layers']} of {stats['total_layers']} layers")
+print(f"Total reduction: {stats['reduction']:,} parameters ({stats['percentage_reduction']:.2f}%)
+
+# Save the pruned model
+pruned_model.save_pretrained("./selective-pruned-llama")
+```
+
+**Key Benefits:**
+- 🎯 **Precision Control**: Choose exactly which layers to optimize
+- 🛡️ **Preserve Critical Layers**: Keep first and last layers at full capacity
+- 🔬 **Data-Driven Selection**: Combine with layer importance analysis
+- ⚡ **Full Compatibility**: Works with all MLP_GLU features (expansion_rate, expansion_divisor, dataloader)
+
+**Use Cases:**
+- Preserve embedding and output layers while pruning middle layers
+- Target specific layer ranges based on importance analysis
+- Implement asymmetric pruning strategies for domain adaptation
+- Experiment with different layer-wise pruning patterns
+
 ### Pruning Transformer Layers (Depth Pruning)
 
 Remove entire layers from a model for significant efficiency gains. Here, we remove the last 4 layers.
@@ -365,6 +449,9 @@ The optipfair project is actively developed. Here's what's planned for the futur
 ### Future Roadmap
 Our goal is to make optipfair the go-to toolkit for efficient and fair model optimization. Key upcoming features include:
 
+* **Selective Layer Width Pruning**: Implemented in v0.2.0 ✓ - Prune neurons in specific layers using layer_indices
+* **Data-Driven Width Pruning**: Implemented in v0.2.0 ✓ - Hybrid importance with calibration data
+* **Hardware-Optimized Pruning**: Implemented in v0.2.0 ✓ - expansion_divisor for GPU optimization
 * **Attention Pruning**: Implementing Attention Bypass and Adaptive Attention Bypass(AAB).
 * **Advanced Benchmarks**: Integrating more comprehensive performance and evaluation benchmarks.
 * **GPU Optimizations**: Creating a v2.0 with significant GPU-specific optimizations for faster execution. 

README.md

@@ -0,0 +1,163 @@
+# 🚀 OptiPFair v0.2.2 - Selective Layer Width Pruning
+
+We're excited to announce **OptiPFair v0.2.2**, bringing powerful new capabilities for fine-grained control over model pruning!
+
+## 🎯 Headline Features
+
+### 1️⃣ Selective Layer Width Pruning
+
+The `layer_indices` parameter now works for **both DEPTH and MLP_GLU pruning**, giving you unprecedented control over which layers to optimize:
+
+```python
+from optipfair import prune_model
+
+# Prune neurons ONLY in specific layers (preserve first & last)
+pruned_model = prune_model(
+    model=model,
+    pruning_type="MLP_GLU",
+    pruning_percentage=30,
+    layer_indices=[5, 10, 15, 20],  # Only these layers are pruned
+    show_progress=True
+)
+```
+
+**Key Benefits:**
+- 🛡️ **Preserve Critical Layers**: Keep embedding and output layers at full capacity
+- 🎯 **Targeted Optimization**: Prune only the layers that matter
+- 🔬 **Data-Driven Selection**: Combine with layer importance analysis
+- ⚡ **Full Feature Support**: Works with expansion_rate, expansion_divisor, dataloader, all methods
+
+### 2️⃣ Optimized Hybrid Importance Calculation
+
+We've streamlined the data-driven pruning algorithm for better performance:
+
+- **Simplified gate_proj & up_proj**: Now use the same fast MAW method as static pruning
+- **Focused Complexity**: Activation-weighted calculation only where it matters (down_proj)
+- **Faster Execution**: Reduced computational overhead while maintaining effectiveness
+- **Consistent Methodology**: Same MAW formula across static and hybrid approaches
+
+## 📊 What's New
+
+### Extended API
+- ✅ `layer_indices` parameter now contextual: removes layers for DEPTH, prunes neurons for MLP_GLU
+- ✅ Comprehensive validation: checks for valid indices, duplicates, empty lists, type errors
+- ✅ Enhanced statistics: reports `pruned_layers` and `total_layers` for selective pruning
+
+### Improved Performance
+- ⚡ Faster hybrid importance calculation
+- 💾 Selective hook registration (only on specified layers)
+- 🎯 More efficient calibration with layer_indices
+
+### Better Documentation
+- 📖 Complete "Selective Layer Width Pruning" guide in README
+- 📝 Extended reference manual with 4+ detailed examples
+- 💻 New example file with 5 practical use cases
+- 🧪 12 comprehensive test cases
+
+## 💡 Common Use Cases
+
+### Use Case 1: Preserve Embedding Layers
+```python
+# Prune all middle layers, preserve first and last 5
+num_layers = len(model.model.layers)
+middle_layers = list(range(5, num_layers - 5))
+
+pruned_model = prune_model(
+    model=model,
+    pruning_type="MLP_GLU",
+    pruning_percentage=25,
+    layer_indices=middle_layers
+)
+```
+
+### Use Case 2: Importance-Based Pruning
+```python
+from optipfair import analyze_layer_importance
+
+# Step 1: Analyze which layers are least important
+importance_scores = analyze_layer_importance(model, dataloader)
+sorted_layers = sorted(importance_scores.items(), key=lambda x: x[1])
+least_important = [idx for idx, score in sorted_layers[:10]]
+
+# Step 2: Prune only those layers
+pruned_model = prune_model(
+    model=model,
+    pruning_type="MLP_GLU",
+    pruning_percentage=30,
+    layer_indices=least_important
+)
+```
+
+### Use Case 3: Data-Driven Selective Pruning
+```python
+# Combine calibration data with selective pruning
+pruned_model = prune_model(
+    model=model,
+    pruning_type="MLP_GLU",
+    neuron_selection_method="MAW",
+    pruning_percentage=20,
+    dataloader=calibration_dataloader,  # Hybrid importance
+    layer_indices=[5, 10, 15, 20],      # Only these layers
+    show_progress=True
+)
+```
+
+## 🔧 Technical Highlights
+
+### Modified Core Functions
+- `prune_model()`: Now passes layer_indices to MLP_GLU pruning
+- `prune_model_mlp_glu()`: Full selective pruning implementation with validation
+- `setup_mlp_hooks_for_importance()`: Selective hook registration
+- `compute_neuron_pair_importance_maw_hybrid()`: Simplified and optimized
+- `get_pruning_statistics()`: Detects and reports selective pruning
+
+### Enhanced CLI
+```bash
+# CLI now supports layer_indices for both pruning types
+optipfair prune \
+  --model-path meta-llama/Llama-3.2-1B \
+  --pruning-type MLP_GLU \
+  --pruning-percentage 30 \
+  --layer-indices "5,10,15,20" \
+  --output-path ./pruned-model
+```
+
+## 🧪 Testing & Validation
+
+- ✅ 12 comprehensive test cases in `tests/test_selective_layer_pruning.py`
+- ✅ Tested with all neuron selection methods (MAW, VOW, PON)
+- ✅ Verified compatibility with expansion_rate, expansion_divisor, dataloader
+- ✅ Validated error handling and edge cases
+- ✅ Confirmed backward compatibility with v0.2.1
+
+## 📦 Installation
+
+```bash
+pip install --upgrade optipfair
+```
+
+Or with visualization support:
+```bash
+pip install --upgrade "optipfair[viz]"
+```
+
+## 📚 Resources
+
+- **Documentation**: [https://peremartra.github.io/optipfair/](https://peremartra.github.io/optipfair/)
+- **GitHub**: [https://github.com/peremartra/optipfair](https://github.com/peremartra/optipfair)
+- **Examples**: Check out `examples/selective_layer_width_pruning.py`
+- **Tests**: See `tests/test_selective_layer_pruning.py`
+
+## 🙏 Acknowledgments
+
+Thank you to our community for the feedback and suggestions that made this release possible!
+
+## 📝 Full Changelog
+
+See [CHANGELOG.md](https://github.com/peremartra/optipfair/blob/main/CHANGELOG.md) for detailed changes.
+
+---
+
+**Upgrade today and take control of your model optimization!** 🚀
+
+Questions or issues? Open an issue on [GitHub](https://github.com/peremartra/optipfair/issues).