Add Feature Importance Analysis Tools

[likeonl]

TimeMixer lacks tools to analyze which input features contribute most to its forecasting performance. Feature importance analysis would provide valuable insights for researchers and practitioners to understand model decisions and improve data collection strategies.

Proposed Implementation

1. Model-agnostic feature importance methods

Implement permutation-based feature importance that works with any model:

def permutation_feature_importance(model, dataset, metric_func, n_repeats=10):
    """Calculate feature importance by permuting features one by one.
    
    Args:
        model: Trained TimeMixer model
        dataset: Dataset with features to analyze
        metric_func: Function to calculate performance metric
        n_repeats: Number of permutation repeats
        
    Returns:
        DataFrame with feature importance scores
    """

2. Model-specific feature importance (for TimeMixer)

Add TimeMixer-specific analysis leveraging model internals:

• Scale-based feature importance: Analyze importance across different time scales
• Season vs. trend component attribution
• Channel-wise importance in multivariate forecasting

3. Visualizations

Create standardized visualization tools:

• Feature importance bar charts
• Heatmaps for temporal importance
• Scale-specific feature importance plots

4. Integration in experiment workflow

Allow feature importance analysis to be run automatically after training:

python run.py ... --analyze_feature_importance True

Expected Outcome

• Quantitative understanding of feature importance
• Insights into how different scales contribute to predictions
• Ability to identify redundant or unimportant features
• Tools to explain model predictions

Implementation Details

• Add new module utils/feature_importance.py
• Extend experiment classes to include optional analysis
• Add plotting utilities in utils/visualization.py
• Include examples in documentation

Dependencies

• Add shap, matplotlib to optional dependencies

[drorhunvural]

That would be a good update!