Refactor Feature Importances, fix shap multiclass, fix T2E FIs by anwurl · Pull Request #362 · emdgroup/octopus

anwurl · 2026-03-12T20:26:11Z

main focus: centralize and remove duplications in the feature importance code
a refactoring and simplification of octo will be done later
make shap work for multi-class
make FI work for T2E
update catboost, shap

Copilot

Pull request overview

This PR refactors feature-importance implementations by centralizing shared computation logic into octopus.predict.feature_importance “Layer 1” primitives, then reusing those primitives from both training and predict code paths to reduce duplication.

Changes:

Added shared FI primitives (compute_per_repeat_stats, compute_internal_fi, compute_permutation_single, compute_shap_single) and refactored predict FI orchestrators to use them.
Refactored Training permutation/internal/SHAP FI methods to delegate into the shared primitives; removed the legacy calculate_fi_group_permutation entry points.
Updated bag + tests to call the unified permutation FI implementation and added a no-groups test variant.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 8 comments.

File	Description
tests/modules/test_training_feature_importances.py	Updates FI method coverage to use unified permutation method and adds a “no groups” permutation variant.
octopus/predict/feature_importance.py	Introduces shared FI primitives and refactors predict FI orchestrators to use them.
octopus/modules/octo/training.py	Removes duplicated FI logic and delegates internal/permutation/SHAP FI to shared primitives.
octopus/modules/octo/bag.py	Switches from the removed group-permutation method to the unified permutation method.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

octopus/predict/feature_importance.py

octopus/modules/octo/training.py

octopus/predict/feature_importance.py

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

octopus/predict/feature_importance.py

octopus/modules/octo/training.py

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

octopus/modules/octo/training.py

octopus/predict/feature_importance.py

Copilot

Pull request overview

Copilot reviewed 6 out of 7 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

tests/modules/test_training_feature_importances.py

octopus/predict/feature_importance.py

octopus/models/classification_models.py

Copilot

Pull request overview

Copilot reviewed 6 out of 7 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

octopus/predict/feature_importance.py

+    # Build target_assignments from target_col
+    target_assignments = {target_col: target_col}



octopus/models/classification_models.py

+        # KNOWN ISSUE: CatBoost multiclass is disabled because shap.Explainer(model, bg)
+        # segfaults in SHAP <=0.51 when using TreeExplainer's interventional mode with
+        # CatBoost multiclass models.  Re-enable once SHAP fixes this upstream.
+        # See datasets_local/specifications_refactorfi/03_shap_catboost_segfault_proposal.md
+        # Original: ml_types=[MLType.BINARY, MLType.MULTICLASS],
+        ml_types=[MLType.BINARY],


Copilot

Pull request overview

Copilot reviewed 6 out of 7 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

tests/modules/test_training_feature_importances.py

+    training = _create_training_instance(
+        data_train, data_dev, data_test, ml_type, model_name, feature_cols, feature_groups
+    )
+    training.fit()



octopus/models/classification_models.py

+        # KNOWN ISSUE: CatBoost multiclass is disabled because shap.Explainer(model, bg)
+        # segfaults in SHAP <=0.51 when using TreeExplainer's interventional mode with
+        # CatBoost multiclass models.  Re-enable once SHAP fixes this upstream.
+        # See datasets_local/specifications_refactorfi/03_shap_catboost_segfault_proposal.md
+        # Original: ml_types=[MLType.BINARY, MLType.MULTICLASS],
+        ml_types=[MLType.BINARY],


anwurl self-assigned this Mar 12, 2026

anwurl added Predict Octo labels Mar 12, 2026

anwurl requested a review from Copilot March 12, 2026 20:49

Copilot started reviewing on behalf of anwurl March 12, 2026 20:50 View session

Copilot AI reviewed Mar 12, 2026

View reviewed changes

anwurl requested a review from Copilot March 12, 2026 22:27

Copilot started reviewing on behalf of anwurl March 12, 2026 22:28 View session

Copilot AI reviewed Mar 12, 2026

View reviewed changes

octopus/predict/feature_importance.py Show resolved Hide resolved

octopus/modules/octo/training.py Show resolved Hide resolved

octopus/modules/octo/training.py Outdated Show resolved Hide resolved

anwurl requested a review from Copilot March 12, 2026 23:11

Copilot started reviewing on behalf of anwurl March 12, 2026 23:11 View session

Copilot AI reviewed Mar 12, 2026

View reviewed changes

octopus/modules/octo/training.py Outdated Show resolved Hide resolved

octopus/modules/octo/training.py Show resolved Hide resolved

octopus/predict/feature_importance.py Outdated Show resolved Hide resolved

anwurl changed the title ~~Refactor feature importances~~ Refactor Feature Importances and fix shap multiclass Mar 13, 2026

anwurl added 5 commits March 13, 2026 18:03

remove fi code duplication

8657130

shap for multiclass

ab73a20

copilot fixes

689d99b

make tests faster

4d5ee33

catboost multi-class fix

fb3d4f7

anwurl force-pushed the refactor/fi branch from c76b547 to fb3d4f7 Compare March 13, 2026 18:03

copilot fixes

ea4b8ba

anwurl requested a review from Copilot March 13, 2026 18:33

Copilot started reviewing on behalf of anwurl March 13, 2026 18:33 View session

Copilot AI reviewed Mar 13, 2026

View reviewed changes

tests/modules/test_training_feature_importances.py Show resolved Hide resolved

octopus/predict/feature_importance.py Show resolved Hide resolved

octopus/models/classification_models.py Show resolved Hide resolved

anwurl added 2 commits March 13, 2026 19:41

test fix

4290b9c

tests now show which method fails

b715661

anwurl requested a review from Copilot March 13, 2026 21:04

Copilot started reviewing on behalf of anwurl March 13, 2026 21:04 View session

Copilot AI reviewed Mar 13, 2026

View reviewed changes

anwurl added 2 commits March 14, 2026 14:06

remove pytest-fork

de3ce5f

uv lock

db7acf9

revert tests

d323de9

anwurl requested a review from Copilot March 14, 2026 17:34

Copilot started reviewing on behalf of anwurl March 14, 2026 17:35 View session

anwurl requested review from dasmy, kalama-ai and nihaase March 14, 2026 17:36

Copilot AI reviewed Mar 14, 2026

View reviewed changes

fix warnings

f3b742b

anwurl changed the title ~~Refactor Feature Importances and fix shap multiclass~~ Refactor Feature Importances, fix shap multiclass, fix T2E FIs Mar 14, 2026

		# Build target_assignments from target_col
		target_assignments = {target_col: target_col}

Conversation

anwurl commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

anwurl commented Mar 12, 2026 •

edited

Loading