Skip to content

[ENH] Predictions: Show errors#6012

Merged
lanzagar merged 1 commit intobiolab:masterfrom
janezd:predictions-show-errors
Jun 22, 2022
Merged

[ENH] Predictions: Show errors#6012
lanzagar merged 1 commit intobiolab:masterfrom
janezd:predictions-show-errors

Conversation

@janezd
Copy link
Contributor

@janezd janezd commented Jun 9, 2022

Issue

Closes #5971.

Description of changes

Add columns with errors for each predictor.

Please comment on functionality and interface. Checking and tests will come afterwards.

Try it on housing and heart disease; errors on iris are too small.

Includes
  • Code changes
  • Tests
  • Documentation

@janezd janezd force-pushed the predictions-show-errors branch from b2695ef to 91a9cf6 Compare June 9, 2022 12:53
@ajdapretnar
Copy link
Contributor

I fully approve this functionality! It enables selecting the most underpredicted, overpredicted and best predicted instances and showing them in the scatter plot. I also like the many options for error computation, discretely placed at the top.

@janezd janezd force-pushed the predictions-show-errors branch from 91a9cf6 to bf07d10 Compare June 12, 2022 16:19
@codecov
Copy link

codecov bot commented Jun 12, 2022

Codecov Report

Merging #6012 (def1d61) into master (6d95493) will increase coverage by 0.01%.
The diff coverage is 96.61%.

@@            Coverage Diff             @@
##           master    #6012      +/-   ##
==========================================
+ Coverage   86.60%   86.61%   +0.01%     
==========================================
  Files         315      315              
  Lines       67317    67479     +162     
==========================================
+ Hits        58297    58447     +150     
- Misses       9020     9032      +12     

@janezd janezd force-pushed the predictions-show-errors branch from bf07d10 to 6994a85 Compare June 12, 2022 16:39
@janezd janezd changed the title [RFC] Predictions: Show errors [ENH] Predictions: Show errors Jun 12, 2022
@janezd janezd force-pushed the predictions-show-errors branch from 6994a85 to 90db1b0 Compare June 12, 2022 17:01
@janezd janezd force-pushed the predictions-show-errors branch from 90db1b0 to 5dd7b1a Compare June 17, 2022 11:19
elif self._probs[column].size:
return 1 - self._probs[column][row, int(actual)]
else:
diff = self._values[column // 2][row] - actual
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

column was already divided by 2 above (line 1271)!

I must say, there is quite a lot of %2==1 and //2 in this file... :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

column was already divided by 2 above (line 1271)!

It is disturbing that (seemingly thorough) tests didn't catch this because they used only a single predictor... :(

I fixed this and modified the tests to use two predictors.

I must say, there is quite a lot of %2==1 and //2 in this file... :)

Apparently (at least) one too many. :)

@janezd janezd force-pushed the predictions-show-errors branch from 5dd7b1a to def1d61 Compare June 20, 2022 09:53
@lanzagar lanzagar merged commit 52cbdb0 into biolab:master Jun 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Show errors in Predictions

3 participants