Adding ignore_index to segmentation mean_iou
#3266
Conversation
VijayVignesh1
requested review from
Borda,
SkafteNicki and
justusschock
as code owners
Borda
changed the title
ignore_index to segmentation mean_iou
Codecov Report❌ Patch coverage is ❌ Your project check has failed because the head coverage (32%) is below the target coverage (95%). You can increase the head coverage or adjust the target coverage.
Additional details and impacted files@@ Coverage Diff @@
## master #3266 +/- ##
========================================
- Coverage 37% 32% -5%
========================================
Files 364 349 -15
Lines 20096 19905 -191
========================================
- Hits 7520 6376 -1144
- Misses 12576 13529 +953 🚀 New features to boost your workflow:
|
Co-authored-by: Jirka Borovec <[email protected]>
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for an ignore_index parameter to the segmentation mean_iou metric, allowing users to specify a class index (such as 255) that should be excluded from both intersection and union computations. This feature is only available when input_format="index".
- Added
ignore_indexparameter to both theMeanIoUclass andmean_ioufunctional implementation - Updated the internal
_mean_iou_updatefunction to filter out ignored indices before processing - Added comprehensive test coverage for the new functionality
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| src/torchmetrics/segmentation/mean_iou.py | Added ignore_index parameter to MeanIoU class constructor and passed it to update function |
| src/torchmetrics/functional/segmentation/mean_iou.py | Added ignore_index parameter to mean_iou function and _mean_iou_update with filtering logic |
| tests/unittests/segmentation/test_mean_iou.py | Enhanced test suite with ignore_index parameter coverage and added specific test case |
| CHANGELOG.md | Added changelog entry for the new feature |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| @pytest.mark.parametrize("include_background", [True, False]) | ||
| class TestMeanIoU(MetricTester): | ||
| """Test class for `MeanIoU` metric.""" | ||
|
|
There was a problem hiding this comment.
The tolerance has been relaxed from 1e-4 to 1e-2 without explanation. This change could mask precision issues and should either be reverted or documented with a comment explaining why the relaxed tolerance is necessary.
| # The tolerance has been relaxed from 1e-4 to 1e-2 due to minor numerical differences | |
| # between the reference and implementation, likely caused by floating point precision | |
| # or differences in third-party library computations (e.g., MONAI, PyTorch). |
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
No additional comment needed here since the rationale is self-understood.
| expected_ious = [0.6667, 0.3333, 0.0] | ||
| for idx, (iou, iou_func) in enumerate(zip(miou_per_class, miou_func)): | ||
| assert torch.allclose(iou, iou_func, atol=1e-4) | ||
| assert iou == pytest.approx(expected_ious[idx], rel=1e-3) |
There was a problem hiding this comment.
The expected IoU values are hardcoded without explanation of how they were calculated. Consider adding a comment explaining the calculation or using a more descriptive variable name to make the test more maintainable.
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
No additional comment needed here since the rationale is self-understood.
Co-authored-by: Copilot <[email protected]>
2 participants
What does this PR do?
Fixes #2747
Before submitting
PR review
Added ignore_index to segmentation mean_iou class and function. Users can now add an index in the target which can be ignored (like 255). Added testcases to verify the same. Note: This is only for the case where input_format = "index"Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃
📚 Documentation preview 📚: https://torchmetrics--3266.org.readthedocs.build/en/3266/