Skip to content

Conversation

@SkafteNicki
Copy link
Collaborator

@SkafteNicki SkafteNicki commented Dec 2, 2025

What does this PR do?

Fixes Lightning-AI/pytorch-lightning#20153
PR does the following:

  • Adds a metric_state property to CompositionalMetric
  • By utilizing this property in MetricCollection, we can now make sure that CompositonalMetric created from indexing the same underlying metric will end up in the same compute group and therefore only be calculated once even though they are now "separate" metrics
  • This will allow for a more advance logging in lightning of metrics that are non-scalar by doing something like this
# initialize a nons-scalar returning metric, in this case we use MeanAveragePrecision
# this metric returns a dict, each being a scalar tensor, assume we are only interested in a few of them
map = torchmetrics.detection.MeanAveragePrecision()

# index into the metric the values we want to log and repack into a MetricCollection
# MetricCollection compute group feature will make sure that calculations are only done once
self.metrics = torchmetrics.MetricCollection(
    {
        "map_50": map["map_50"],
        "map_75": map["map_75"],
        "map_small": map["map_small"],
    },
    prefix="val_",
)
# then all can be logged using log_dict
self.log_dict(self.metrics)

This has the intended purpose of getting users to just log the metric object instead of calling compute and forward then logging the values themself, which can lead to problems (see linked issue).

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃


📚 Documentation preview 📚: https://torchmetrics--3309.org.readthedocs.build/en/3309/

@SkafteNicki SkafteNicki added the enhancement New feature or request label Dec 2, 2025
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Dec 2, 2025
@codecov
Copy link

codecov bot commented Dec 2, 2025

Codecov Report

❌ Patch coverage is 13.33333% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 37%. Comparing base (88bca94) to head (d56d894).

❌ Your project check has failed because the head coverage (37%) is below the target coverage (95%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #3309    +/-   ##
=======================================
- Coverage      37%     37%    -1%     
=======================================
  Files         364     349    -15     
  Lines       20096   19911   -185     
=======================================
- Hits         7520    7328   -192     
- Misses      12576   12583     +7     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Confusing recommendation to use sync_dist=True even with TorchMetrics

1 participant