Please get back compute on step parameter. #1082
PososikTeam
started this conversation in
Ideas
Replies: 1 comment
-
Are you alive guys? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
The reason to return compute on step parameter is in retrieval metrics.
My use retrieval metrics case is that i train the model and want to calculate retrieval metrics on the validation dataset during training.
I wrote the own metric inheritor of TorchMetrics Metric class, that in validation step it call only update function which inference the model and write feature vector to storage. And in validation epoch end it call compute function which use faiss library to calculate distance between queries and database, and then calculate the main metric for example Recall@k.
So, when you removed compute on step parameter my class stopped working, because it compute only batch metric, which not contain all database for queries. Because of that i can't use newer version of TorcheMetrics.
My suggestion is to return compute on step parameter, for bring the developers who use yours library more flexible. Because more cases need to do some function in every step i.e. old update, and only then compute the metric on full data obtained by updates.
Actually now i don't clearly understand why we need 2 function to override i.e compute and update. Because when you remove compute on step parameter it calls 2 funсtion update and compute, why not leave one function to override.
If you know any over way to solve my metric calculate case, please let me know, because i really don't understand how to solve my case with TorchMetrics retrival metrics.
Beta Was this translation helpful? Give feedback.
All reactions