Logging custom metric with multiple dataloaders at the end of validation

[keremyldrr]

Hello,
I am using 2 validation dataloaders for validation, and logging is working great. However for one metric,, i need to log the division of the sum of two metrics. To clarify , i need to log the average IOU, so i need to sum over all the values and number of predictions and divide them for logging. I cannot do this in validation_step() as things haven't summed over yet. But when I use the validation_epoch_end(), i get the a list of outputs, each corresponding to one dataloader. However, when I do logging, i cannot log with respect to multiple dataloaders.

To clarify, I can compute average IOU by parsing the outputs of two dataloaders and log them under different names. This is fine during training, but for validation I want to see the results of each dataloader separately. Is this possible without duplicates? Right now, i am doing the logging with different names for both dataloaders, so i have them during training but when I do trainer.validate(), i get duplicates on two dataloaders, as both are logging the same things. Can I have the separation without having unique names and have the respective columns w.r.t dataloaders ?

[rohitgr7]

this is not possible yet. You have to have different names when using multiple dataloaders since internally Lightning needs a key (metric name) to manage the accumulation and logging-related stuff and if two or more different metrics have the same name, this isn't possible.