Stats Handler not printing iter/loss values

[skrishnanv]

Hello,
I have a weird problem. I first ran the code here with modification for data loaders. The output was as expected with printings on console. For example,

trainer = create_multigpu_supervised_trainer(net=net,
                                                     optimizer=opt,
                                                     loss_fn=loss,
                                                     prepare_batch=prepare_batch
                                                     )

<Logger ignite.engine.engine.Engine (INFO)> # print(trainer.logger)
INFO:ignite.engine.engine.Engine:Engine run starting with max_epochs=4.
INFO:trainer:Epoch: 1/4, Iter: 1/20 -- Loss: 0.6601
INFO:trainer:Epoch: 1/4, Iter: 2/20 -- Loss: 0.6941

Next i modified this code to make some classes around it to add things like loss, optimiser, trainer event handlers etc. After this modification, the console printed nothing.
However this time, the logger level changed; i have no clue why.
<Logger ignite.engine.engine.Engine (WARNING)>
Despite this newer level of logging, I added ignite events and was able to print. Even changing the debug levels did not help.

print(self.trainer._event_handlers) gives below.
defaultdict(<class 'list'>, {<Events.ITERATION_COMPLETED: 'iteration_completed'>: [(<bound method StatsHandler.iteration_completed of <monai.handlers.stats_handler.StatsHandler object at 0x000001C2C164B220>>, (<ignite.engine.engine.Engine object at 0x000001C2C164B040>,), {})], <Events.EPOCH_COMPLETED: 'epoch_completed'>: [(<bound method StatsHandler.epoch_completed of <monai.handlers.stats_handler.StatsHandler object at 0x000001C2C164B220>>, (<ignite.engine.engine.Engine object at 0x000001C2C164B040>,), {})], <Events.EXCEPTION_RAISED: 'exception_raised'>: [(<bound method StatsHandler.exception_raised of <monai.handlers.stats_handler.StatsHandler object at 0x000001C2C164B220>>, (<ignite.engine.engine.Engine object at 0x000001C2C164B040>,), {})]})

Any suggestions?

[Nic-Ma]

Hi @skrishnanv ,

Thanks for your interest here.
Could you please help share your latest test code for further debug?

Thanks in advance.

[skrishnanv]

i have understood the source of my troubles. In the example tutorial, there is the line
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
which i had forgotten to add in my refactored class. This caused several wasted hours :-(
However, it is a little confusing to the end user, ignite default logger level is WARNING, whereas monai stats logger operates at info level. This is not immediately obvious or sometimes forgotten. Would it make sense to emit a warning message inside stats logger if the overall log level is set to WARN than INFO? or it makes sense for ignite to reduce their level?
I thank you for your prompt response.

[Nic-Ma]

Thanks for your great feedback!
@vfdev-5 May I know the reason why you set WARNING level for the ignite logging?
Maybe I should change MONAI logging level(currently always INFO level) to follow yours.

Thanks in advance.

[vfdev-5]

Hi @Nic-Ma , thanks for pinging!
We put logger default level to warning as engine reports various things like started/ended etc with info level and even more info about events system at debug level. So to avoid spamming, we decided to set warning level.

[Nic-Ma]

Thanks @skrishnanv @vfdev-5 for the suggestions.
I submitted PR #3774 to enhance the logging logic.

Thanks.