My model stops updating to tensorboard after a few epochs? #11320
Unanswered
mm04926412
asked this question in
Lightning Trainer API: Trainer, LightningModule, LightningDataModule
Replies: 1 comment
-
I don't think there is any such limitation. Can you reproduce the issue with https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pl_examples/bug_report/bug_report_model.py? or if you can, maybe try another logger, just to know whether this issue exists in lightning or tensorboard. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
This issue has appeared without warning and I'm unsure of any changes I've made that could have caused it. whenever I run a lightning model it only updates a few epochs to tensorboard before going "dark", if I stop the model and resume it then the logging continues from the new start location (step-wise) with a gap on the plot. Is there a known issue / accidental configuration option I might have applied which limits the total number of logs to tensorboard?
Beta Was this translation helpful? Give feedback.
All reactions