SWAResnet training of cifar10-baseline.ipynb is very slow on Colab Pro+ GPU(A100 Nvidia GPU)

[Keiku]

In the linked notebook, the initial setup causes an error, so I changed it to the following.

! pip install --quiet "pandas" "torch==1.11.0+cu113" "torchvision" "ipython[notebook]" "seaborn" "pytorch-lightning==1.6.4" "torchmetrics>=0.6" "lightning-bolts==0.5.0" "torchtext==0.12.0" --extra-index-url https://download.pytorch.org/whl/cu113

Resnet training finishes fine, but SWAResnet training is very slow. Anyone know the cause of this?

https://colab.research.google.com/github/PytorchLightning/lightning-tutorials/blob/publication/.notebooks/lightning_examples/cifar10-baseline.ipynb

[akihironitta]

Hi @Keiku, thank you for reporting the issue. What error did you see exactly? If it's something that needs to be updated in the notebook, would you mind opening an issue (or directly sending a PR with your patch) in the tutorials repository?

For the performance issue, the Bolts repo might be the best place to report. Please mind that the whole Bolts package is currently under review as tracked in Lightning-Universe/lightning-bolts#839.

Thank you again!

[Keiku]

@akihironitta Thank you for your reply. It is true that the notebook is old and needs an update. I will create a PR, so please wait a moment. Thanks for the info on the Bolts package as well.