Optimizers for nested modules

[Co1lin]

class MyNet(pl.LightningModule):
    def __init__(self):
        self.m1 = MyMod1()
        self.m2 = MyMod2()

If I implement different configure_optimizers for different submodules MyMod(also pl.LightningModule), is it correct that parameters in each MyMod will be updated by their own optimizers returned by configure_optimizers? If I only implement configure_optimizers for the top module, is it correct that parameters in submodules will be optimized by the same optimizer returned by configure_optimizers of the top module?

[justusschock]

When you have nested LightningModules, their configure_optimizers will never be called unless you explicitly call it in the top-level configure_optimizers.

That being said, if you call, merge and return the optimizers created there, these optimizers should only contain parameters from the respective submodule (if implemented correctly)