ModelSummary ignores nn.Parameter

[ZhaofengWu]

🐛 Bug

I don't know if this is a bug of a feature, but currently ModelSummary only reports member nn.Modules and ignores nn.Parameters that are directly created. I believe these parameters are an important part (and sometimes, a big part in terms of the number of weights) of the model so that they should be reported.

cc @Borda @kaushikb11 @awaelchli @rohitgr7

[awaelchli]

Hey @ZhaofengWu

Happy to see you find the model summary useful!
I looked at this very briefly now. With the following code I get:

import torch

from pytorch_lightning import LightningModule
from pytorch_lightning.utilities.model_summary import ModelSummary

class BoringModel(LightningModule):
    def __init__(self):
        super().__init__()
        self.param = torch.nn.Parameter(torch.rand(2, 2))  # 2 * 2 = 4
        self.layer = torch.nn.Linear(2, 2)  # 2 * 2 + 2 = 6


if __name__ == "__main__":
    model = BoringModel()
    summary = ModelSummary(model)
    print(summary)

  | Name  | Type   | Params
---------------------------------
0 | layer | Linear | 6     
---------------------------------
10        Trainable params
0         Non-trainable params
10        Total params
0.000     Total estimated model params size (MB)

We can see that the correct amount of parameters are getting reported (10 trainable parameters). Perhaps the size in MB is wrong.

Are you suggesting to include the nn.Parameter as a separate row in the table?

[ZhaofengWu]

Perhaps the size in MB is wrong.

That's probably just a rounding thing?

Are you suggesting to include the nn.Parameter as a separate row in the table?

Yes. It's a bit confusing that the individual param counts don't add up to the total number as well.

[ZhaofengWu]

But definitely not a huge issue and probably has low priority.

[stale]

This issue has been automatically marked as stale because it hasn't had any recent activity. This issue will be closed in 7 days if no further activity occurs. Thank you for your contributions, Pytorch Lightning Team!

[ZhaofengWu]

Commenting for activity

[qixiang109]

I believe this problem still exists in pl version 1.6.4

import torch
import torch.nn as nn
import pytorch_lightning as pl
from pytorch_lightning.utilities.model_summary import ModelSummary

class Example(pl.LightningModule):
    def __init__(self):
        super().__init__()
        self.lin1 = nn.Linear(32,64)
        self.lin2 = nn.Parameter(torch.rand([32,64]))
model = Example()
print(ModelSummary(model,max_depth=-1))
print(pl.__version__)

shows only lin1 but not lin2

  | Name | Type   | Params
--------------------------------
0 | lin1 | Linear | 2.1 K 
--------------------------------
4.2 K     Trainable params
0         Non-trainable params
4.2 K     Total params
0.017     Total estimated model params size (MB)
1.6.4

[stale]

This issue has been automatically marked as stale because it hasn't had any recent activity. This issue will be closed in 7 days if no further activity occurs. Thank you for your contributions, Pytorch Lightning Team!