"model" ambiguity

[adamjstewart]

I'm sure this has been brought up before, but use of the word "model" in Lightning seems to conflict with the use of the word "model" in other deep learning libraries.

In libraries like torchvision, "model" refers to the actual NN architecture (e.g., ResNet), whereas in Lightning, "model" refers to the LightningModule that defines how a NN architecture is trained. I myself, and likely many new users, accidentally pass something like a ResNet to pl.Trainer(model=...) due to this ambiguity.

In TorchGeo, we generally follow the torchvision API but also add LightningDataModule and LightningModule objects for reproducibility and experimentation with geospatial datasets. We have submodules like datasets, datamodules, transforms, and models, but we aren't sure where to put LightningModules. Currently, we put them in torchgeo.trainers, but this name conflicts with the Lightning pl.Trainer class. Each object is named something like RegressionTask, ClassificationTask, etc., so maybe torchgeo.tasks would be a better submodule?

Are there any other users who do something similar in their libraries? What submodules do they use? We would like to match the standards used in other libraries as much as possible.

@calebrob6 @isaaccorley

[rohitgr7]

I'd call it systems, that contains the core-logic, and everything else is just boilerplate.
https://pytorch-lightning.readthedocs.io/en/stable/starter/style_guide.html?highlight=system#systems-vs-models

tasks is suitable too. That's what we use in lightning flash.

[adamjstewart]

The style guide is even more ambiguous, they introduce the idea of a "system" and then almost every subclass of LightningModule is *Model 😆

It seems like AutoEncoderSystem is just a collection of architectures and LitModel is the actual code that I'm trying to figure out what to name.

In the flash example, it seems like your "task" is called ImageClassifier and is stored in flash.image. This naming scheme is very different from torchvision/torchtext/torchaudio where every library has a single datasets/transforms/models submodule scheme.

[rohitgr7]

The reason I suggested system is because it's not just about the collection of embedded architecture but how these sub-models are used to define training/evaluation logic inside training_step/validation_step. Moreover LightningModule helps logging, configuring optimizers, optimization logic etc.. Not only that, the same system can be used for various datasets if the embedded logic is compatible with them.

@ethanwharris can give more idea why flash was designed in such a way. IMO, torchvision/torchtext/torchaudio covers only one domain, but flash covers many, so maybe that's why they are structured in such a way.

[adamjstewart]

Yes, I understand that LightningModule encompasses both collections of architectures and training/evaluation logic. What I'm saying is that in the style guide, the name "system" is only used to apply to the former, while the name "model" is only used to apply to the latter. So AutoEncoderSystem is a collection of architectures and LitModel contains the training/evaluation logic.

[rohitgr7]

So AutoEncoderSystem is a collection of architectures and LitModel contains the training/evaluation logic

oh! that's a separate section altogether and LitModel is different in both of them. Will update this doc to avoid this confusion.

[adamjstewart]

A couple more suggestions:

Modules

We could call these things "modules". This would match the naming structure of the base classes:

• LightningDataModule -> datamodules
• LightningModule -> modules

Downside is that it's a bit ambiguous what a "module" does, and name partially conflicts with nn.Module which is something else.

E.g. from torchgeo.modules import RegressionModule

Estimators

This naming scheme matches sklearn's base class. Many of these things in the docs are named like LitClassifier or LitRegressor. It seems like Lightning is designed to emulate sklearn anyway (foo.fit(), foo.predict(), ...), so this would be consistent.

E.g. from torchgeo.estimators import Regressor

[adamjstewart]

We're also discussing this internally in TorchGeo, adding a backwards reference here: torchgeo/torchgeo#996