How to refer to dataset attributes in CLI config file?

[wwbrannon]

I'm writing a yaml config file for a pytorch-lightning CLI script. The file specifies model, data, trainer arguments and misc parameters. The model arguments depend on the dataset -- specifically, the dimension of the model's first layer needs to match what's in the data.

Do I need to hardcode this value or is there a way to tell PL to discover it dynamically? Here's an example:

seed_everything: 42
ckpt_path: /path/to/ckpt

model:
    class_path: src.mymodel.litclass
    init_args:
        n_classes: 1024
        key: 'target'
        lr: 0.0005
        weight_decay: 0
        model:
            class_path: src.mymodel.baseclass
            init_args:
                n_layers: 3
                n_heads: 4
                d_input: ??????
                d_hidden: 64
                d_feedforward: 128
                p_dropout: 0.5

data:
    class_path: src.mydataset.mydatamodule
    init_args:
        data_dir: ../data

How should I fill in d_input?

[rohitgr7]

I think you have to hard-code it here, but @carmocca can confirm.

[wwbrannon]

thanks!

[carmocca]

You can use argument linking: https://pytorch-lightning.readthedocs.io/en/stable/common/lightning_cli.html#argument-linking
And have any initial default value

Or variable interpolation with omegaconf: https://pytorch-lightning.readthedocs.io/en/stable/common/lightning_cli.html#variable-interpolation

[wwbrannon]

hmm, but does either of these let me get something out of a python object that's only known at runtime, not yaml parse time? I'm not (other than this) passing the length of the data into the config as a parameter

[wwbrannon]

thanks for all your work on this very useful package, btw

[carmocca]

You can use apply_on='instantiate' for the cases when it's known at instantiation (runtime), not parse time: https://jsonargparse.readthedocs.io/en/stable/#argument-linking