How to do fit and test at the same time with Lightning CLI ? #17300
-
Hi, I am wondering how to
This is not very convenient for me since and the log dir is unknown before |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
Instead of having a CLI with subcommands, you can use the instantiation only mode and call test right after fit. However, a fair warning. The test set should be used as few times as possible. Measuring performance on the test set too often is a bad practice because you end up optimizing on the test. So, technically it is better to use the test subcommand giving explicitly a checkpoint (only one among many you may have) and not plan to run the test for every fit you do. |
Beta Was this translation helpful? Give feedback.
Instead of having a CLI with subcommands, you can use the instantiation only mode and call test right after fit.
However, a fair warning. The test set should be used as few times as possible. Measuring performance on the test set too often is a bad practice because you end up optimizing on the test. So, technically it is better to use the test subcommand giving explicitly a checkpoint (only one among many you may have) and not plan to run the test for every fit you do.