This example is adapted from an official optuna example.
It shows how more complicated experiments can be configured with hydra or
hydra-zen.
Pytorch-Lightning is used to avoid boilerplate for
training the neural networks. This example also shows how the
OptunaPruningSweeper
can be used.
To show a more complicated configuration a ResNet model is implemented adapted from
torchvision. However, pre-activation
ResNet blocks are used. The
FashionMNIST
dataset is used.
The hydra specific code/configuration is located in run_hydra.py and config_hydra.
The code/configuration specific to hydra_zen can be found in run_hydra_zen.py and config_hydra_zen.
It is recommended to first create a virtual environment. In this environment install the dependencies with
pip install -r requirements.txt
To run the hyperparameter optimization with pruning install the OptunaPruningSweeper. This plugin has not yeet
been added to PyPI. Install it by cloning this repository and execute in your virtual environment
pip install PATH-TO-CLONED-REPOSITORY-OF-HYDRA-OPTUNA-PRUNING-SWEEPER
If you are interested in a template for pytorch-lightning + hydra also take a look at this repository.