Skip to content

Add monitor as a training hyperparameter to training_config.yaml #174

@jeipollack

Description

@jeipollack

We may want to explore monitoring different metrics when determining which model checkpoint to save during training.

Currently, the code conditionally monitors:

  • "mean_squared_error" when using Keras' built-in MSE (typically with unmasked data)
  • "loss" when using our custom MaskedMeanSquaredError loss function (which embeds masking logic within y_true)

This logic is currently hard-coded, which limits flexibility when experimenting with alternative loss functions or evaluation metrics.

Recommendation:

Add a new configurable parameter, monitor, to the training_config.yaml file to specify which validation metric to monitor in the ModelCheckpoint callback.

Benefits:

  • Enables switching between "val_loss", "val_mean_squared_error", "val_masked_mean_squared_error", etc.
  • Avoids the need to modify source code when changing loss/metric behavior
  • Aligns with standard Keras patterns by exposing monitor as a top-level training hyperparameter

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions