Skip to content

Conversation

@sfc-gh-mwyatt
Copy link
Collaborator

Adding the ability to override yaml config values via the CLI:

arctic_training my_config.yaml --overrides epochs=2 model.dtype=fp32 skip_validation=True

Currently a bit limited in that we can only override numeric, string, or boolean type config values. We might be able to extend this to other types if we think that would be useful.

Copy link
Collaborator

@sfc-gh-sbekman sfc-gh-sbekman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a CLI API this feels a bit odd. should we perhaps just enclose all the override into quotes as a single argument?

--overrides "epochs=2 model.dtype=fp32 skip_validation=True"

otherwise it'll quickly become a problem as one would accidentally start adding other -- arguments in between. Example:

--overrides epochs=2 model.dtype=fp32 --gpus 10 skip_validation=True

it's because we are used to args to be --key value and that --overrides is too far from the values. I hope it makes sense.

If you add a top level config layer then it'd become much easier to use normal argparse, as you can then do --main.data.max_path 10 --main.optimizer.lr 0.1 etc. I'm giving main as an example - we can discuss what to call it. we briefly discussed that it'd be good not to have flat config files w/o an umbrella to contain them all.

I guess it could be even --config.foo.bar value - let's discuss tomorrow?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants