Enhancement: Validate config with pydantic

[benatouba]

Would you be interested in implementing Pydantic classes for config model validation? This would yield type safety and greatly help with testing/lower the need for tests of config?

Pydantic GitHub is well-established, widely used and very perfomant (maybe not the most important point here). But I don't think it will go away anytime soon and greatly helps with further code development.

[gampnico]

Type validation with pydantic-settings would be great, as tomllib is quite brittle and users might get caught out on "true" and "True". I don't think there would be a compatibility issue with numba as njitted functions reference the Config class' attributes directly.

Performance is actually quite important here, as the majority of the performance increase with the latest release comes from the configuration system.

[benatouba]

Ah okay. I agree. The first draft I created only uses pydantic base, not pydantic-settings. I was under the impression that the main advantage of pydantic-settings is the management of environment variables and secrets.

I think we can go from my implementation to adding support for pydantic-settings quite easily though.