Unexpected behaviour in `CliSettingsSource` when `cli_parse_args=True` in `model_config`

[trygve-baerland]

Suppose I want to create a simple CLI tool, where the only allowed source is from the command line.
Something along the lines of:

import sys

from pydantic_settings import (
    BaseSettings,
    CliSettingsSource,
    PydanticBaseSettingsSource,
    SettingsConfigDict,
)


class Settings(BaseSettings):
    model_config = SettingsConfigDict(cli_parse_args=True)

    value: int

    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> tuple[PydanticBaseSettingsSource, ...]:
        """Set up config priority."""
        return (CliSettingsSource(settings_cls),)


sys.argv = ["example.py", "--value", "3"]

print(Settings().model_dump())

which, for me, will output the following:

pydantic_core._pydantic_core.ValidationError: 1 validation error for Settings
value
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.11/v/missing

whereas I would expect {'value': 3}.

I understand that in this particular case, I can easily fix this by using CliSettingsSource(settings_cls, cli_parse_args=True) instead of using a model_config.
However, this requirement is not consistent with other CLI-related config variables (e.g. cli_prog_name or cli_kebab_case), which will be read from settings_cls.model_config if not provided in the CliSettingsSource constructor.

My suspicion is that this is due to the check here, where both None and False are used to represent "do not parse command line arguments".
This can very well be the intended behaviour, and there's probably some use case I'm not being properly aware of. In that case, feel free to say so, and I'll happily close this issue. Otherwise, I'm also more than happy to contribute and submit a PR if you want.

[hramezani]

@kschwab Do you have time to take a look?

[kschwab]

@trygve-baerland @hramezani looking closer at the example, the reason the above was not working is CliSettingsSource requires the cli_parse_args to be set explicitly. i.e, refer to the example in the basics about setting priority.

Whereas other configurations get pulled implicitly from settings_cls, cli_parse_args is unique in that instructs the CliSettingsSource to parse the cli args during instantiation if True. When integrating with other parsers, this may not be desired.

For the case above, it should just be:

        """Set up config priority."""
        return (CliSettingsSource(settings_cls, cli_parse_args=True),)

I am suggesting the PR be reverted as it breaks backwards compatibility.

[hramezani]

Here is the revert PR #655

[trygve-baerland]

Great work with the investigation @kschwab ! That wasn't a case I was thinking of when writing up this issue, but it makes sense, so thanks for adding the comment here :)

[trygve-baerland]

I agree with what you're saying in #654, that #611 causes the actual parsing of command line arguments to happen in CliSettingsSource.__init__ when cli_parse_args=True, but could a different approach be to not have the call to _load_env_var in the __init__ method? (https://github.com/pydantic/pydantic-settings/blob/main/pydantic_settings/sources/providers/cli.py#L269)
Actually, it seems like CliSettingsSource(..., cli_parse_args=True) eagerly parses the command line arguments in earlier versions as well. What's the reason for this, Instead of doing the parsing during the settings collection (Which I'm guessing is the BaseSettings.__call__ method)?

Having tested a little bit, it seems to be a little more involved than just removing that line, but I still think the overall point I made in #610 is reasonable: If cli_parse_args is set in the model_config, and not in the constructor call to CliSettingsSource, the value from the model_config should be used, similar to other cli-related arguments.

In summary, I think reverting #611 is a good idea, to maintain backwards compatability, but I would like to have a look at other solutions to this "issue", if that's OK? :)

PS: I realize I may be on some thin ice here, as I'm not too familiar with the code base yet, so please correct where I'm making some bad assumptions.

EDIT: I created a PR as an alternative to solution to reverting #611 . Cf. #656