pydantic-settings fails to handle unions with defaults; raises Attribute error

[klieret]

Thank you for this wonderful addition to pydantic.

However, the following snippet

from pydantic_settings import BaseSettings, CliApp


class Cat(BaseSettings):
    meow: str = "meow"
    
class Dog(BaseSettings):
    bark: str = "bark"
    

class Car(BaseSettings):
    driver: Cat | Dog = Cat()


CliApp.run(Car)

raises

Traceback (most recent call last):
  File "/Users/fuchur/Documents/24/git_sync/config-comparison/union_type_issues.py", line 14, in <module>
    CliApp.run(Car)
  File "/opt/miniconda3/envs/confexp/lib/python3.11/site-packages/pydantic_settings/main.py", line 516, in run
    return CliApp._run_cli_cmd(model_cls(**model_init_data), cli_cmd_method_name, is_required=False)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/confexp/lib/python3.11/site-packages/pydantic_settings/main.py", line 168, in __init__
    **__pydantic_self__._settings_build_values(
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/confexp/lib/python3.11/site-packages/pydantic_settings/main.py", line 362, in _settings_build_values
    cli_settings = CliSettingsSource[Any](
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/confexp/lib/python3.11/typing.py", line 1289, in __call__
    result = self.__origin__(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/confexp/lib/python3.11/site-packages/pydantic_settings/sources.py", line 1163, in __init__
    self._connect_root_parser(
  File "/opt/miniconda3/envs/confexp/lib/python3.11/site-packages/pydantic_settings/sources.py", line 1572, in _connect_root_parser
    self._add_parser_args(
  File "/opt/miniconda3/envs/confexp/lib/python3.11/site-packages/pydantic_settings/sources.py", line 1685, in _add_parser_args
    self._add_parser_submodels(
  File "/opt/miniconda3/envs/confexp/lib/python3.11/site-packages/pydantic_settings/sources.py", line 1787, in _add_parser_submodels
    self._add_parser_args(
  File "/opt/miniconda3/envs/confexp/lib/python3.11/site-packages/pydantic_settings/sources.py", line 1653, in _add_parser_args
    kwargs['help'] = self._help_format(field_name, field_info, model_default)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/confexp/lib/python3.11/site-packages/pydantic_settings/sources.py", line 1898, in _help_format
    default = f'(default: {getattr(model_default, field_name)})'
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/confexp/lib/python3.11/site-packages/pydantic/main.py", line 856, in __getattr__
    raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
AttributeError: 'Cat' object has no attribute 'bark'

[klieret]

The following works:

...

class Car(BaseSettings):
    driver: Cat | Dog


CliApp.run(Car, ["--driver.meow=meow"])

So this has to do with default arguments of union types (it makes sense that it's not trivial to provide a list of available arguments in that case, but it still shouldn't error...)

[klieret]

A workaround seems to be to use a default_factory like so:

class Car(BaseSettings):
    driver: Cat | Dog = Field(default_factory=Cat)

This feels pretty normal coming from dataclasses, but pydantic states that mutable defaults are ok, or did I misunderstand this?

[hramezani]

Thanks @klieret for reporting the issue.

@kschwab can we provide the default value at the line that we have an error in getattr to fix the error?

[kschwab]

@hramezani no, it's slightly more complicated. We have to ignore the model default that was propagated down the tree. i.e., at the point of failure it's too late for correction, it has to be resolved further up. e.g., "purr" has to be cascaded down the tree to in order to override the Cat default "meow":

from pydantic_settings import BaseSettings, CliApp


class Cat(BaseSettings):
    meow: str = "meow"

class Dog(BaseSettings):
    bark: str = "bark"

class Car(BaseSettings):
    driver: Cat | Dog = Cat(meow='purr')


CliApp.run(Car, cli_args=['--help'])
"""
usage: example.py [-h] [--driver JSON] [--driver.meow str] [--driver.bark str]

options:
  -h, --help         show this help message and exit

driver options:
  --driver JSON      set driver from JSON string
  --driver.meow str  (default: purr)
  --driver.bark str  (default: bark)
"""

The issue occurs when applying defaults from Cat(meow='purr') onto Dog which doesn't apply, so we have to stop the propagation of Cat onto Dog if that makes sense.

I opened PR #472 for resolution.

[hramezani]

Thanks @kschwab for the PR.

@klieret can you confirm the fix?

[klieret]

Won't have time to check it out this week, but the tests in @kschwab 's PR are great, so if they pass, this should be resolved.

Thank you for the lightning fast fix! @kschwab @hramezani ❤️