[Bug] On programatic invocations CLI params with type checks cause failures

[dumkydewilde]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

On programatic invocations, i.e. using dbtRunner.invoke(), the runner fails when e.g. a profiles_dir is specified. This appears to be because, when this is provided as a keyword argument (e.g. dbtRunner.invoke(["build"], profiles_dir='./my_profiles') the kwargs are only passed to the context after creation, meaning any validation on creating the context fails. And this is exactly what happens, because the profiles_dir option checks for type=click.Path(exists=True).

Why is this annoying? Because everytime you run this in a Docker container (or e.g. through an orchestrator) it will fall back to the default directory for this option (~/username/.dbt) which it of course doesn't find.

Expected Behavior

Any additional parameters provided as kwargs to dbtRunner.invoke() should be added (and validated) on creation of the Click context.

Steps To Reproduce

1. Make sure the default directory ~/YourName/.dbt does not exists.
2. Run the following

dbtRunner.invoke(["build"], profiles_dir='./my_profiles')

Relevant log output

Environment

- OS: MacOS
- Python: 3.12
- dbt: 1.9

Which database adapter are you using with dbt?

No response

Additional Context

On this line the context is created, but params are only set after