Fix: convert kwargs to cli params, add aliases#12029
Open
dumkydewilde wants to merge 1 commit intodbt-labs:mainfrom
Open
Fix: convert kwargs to cli params, add aliases#12029dumkydewilde wants to merge 1 commit intodbt-labs:mainfrom
dumkydewilde wants to merge 1 commit intodbt-labs:mainfrom
Conversation
|
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR. CLA has not been signed by users: @dumkydewilde |
2 tasks
Contributor
|
@cla-bot check |
|
The cla-bot has been summoned, and re-checked this pull request! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #12028
Problem
On programatic invocations, i.e. using dbtRunner.invoke(), the runner fails when e.g. a profiles_dir is specified. This appears to be because, when this is provided as a keyword argument (e.g. dbtRunner.invoke(["build"], profiles_dir='./my_profiles') the kwargs are only passed to the context after creation, meaning any validation on creating the context fails. And this is exactly what happens, because the profiles_dir option checks for type=click.Path(exists=True).
Why is this annoying? Because everytime you run this in a Docker container (or e.g. through an orchestrator) it will fall back to the default directory for this option (~/username/.dbt) which it of course doesn't find.
Solution
Params can not be added to the Click context before it is created, which means they have to be added as string arguments.
flags.pyalready provides functionality for this with thecommand_params()function. What we can do is take the kwargs through this function, and combine them with provided args so that params can be set before the context creation and validation happens.Checklist