[MAINTENANCE] Avoid unnecessary _get_default_value() calls for non-field keys #11626
+131
−8
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Skip calling
_get_default_value()for keys that aren't Pydantic fields in_get_domain_kwargs(),_get_success_kwargs(), and_get_runtime_kwargs().Problem
During checkpoint validation, GX logs a noisy INFO message repeatedly:
For a simple 5-expectation suite, this message fires 6 times for a single
ExpectColumnToExistexpectation - the only expectation with"table"in itsdomain_keys.The root cause:
_get_domain_kwargs()iterates over alldomain_keysand calls_get_default_value()for each key not inconfiguration.kwargs. For keys like"table"that are valid domain keys but not Pydantic fields,_get_default_value()logs the message and returnsNone.Solution
Add an inline conditional to only call
_get_default_value()when the key is actually a Pydantic field:This preserves the existing behavior (returning
Nonefor non-field keys) while avoiding the unnecessary log message.Additionally, adds a
_get_success_kwarg(key, default=None)single-key accessor method for callers that only need one or two specific values, avoiding the overhead of building the full merged dictionary.Changes
great_expectations/expectations/expectation.py:_get_domain_kwargs(),_get_success_kwargs(),_get_runtime_kwargs()_get_success_kwarg()methodColumnMapExpectation._validate(),ColumnPairMapExpectation._validate(),MulticolumnMapExpectation._validate()to use_get_success_kwarg()great_expectations/expectations/core/expect_column_to_exist.py:_validate()to use_get_success_kwarg()tests/expectations/test_expectation.py:TestGetSuccessKwargclass with parametrized testsTestKwargsMethodsNoLoggingForNonFieldsclass to verify no logging for non-field keysTest Plan
_get_success_kwarg()(9 parametrized tests)