Skip to content

Commit 6afe474

Browse files
refactor: Eliminate code duplication in expand_record method
Unified the wildcard and non-wildcard branches by collecting all arrays to process into a single list, then using one common loop for expansion. This eliminates the duplicated item iteration and record expansion logic. All 29 tests passing. MyPy check passes. Co-Authored-By: unknown <>
1 parent c6448e5 commit 6afe474

File tree

1 file changed

+12
-20
lines changed

1 file changed

+12
-20
lines changed

airbyte_cdk/sources/declarative/expanders/record_expander.py

Lines changed: 12 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -68,35 +68,27 @@ def expand_record(self, record: MutableMapping[Any, Any]) -> Iterable[MutableMap
6868

6969
expand_path = [path.eval(self.config) for path in self._expand_path]
7070

71+
arrays: List[List[Any]] = []
72+
7173
if "*" in expand_path:
7274
matches = cast(Iterable[Any], dpath.values(record, expand_path))
73-
list_nodes = [m for m in matches if isinstance(m, list)]
74-
if not list_nodes:
75-
return
76-
77-
for nested_array in list_nodes:
78-
if len(nested_array) == 0:
79-
continue
80-
for item in nested_array:
81-
if isinstance(item, dict):
82-
expanded_record = dict(item)
83-
if self.remain_original_record:
84-
expanded_record["original_record"] = record
85-
yield expanded_record
86-
else:
87-
yield item
75+
arrays = [m for m in matches if isinstance(m, list)]
8876
else:
8977
try:
90-
nested_array = cast(Any, dpath.get(record, expand_path))
78+
nested = cast(Any, dpath.get(record, expand_path))
9179
except KeyError:
9280
return
93-
94-
if not isinstance(nested_array, list):
81+
if isinstance(nested, list):
82+
arrays = [nested]
83+
else:
9584
return
9685

97-
if len(nested_array) == 0:
98-
return
86+
if not arrays:
87+
return
9988

89+
for nested_array in arrays:
90+
if not nested_array:
91+
continue
10092
for item in nested_array:
10193
if isinstance(item, dict):
10294
expanded_record = dict(item)

0 commit comments

Comments
 (0)