Skip to content

Commit 20af8bd

Browse files
devin-petersohnHyukjinKwon
authored andcommitted
[SPARK-54787][PS] Use list comprehension in pandas _bool_column_labels
### What changes were proposed in this pull request? Use list comprehension in the pandas.DataFrame method _bool_column_labels. This will modestly improve memory and performance, but also reduces code to a single line. ### Why are the changes needed? For mantainability and performance ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI ### Was this patch authored or co-authored using generative AI tooling? No Closes #53550 from devin-petersohn/devin/pandas_maintain_01. Authored-by: Devin Petersohn <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent 27c5100 commit 20af8bd

File tree

1 file changed

+3
-9
lines changed

1 file changed

+3
-9
lines changed

python/pyspark/pandas/frame.py

Lines changed: 3 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11268,15 +11268,9 @@ def _bool_column_labels(self, column_labels: List[Label]) -> List[Label]:
1126811268
"""
1126911269
Filter column labels of boolean columns (without None).
1127011270
"""
11271-
bool_column_labels = []
11272-
for label in column_labels:
11273-
psser = self._psser_for(label)
11274-
if is_bool_dtype(psser):
11275-
# Rely on dtype rather than spark type because
11276-
# columns that consist of bools and Nones should be excluded
11277-
# if bool_only is True
11278-
bool_column_labels.append(label)
11279-
return bool_column_labels
11271+
# Rely on dtype rather than spark type because columns that consist of bools and
11272+
# Nones should be excluded if bool_only is True
11273+
return [label for label in column_labels if is_bool_dtype(self._psser_for(label))]
1128011274

1128111275
def _result_aggregated(
1128211276
self, column_labels: List[Label], scols: Sequence[PySparkColumn]

0 commit comments

Comments
 (0)