Skip to content

Commit 8860509

Browse files
committed
[SPARK-53610][PYTHON][TESTS][FOLLOW-UP] Add tests for negative maxRecordsPerBatch
### What changes were proposed in this pull request? Add tests for negative maxRecordsPerBatch ### Why are the changes needed? to improve test coverage ### Does this PR introduce _any_ user-facing change? no, test-only ### How was this patch tested? CI ### Was this patch authored or co-authored using generative AI tooling? No Closes #52775 from zhengruifeng/cogroup_followup. Authored-by: Ruifeng Zheng <ruifengz@apache.org> Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>
1 parent d14c355 commit 8860509

File tree

2 files changed

+10
-0
lines changed

2 files changed

+10
-0
lines changed

python/pyspark/sql/tests/arrow/test_arrow_cogrouped_map.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -362,6 +362,11 @@ def summarize(key, left, right):
362362

363363
self.assertEqual(expected, result)
364364

365+
def test_negative_and_zero_batch_size(self):
366+
for batch_size in [0, -1]:
367+
with self.sql_conf({"spark.sql.execution.arrow.maxRecordsPerBatch": batch_size}):
368+
CogroupedMapInArrowTestsMixin.test_apply_in_arrow(self)
369+
365370

366371
class CogroupedMapInArrowTests(CogroupedMapInArrowTestsMixin, ReusedSQLTestCase):
367372
@classmethod

python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -709,6 +709,11 @@ def summarize(key, left, right):
709709

710710
self.assertEqual(expected, result)
711711

712+
def test_negative_and_zero_batch_size(self):
713+
for batch_size in [0, -1]:
714+
with self.sql_conf({"spark.sql.execution.arrow.maxRecordsPerBatch": batch_size}):
715+
CogroupedApplyInPandasTestsMixin.test_with_key_right(self)
716+
712717

713718
class CogroupedApplyInPandasTests(CogroupedApplyInPandasTestsMixin, ReusedSQLTestCase):
714719
pass

0 commit comments

Comments
 (0)