Skip to content

Commit dfc8175

Browse files
committed
[SPARK-52077][PYTHON][CONNECT][TEST] Skip ArrowUDTFParityTests in Spark Connect compatibility test
### What changes were proposed in this pull request? This PR proposes to skip ArrowUDTFParityTests in Spark Connect compatibility test for now. ### Why are the changes needed? After #50099, the compatibility test fails https://github.com/apache/spark/actions/runs/14959668798/job/42019945629 In fact, UDTF with Arrow is still under development so we can skip the tests for now ### Does this PR introduce _any_ user-facing change? No, test-only. ### How was this patch tested? Will monitor the build. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #50856 from HyukjinKwon/SPARK-44856-followup. Authored-by: Hyukjin Kwon <gurwls223@apache.org> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
1 parent dcac3f5 commit dfc8175

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

python/pyspark/sql/tests/connect/test_parity_udtf.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
# See the License for the specific language governing permissions and
1515
# limitations under the License.
1616
#
17+
import os
1718
import unittest
1819

1920
from pyspark.testing.connectutils import should_test_connect
@@ -122,6 +123,10 @@ def eval(self):
122123
TestUDTF().collect()
123124

124125

126+
@unittest.skipIf(
127+
os.environ.get("SPARK_SKIP_CONNECT_COMPAT_TESTS") == "1",
128+
"Python UDTF with Arrow is still under development.",
129+
)
125130
class ArrowUDTFParityTests(UDTFArrowTestsMixin, UDTFParityTests):
126131
@classmethod
127132
def setUpClass(cls):

0 commit comments

Comments
 (0)