[SPARK-53387][PYTHON] Add support for Arrow UDTFs with PARTITION BY

### What changes were proposed in this pull request?

This PR adds support for Arrow UDTFs with PARTITION BY functionality. It adds a new `ArrowUDTFWithPartition` wrapper in worker.py that handle partition by semantics. It handles partitioning logic specifically for Arrow UDTFs that receive PyArrow RecordBatch objects (table argument) and efficiently detects partition boundaries.

### Why are the changes needed?

To support PARTITION BY for Arrow Python UDTFs.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

New unit tests in test_arrow_udtfs

### Was this patch authored or co-authored using generative AI tooling?

Yes

Closes #52317 from allisonwang-db/spark-53387-partition-by-udtf.

Authored-by: Allison Wang <[email protected]>
Signed-off-by: Allison Wang <[email protected]>

Author: allisonwang-db

python/pyspark/errors/error-conditions.json

@@ -384,6 +384,11 @@
       "The return type of the arrow-optimized Python UDTF should be of type 'pandas.DataFrame', but the '<func>' method returned a value of type <return_type> with value: <value>."
     ]
   },
+  "INVALID_ARROW_UDTF_TABLE_ARGUMENT": {
+    "message": [
+      "Arrow UDTFs with PARTITION BY must have a TABLE argument that results in a PyArrow RecordBatch, but got <actual_type>."
+    ]
+  },
   "INVALID_ARROW_UDTF_WITH_ANALYZE": {
     "message": [
       "The arrow UDTF '<name>' is invalid. Arrow UDTFs do not support the 'analyze' method. Please remove the 'analyze' method from '<name>' and specify a returnType instead."