Inconsistencies between `RecordBatch` and `DataFrame` schemas cause `to_arrow_table` to fail

[nuno-faria]

Describe the bug

When the nullability of a RecordBatch column does not match with the DataFrame's schema, the conversion to a pyarrow table fails.

To Reproduce

from datafusion import SessionContext

ctx = SessionContext()
ctx.sql("create table t_(a int not null)").collect()
ctx.sql("insert into t_ values (1), (2), (3)").collect()
ctx.sql(f"copy (select * from t_) to 't.parquet'").collect()
ctx.register_parquet("t", path)
pyarrow_table = ctx.sql("select max(a) as m from t").to_arrow_table()

...
pyarrow.lib.ArrowInvalid: Schema at index 0 was different: 
m: int32
vs
m: int32 not null

Expected behavior
Execute without crashing.

Additional context

[IshaGudewar]

Thanks for opening this issue and the fix!
I reviewed the problem and the proposed solution in PR #1315, and I can confirm that the root cause is the schema mismatch between:

the DataFrame schema (which may mark fields as NOT NULL), and
the RecordBatch schema (which often marks aggregated columns as nullable).

PyArrow requires all RecordBatch schemas to match exactly, so this mismatch causes:

ArrowInvalid: Schema at index 0 was different

To help strengthen the PR, I will add:

1. A regression test for the zero-record-batch case, verifying that when no batches are returned, to_arrow_table() still uses the DataFrame schema.

2. A test for an aggregation that returns NULL, such as max(a) on an empty table, confirming the output is [None] and schema nullability is handled correctly.

I will contribute these tests to PR #1315 shortly.

[nuno-faria]

Thanks @IshaGudewar, but those tests have already been added.