fix: [iceberg] Remove IcebergFileStream and use iceberg-rust's parallelization#3051

Open

mbutrovich wants to merge 12 commits intoapache:mainfrom

mbutrovich:more_more_iceberg_file_stream

Commits on Jan 7, 2026

Commits on Jan 9, 2026

Set DataFusion's target_partitions to match Spark's spark.task.cpus.
mbutrovich
committed
Merge branch 'task_cpus' into more_more_iceberg_file_stream
mbutrovich
committed
Set Iceberg-Rust's parallelism to match DataFusion's target_partitions, which in turn is set to match Spark's spark.task.cpus.
mbutrovich
committed

Commits on Jan 10, 2026