Iceberg's parquets do not have field_id (schema evolution is broken)

[4ertus2]

Backend

VL (Velox)

Bug description

Iceberg spec requires field_ids are set:

Column IDs are required to be stored as field IDs on the parquet schema.

As I could understand the actual column ids from Iceberg schema are not passed here. So they cannot be written in Velox later.

incubator-gluten/cpp/velox/compute/VeloxRuntime.cc

Line 232 in 2ec3ba7

return std::make_shared<IcebergWriter>(

It looks like it would be possible to pass the ids in Velox part through IcebergColumnHandle after this PR

Am I right that there's no info about actual Iceberg column_ids in Java_org_apache_gluten_execution_IcebergWriteJniWrapper_init right now?

Gluten version

main branch

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs