[TRACKING] DataFusion issues raised by Ballista

[milenkovicm]

Tracking tasks with DataFusion issues which ballista benefits from:

DataFusion <46

• LogicalPlan serde is not yet implemented for Dml datafusion#13616
• Preserve session context id when ctx.enable_url_table() datafusion#14003
• feat: add TableSource to DML proto to eliminate need for table lookup after deserialisation datafusion#14654

DataFusion 46

• bug: Physical plan round trip fails in some cases after datasource refactor datafusion#14679

DataFusion 47

• feat: Support serde for JsonSource PhysicalPlan datafusion#15311
• stack overflow on PhysicalPlanNode::try_from_physical_plan datafusion#15087

DataFusion 48

• Partitioned ListingTable read fails after logical plan ser/de datafusion#15718

DataFusion 50

• Support SortMergeJoin proto serde datafusion#17292
• feat: FunctionRegistry should expose methods to return names of udaf and udwf datafusion#17649

DataFusion 51

• SessionContext propagation in physical plan proto functions datafusion#17596

DataFusion 52

• DataFrame.cache() does not work in distributed environments datafusion#17297

Open Issues

Raised issues which have no fix version yet

• make DefaultLogicalExtensionCodec support serialisation of build in file formats datafusion#16944
• MetricValue::Custom encoding\decoding support datafusion#16945

It's getting harder and harder to track what is going to get supported in which version