Comet writer doesn't respect OVERWRITE mode #2970
Description
Describe the bug
Comet fails if the folder exists and cannot overwrite it
Steps to reproduce
Folder output2 exists
spark.read.parquet("/tmp/writertest/input").repartition(1).write.mode("overwrite").parquet("/tmp/writertest/output2")
25/12/23 08:31:40 ERROR CometExecIterator: Native execution for task 17 failed
org.apache.comet.CometNativeException: Failed to create output file '/tmp/writertest/output2/part-00000-00017.parquet': Is a directory (os error 21)
Expected behavior
Should overwrite the folder with new content
Additional context
No response