[Iceberg AddFiles] Use thread pool to process incoming datafiles (#38018)

* use thread pool to process incoming datafiles

* trigger ITs

* add drain logic if too many files in queue; synchronize table

* compile fix

* fix tests

* fix tests

Author: ahmedabu98

.github/trigger_files/IO_Iceberg_Integration_Tests.json

@@ -1,4 +1,4 @@
 {
     "comment": "Modify this file in a trivial way to cause this test suite to run.",
-    "modification": 2
+    "modification": 1
 }