Rework AWS CDK code for the benchmarking cluster and add ballista to benchmarks

[gabotechs]

This PR:

• Refactors the code in order to better hold new query engines in the remote benchmarks
• Adds ballista to benchmarks

---

TPCH SF1 Ballista (prev) VS DataFusion Distributed (new)

  q1.sql: prev=1058 ms, new= 534 ms, 1.98x faster ✅
  q2.sql: prev=2410 ms, new= 561 ms, 4.30x faster ✅
  q3.sql: prev=2144 ms, new= 809 ms, 2.65x faster ✅
  q4.sql: prev=1539 ms, new= 379 ms, 4.06x faster ✅
  q5.sql: prev=2750 ms, new= 621 ms, 4.43x faster ✅
  q6.sql: prev= 773 ms, new= 395 ms, 1.96x faster ✅
  q7.sql: prev=2630 ms, new= 747 ms, 3.52x faster ✅
  q8.sql: prev=3425 ms, new=1012 ms, 3.38x faster ✅
  q9.sql: prev=3054 ms, new=1028 ms, 2.97x faster ✅
 q10.sql: prev=2367 ms, new= 513 ms, 4.61x faster ✅
 q11.sql: prev=1678 ms, new= 417 ms, 4.02x faster ✅
 q12.sql: prev=1455 ms, new= 432 ms, 3.37x faster ✅
 q13.sql: prev=1280 ms, new= 452 ms, 2.83x faster ✅
 q14.sql: prev=1140 ms, new= 449 ms, 2.54x faster ✅
 q15.sql: prev=1754 ms, new= 675 ms, 2.60x faster ✅
 q16.sql: prev=1930 ms, new= 328 ms, 5.88x faster ✅
 q17.sql: prev=1614 ms, new= 864 ms, 1.87x faster ✅
 q18.sql: prev=2073 ms, new= 591 ms, 3.51x faster ✅
 q19.sql: prev=1256 ms, new= 528 ms, 2.38x faster ✅
 q20.sql: prev=1901 ms, new= 709 ms, 2.68x faster ✅
 q21.sql: prev=2508 ms, new=1263 ms, 1.99x faster ✅
 q22.sql: prev=1282 ms, new= 327 ms, 3.92x faster ✅

TPCH SF10 Ballista (prev) VS DataFusion Distributed (new)

  q1.sql: prev=2857 ms, new=2896 ms, 1.01x slower ✖
  q2.sql: prev=4327 ms, new=1298 ms, 3.33x faster ✅
  q3.sql: prev=4306 ms, new=2302 ms, 1.87x faster ✅
  q4.sql: prev=2389 ms, new=1998 ms, 1.20x faster ✔
  q5.sql: prev=6682 ms, new=3707 ms, 1.80x faster ✅
  q6.sql: prev=1369 ms, new=1332 ms, 1.03x faster ✔
  q7.sql: prev=9022 ms, new=4351 ms, 2.07x faster ✅
  q8.sql: prev=8147 ms, new=3720 ms, 2.19x faster ✅
  q9.sql: prev=9715 ms, new=5664 ms, 1.72x faster ✅
 q10.sql: prev=5563 ms, new=2649 ms, 2.10x faster ✅
 q11.sql: prev=2945 ms, new=1331 ms, 2.21x faster ✅
 q12.sql: prev=3007 ms, new=1395 ms, 2.16x faster ✅
 q13.sql: prev=2538 ms, new=1881 ms, 1.35x faster ✅
 q14.sql: prev=2164 ms, new=1410 ms, 1.53x faster ✅
 q15.sql: prev=3398 ms, new=1966 ms, 1.73x faster ✅
 q16.sql: prev=2330 ms, new= 785 ms, 2.97x faster ✅
 q17.sql: prev=7066 ms, new=4540 ms, 1.56x faster ✅
 q18.sql: prev=7958 ms, new=4136 ms, 1.92x faster ✅
 q19.sql: prev=2389 ms, new=1687 ms, 1.42x faster ✅
 q20.sql: prev=3142 ms, new=1942 ms, 1.62x faster ✅
 q21.sql: prev=9751 ms, new=7852 ms, 1.24x faster ✅
 q22.sql: prev=1958 ms, new= 697 ms, 2.81x faster ✅

Clickbench partitioned 100 files Ballista (prev) VS DataFusion Distributed (new)

  q0.sql: prev= 398 ms, new= 140 ms, 2.84x faster ✅
  q1.sql: prev=1398 ms, new= 824 ms, 1.70x faster ✅
  q2.sql: prev=1619 ms, new=1120 ms, 1.45x faster ✅
  q3.sql: prev=1612 ms, new=1178 ms, 1.37x faster ✅
  q4.sql: prev=2581 ms, new=1818 ms, 1.42x faster ✅
  q5.sql: prev=3735 ms, new=2601 ms, 1.44x faster ✅
  q6.sql: prev= 442 ms, new= 143 ms, 3.09x faster ✅
  q7.sql: prev=1454 ms, new= 849 ms, 1.71x faster ✅
  q8.sql: prev=3864 ms, new=2338 ms, 1.65x faster ✅
  q9.sql: prev=3424 ms, new=2628 ms, 1.30x faster ✅
 q10.sql: prev=2370 ms, new=1373 ms, 1.73x faster ✅
 q11.sql: prev=2289 ms, new=1378 ms, 1.66x faster ✅
 q12.sql: prev=3836 ms, new=2815 ms, 1.36x faster ✅
 q13.sql: prev=5125 ms, new=5314 ms, 1.04x slower ✖
 q14.sql: prev=4131 ms, new=2525 ms, 1.64x faster ✅
 q15.sql: prev=3264 ms, new=2288 ms, 1.43x faster ✅
 q16.sql: prev=5331 ms, new=4085 ms, 1.31x faster ✅
 q17.sql: prev=5866 ms, new=3753 ms, 1.56x faster ✅
 q18.sql: prev=8507 ms, new=7133 ms, 1.19x faster ✔
 q19.sql: prev=1251 ms, new=1066 ms, 1.17x faster ✔
 q20.sql: prev=5839 ms, new=5865 ms, 1.00x slower ✖
 q21.sql: prev=6133 ms, new=5967 ms, 1.03x faster ✔
 q22.sql: prev=10154 ms, new=9672 ms, 1.05x faster ✔
 q23.sql: prev=23440 ms, new=24130 ms, 1.01x slower ✖
 q24.sql: prev=2134 ms, new=1990 ms, 1.07x faster ✔
 q25.sql: prev=1824 ms, new=1593 ms, 1.15x faster ✔
 q26.sql: prev=2248 ms, new=2030 ms, 1.11x faster ✔
 q27.sql: prev=6449 ms, new=6503 ms, 1.01x slower ✖
 q28.sql: prev=22974 ms, new=26314 ms, 1.15x slower ✖
 q29.sql: prev=1960 ms, new=1792 ms, 1.09x faster ✔
 q30.sql: prev=3358 ms, new=2627 ms, 1.28x faster ✅
 q31.sql: prev=3334 ms, new=3134 ms, 1.06x faster ✔
 q32.sql: prev=7403 ms, new=8373 ms, 1.13x slower ✖
 q33.sql: prev=12352 ms, new=9336 ms, 1.32x faster ✅
 q34.sql: prev=13255 ms, new=9108 ms, 1.46x faster ✅
 q35.sql: prev=3846 ms, new=2599 ms, 1.48x faster ✅
 q36.sql: prev=1284 ms, new= 791 ms, 1.62x faster ✅
 q37.sql: prev= 919 ms, new= 496 ms, 1.85x faster ✅
 q38.sql: prev=1291 ms, new= 742 ms, 1.74x faster ✅
 q39.sql: prev=1659 ms, new=1186 ms, 1.40x faster ✅
 q40.sql: prev= 955 ms, new= 451 ms, 2.12x faster ✅
 q41.sql: prev= 972 ms, new= 444 ms, 2.19x faster ✅
 q42.sql: prev= 958 ms, new= 417 ms, 2.30x faster ✅

[NGA-TRAN]

Wow, great numbers. Congrats again!