Potential Stall in Continuous Run: Estimated Remaining Time Increasing (Run e297014e)

[HassanSharara]

Hello TechEmpower Team,

I am monitoring the current continuous benchmarking run (ID: e297014e-de7d-4340-bf30-417907caf4b6) and it appears the run might be stuck or experiencing significant delays.

Observation:
The estimated remaining time has been fluctuating inconsistently. Despite the run being active for over 3 days, the "Estimated remaining" time is not decreasing as expected. Instead, it has increased from approximately 14 hours to over 17 hours recently.

Details:

Run URL: https://tfb-status.techempower.com/results/e297014e-de7d-4340-bf30-417907caf4b6

Status: The run seems to be progressing through frameworks, but the throughput or the cleanup phase for certain tests might be hanging, causing the scheduler to push back the completion estimate.

Issue: Estimated time jumped from ~14h to 17h+ while the run was active.

Has there been a known issue with the environment or a specific framework in this batch that is causing a bottleneck? It might be worth checking if a specific container is failing to terminate properly or if the load generator is hitting a timeout loop.

Looking forward to your insights.

Best regards,