[Feature] Add the possibility to reschedule automatically a sync if Airbyte server isn't capable to execute it at the precised schedule

[Dynnammo]

What area the feature impact?

Airbyte Platform

Relevant Information

Disclaimer : quite newby on Airbyte, however I know how OSS works, therefore no problem for closing/reformulating this issue. I wrote all alternative solutions below, I'm however posting this since I think it may be relevant if any other folks are facing similar need.

Context

As a data engineer, I have about hundred connections that are running every day. However, since the capacity on which my Airbyte server is hosted, it often happens that syncs that are running simultaneously are too greedy in terms of CPU usage and RAM, making them break.

Solution

Having the possibility to delay a sync if it has failed .
As instance, I have Connection 1 and Connection 2 running at 8:00 UTC, and because these are heavy syncs, the sync of connection 2 is rescheduled after connection 1 has finished run

Alternatives

• Manually setup connections to sync in order not to "overflow" on each other
• Change ways of syncing in order to have less "greedy" syncs (currently, we're on the Full Refresh | Overwrite mode, which is efficient but not adapted for regular syncs)