Getting timeout from source

[fabianboerner]

Hi, we currently have a problem that the oplog feature from airbyte is causing a great issue.

The data sets are in oplog in our older application because the date is regulary refreshed and with more customers the oplog stack gets very high.

`Sync Failed

308.85 MB
|
5,279 records extracted
|
no records loaded
|
Job id: 66
|
51m 38s
Failure Origin: source, Message: Something went wrong within the source connector`

This is why the source connector is running into a timeout transfering the data internally

2024-07-28 18:37:55 replication-orchestrator > failures: [ { "failureOrigin" : "source", "internalMessage" : "Source process exited with non-zero exit code 3", "externalMessage" : "Something went wrong within the source connector", "metadata" : { "attemptNumber" : 3, "jobId" : 66, "connector_command" : "read" }, "stacktrace" : "io.airbyte.workers.internal.exception.SourceException: Source process exited with non-zero exit code 3\n\tat io.airbyte.workers.general.BufferedReplicationWorker.readFromSource(BufferedReplicationWorker.java:390)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithHeartbeatCheck$3(BufferedReplicationWorker.java:259)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n", "timestamp" : 1722191867529 } ]

we are not sure what we can do here