You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/docs/08-workflow-troubleshooting/01-timeouts.md
+10-10Lines changed: 10 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -27,7 +27,7 @@ Optionally you can also increase the number of pollers per worker by providing t
27
27
[Link to options in go client](https://pkg.go.dev/go.uber.org/[email protected]/internal#WorkerOptions)
28
28
[Link to options in java client](https://github.com/uber/cadence-java-client/blob/master/src/main/java/com/uber/cadence/internal/worker/PollerOptions.java#L124)
29
29
30
-
## Timeouts without heartbeat timeout or retry policy configured
30
+
## No heartbeat timeout or retry policy configured
31
31
32
32
Activities time out StartToClose or ScheduleToClose if the activity took longer than the configured timeout.
33
33
@@ -37,35 +37,35 @@ For long running activities, while the activity is executing, the worker can die
37
37
38
38
Mitigation: Consider configuring heartbeat timeout and a retry policy
[Check retry policy for activity](https://cadenceworkflow.io/docs/concepts/activities/#retries)
42
42
43
43
For short running activities, heart beating is not required but maybe consider increasing the timeout value to suit the actual activity execution time.
44
44
45
-
## Timeouts without heartbeat timeout configured but a retry policy configured
45
+
## Retry policy configured without setting heartbeat timeout
46
46
47
-
Retry policies are good to be configured so that activities can be retried after timeouts or failures. For long running activities, while the activity is executing, the worker can die due to regular deployments or host restarts or failures. Cadence doesn't know about this and will wait for StartToClose or ScheduleToClose timeouts to kick in. The retry is attempted only after this timeout. Enabling heartbeating would cause the activity to timeout earlier and will be retried on another worker.
47
+
Retry policies are good to be configured so that activities can be retried after timeouts or failures. For long running activities, while the activity is executing, the worker can die due to regular deployments or host restarts or failures. Cadence doesn't know about this and will wait for StartToClose or ScheduleToClose timeouts to kick in. The retry is attempted only after this timeout. Configuring heartbeat timeout would cause the activity to timeout earlier and will be retried on another worker.
## Timeouts with heartbeating enabled but without a retry policy configured
53
+
## Heartbeat timeout configured without a retry policy
54
54
55
-
Heartbeat timeouts are used to detect when a worker died or restarted during deployments. With heartbeat timeout enabled, the activity will timeout faster. But without a retry policy, it will not be scheduled again on a healthy worker.
55
+
Heartbeat timeouts are used to detect when a worker died or restarted during deployments. With heartbeat timeout configured, the activity will timeout faster. But without a retry policy, it will not be scheduled again on a healthy worker.
56
56
57
57
Mitigation: Consider adding retry policy to an activity
58
58
59
59
[Check retry policy for activity](https://cadenceworkflow.io/docs/concepts/activities/#retries)
60
60
61
-
## Heartbeat Timeouts after enabling heartbeating
61
+
## Heartbeat Timeouts after configuring heartbeat timeout
62
62
63
-
Activity has enabled heart beating but the activity timed out with heart beat timeout. This is because the server did not receive a heart beat in the time interval configured as the heart beat timeout.
63
+
Activity has configured heartbeat timeout and the activity timed out with heart beat timeout. This is because the server did not receive a heart beat in the time interval configured as the heart beat timeout.
64
64
65
65
Mitigation: Once heartbeat timeout is configured in activity options, you need to make sure the activity periodically sends a heart beat to the server to make sure the server is aware of the activity being alive.
66
66
67
67
[Example to send periodic heart beat](https://github.com/uber-common/cadence-samples/blob/df6f7bdba978d6565ad78e9f86d9cd31dfac9f78/cmd/samples/fileprocessing/activities.go#L111)
68
68
69
69
In go client, there is an option to register the activity with auto heart beating so that it is done automatically
70
70
71
-
[Enabling auto heart beat during activity registration example](https://pkg.go.dev/go.uber.org/[email protected]/internal#WorkerOptions)
71
+
[Configuring auto heart beat during activity registration example](https://pkg.go.dev/go.uber.org/[email protected]/internal#WorkerOptions)
0 commit comments