1
- # Java framework for Cadence [ ![ Build Status ] ( https://travis-ci.org/uber-java/cadence-client.svg?branch=master )] ( https://travis-ci.org/uber-java/cadence-client ) [ ![ Coverage Status ] ( https://coveralls.io/repos/uber-java/cadence-client/badge.svg?branch=master&service=github )] ( https://coveralls.io/github/uber-java/cadence-client?branch=master )
1
+ # Java framework for Cadence
2
2
[ Cadence] ( https://github.com/uber/cadence ) is a distributed, scalable, durable, and highly available orchestration engine we developed at Uber Engineering to execute asynchronous long-running business logic in a scalable and resilient way.
3
3
4
- ` cadence-client ` is the framework for authoring workflows and activities.
4
+ ` cadence-client ` is the framework for authoring workflows and activities in Java .
5
5
6
+ There is also [ Go Cadence Client] ( https://github.com/uber-go/cadence-client ) .
6
7
7
8
## Samples
8
9
9
- [ Samples for the Java Cadence client] ( https://github.com/mfateev/ uber-java- cadence-samples ) .
10
+ [ Samples for the Java Cadence client] ( https://github.com/uber-java/ cadence-client ) .
10
11
11
- ## Service Installation
12
+ ## Running Cadence Server
12
13
13
- For development install a local copy of Cadence service.
14
- The simplest is to use a [ docker version] ( https://github.com/uber/cadence/blob/master/docker/README.md ) .
15
- If for whatever reason docker is not an option follow instructions
16
- from the [ Cadence Getting Started] ( https://github.com/uber/cadence#getting-started )
14
+ Run Cadence Server using Docker Compose
17
15
18
- If you work for Uber install the Cadence CLI. Search 'Cadence CLI' on EngDocs for installation and usage instructions.
19
- For not so fortunate it will be open sourced very soon.
16
+ curl -O https://raw.githubusercontent.com/uber/cadence/master/docker/docker-compose.yml
17
+ docker-compose up
18
+
19
+ If it does not work see instructions for running the Cadence Server at https://github.com/uber/cadence/blob/master/README.md
20
+
21
+ ## Get CLI
22
+
23
+ TODO
20
24
21
25
# Build Configuration
22
26
@@ -38,16 +42,17 @@ activities and workflows.
38
42
39
43
40
44
## Cadence Terminology
45
+
41
46
- * Activity* is a business level task that implement your application logic like calling a service or transcoding a media file.
42
47
Usually it is expected that an activity implements a single well defined action. Activity can be both short and long running.
43
- It can be implemented as synchronous method or fully asynchronously involving multiple processes. Activity is executed * at most once* .
44
- It means that Cadence service never request activity execution more than once. If for any reason activity is not completed
48
+ It can be implemented as a synchronous method or fully asynchronously involving multiple processes. Activity is executed * at most once* .
49
+ It means that the Cadence service never requests activity execution more than once. If for any reason activity is not completed
45
50
within specified timeout an error is reported to the workflow and it decides how to handle it.
46
51
There is no limit on potential activity duration.
47
52
48
- - * Workflow* is a program that orchestrates activities. It has a full control which activities and in what order are executed.
53
+ - * Workflow* is a program that orchestrates activities. It has a full control over which activities and in what order are executed.
49
54
Workflow must not affect external world directly, only through activities. What makes workflow code "a workflow" is that its state
50
- is preserved by Cadence. So any failure of a worker process that hosts a workflow code does not affect workflow execution.
55
+ is preserved by Cadence. So any failure of a worker process that hosts the workflow code does not affect the workflow execution.
51
56
It continues as if these failures do not happen. At the same time activities can fail any moment for any reason.
52
57
But as workflow code is fully fault tolerant it is guaranteed to get notification about activity failure or timeout and
53
58
act accordingly. There is no limit on a potential workflow duration.
@@ -58,8 +63,8 @@ Usually it is expected that an activity implements a single well defined action.
58
63
- * Domain* is a namespace like concept. Any entity stored in Cadence is always stored in a specific domain. For example when
59
64
a workflow is started it is started in a specific domain. Cadence guarantees a workflow id uniqueness within a domain. Domains
60
65
are created and updated though a separate CRUD API or through CLI.
61
- - * Task List* When a workflow requests an execution of an activity behind the scene Cadence creates an * activity task * and puts it
62
- into a * Task List * which is essentially a queue persisted inside a Cadence service . Then a client side worker that implement the activity
66
+ - * Task List* is essentially a queue persisted inside a Cadence service. When a workflow requests an activity execution
67
+ Cadence service creates an * activity task * and puts it into a * Task List * . Then a client side worker that implement the activity
63
68
receives the task from the task list and invokes an activity implementation. For this to work task list name that is used to request an
64
69
activity execution and configure a worker should match.
65
70
- * Workflow ID* is a business level ID of a workflow execution (aka instance). Cadence guarantees uniqueness of an ID within a domain.
@@ -83,25 +88,23 @@ The only requirement is that activity method arguments and return values are ser
83
88
[ DataConverter] ( src/main/java/com/uber/cadence/converter/DataConverter.java ) interface. The default implementation uses
84
89
JSON serializer, but an alternative implementation can be easily configured.
85
90
91
+ Example of an interface that defines four activities:
86
92
``` Java
87
93
public interface FileProcessingActivities {
88
94
89
95
void upload (String bucketName , String localName , String targetName );
90
96
91
- /**
92
- * @return local name
93
- */
94
97
String download (String bucketName , String remoteName );
95
98
96
- /**
97
- * @return local name
98
- */
99
+ @ActivityMethod (scheduleToCloseTimeoutSeconds = 2 )
99
100
String processFile (String localName );
100
101
101
102
void deleteLocalFile (String fileName );
102
103
}
103
104
104
105
```
106
+ An optional @ActivityMethod annotation can be used to specify activity options like timeouts or task list. Required options
107
+ that are not specified through the annotation must be specified at run time.
105
108
106
109
## Activity Implementation
107
110
@@ -163,10 +166,11 @@ public class FileProcessingActivitiesImpl implements FileProcessingActivities {
163
166
...
164
167
}
165
168
```
169
+
166
170
### Asynchronous Activity Completion
167
171
168
- Sometimes activity lifecycle goes beyond a synchronous method invocation. For example a request can be put in a queue
169
- and later reply comes and picked up by a different worker process. The whole such request reply interaction can be modeled
172
+ Sometimes an activity lifecycle goes beyond a synchronous method invocation. For example a request can be put in a queue
173
+ and later a reply comes and picked up by a different worker process. The whole such request- reply interaction can be modeled
170
174
as a single Cadence activity.
171
175
172
176
To indicate that an activity should not be completed upon its method return annotate it with @DoNotCompleteOnReturn .
@@ -179,7 +183,7 @@ public class FileProcessingActivitiesImpl implements FileProcessingActivities {
179
183
public String download (String bucketName , String remoteName , String localName ) {
180
184
byte [] taskToken = Activity . getTaskToken(); // Used to correlate reply
181
185
asyncDownloadFileFromS3(taskToken, bucketName, remoteName, localDirectory + localName);
182
- return " ignored" ; // This value is ignored when annotated with @DoNotCompleteOnReturn
186
+ return " ignored" ; // Return value is ignored when annotated with @DoNotCompleteOnReturn
183
187
}
184
188
...
185
189
}
@@ -194,6 +198,7 @@ When download is complete the download service calls back potentially from a dif
194
198
completionClient. completeExceptionally(taskToken, failure);
195
199
}
196
200
```
201
+
197
202
### Activity Heartbeating
198
203
199
204
Some activities are potentially long running. To be able to react to their crashes quickly heartbeat mechanism is used.
@@ -225,18 +230,22 @@ public class FileProcessingActivitiesImpl implements FileProcessingActivities {
225
230
}
226
231
```
227
232
# Workflows
233
+
228
234
Workflow encapsulates orchestration of activities and child workflows.
229
235
It can also answer to synchronous queries and receive external events (aka signals).
236
+
230
237
## Workflow Interface
238
+
231
239
A workflow must define an interface class. All its methods must have one of the following annotations:
232
- - @WorkflowMethod indicates an entry point to a workflow
240
+ - @WorkflowMethod indicates an entry point to a workflow. It contains parameters like timeouts and task list. Required
241
+ parameters (like executionStartToCloseTimeoutSeconds) that are not specified through the annotation must be provided at runtime.
233
242
- @Signal indicates a method that reacts to external signals. Must have a ` void ` return type.
234
243
- @Query indicates a method that reacts to synchronous query requests.
235
244
It is possible to have more than method with the same annotation.
236
245
``` java
237
246
public interface FileProcessingWorkflow {
238
247
239
- @WorkflowMethod
248
+ @WorkflowMethod ( executionStartToCloseTimeoutSeconds = 10 , taskList = " file-processing " )
240
249
String processFile (Arguments args );
241
250
242
251
@QueryMethod (name = " history" )
@@ -255,13 +264,8 @@ Given a workflow interface executing a workflow requires initializing a `Workflo
255
264
a client side stub to the workflow and then calling a method annotated with @WorkflowMethod .
256
265
``` java
257
266
WorkflowClient workflowClient = WorkflowClient . newClient(cadenceServiceHost, cadenceServicePort, domain);
258
- // At least workflow timeout and task list to use are required.
259
- WorkflowOptions options = new WorkflowOptions .Builder ()
260
- .setExecutionStartToCloseTimeoutSeconds(300 )
261
- .setTaskList(WORKFLOW_TASK_LIST )
262
- .build();
263
267
// Create workflow stub
264
- FileProcessingWorkflow workflow = workflowClient. newWorkflowStub(FileProcessingWorkflow . class, options );
268
+ FileProcessingWorkflow workflow = workflowClient. newWorkflowStub(FileProcessingWorkflow . class);
265
269
```
266
270
There are two ways to start workflow execution. Synchronously and asynchronously. Synchronous invocation starts a workflow
267
271
and then waits for its completion. If process that started workflow crashes or stops waiting workflow continues execution.
@@ -306,8 +310,8 @@ can communicate with the other parts of the workflow through workflow object fie
306
310
### Calling Activities
307
311
308
312
` Workflow.newActivityStub ` returns a client side stub that implements an activity interface.
309
- It takes activity type and activity options as arguments. Activity options are needed to tell the Cadence service
310
- the required timeouts and which task list to use when dispatching a correspondent activity task to a worker .
313
+ It takes activity type and activity options as arguments. Activity options are needed only if some of the required
314
+ timeouts are not specified through @ ActivityMethod annotation .
311
315
312
316
Calling a method on this interface invokes an activity that implements this method.
313
317
An activity invocation synchronously blocks until the activity completes (or fails or times out). Even if activity
@@ -320,11 +324,7 @@ public class FileProcessingWorkflowImpl implements FileProcessingWorkflow {
320
324
private final FileProcessingActivities activities;
321
325
322
326
public FileProcessingWorkflowImpl () {
323
- // Options are required as there are no good defaults for the timeouts.
324
- ActivityOptions ao = new ActivityOptions .Builder ()
325
- .setScheduleToCloseTimeoutSeconds(300 )
326
- .build();
327
- this . store = Workflow . newActivityStub(FileProcessingActivities . class, ao);
327
+ this . store = Workflow . newActivityStub(FileProcessingActivities . class);
328
328
}
329
329
330
330
@Override
@@ -347,9 +347,24 @@ public class FileProcessingWorkflowImpl implements FileProcessingWorkflow {
347
347
...
348
348
}
349
349
```
350
- If different activities need different scheduling options just create multiple client side stubs with different options.
350
+ If different activities need different options (like timeouts or task list) multiple client side stubs could be created
351
+ with different options.
352
+
353
+ public FileProcessingWorkflowImpl() {
354
+ ActivityOptions options1 = new ActivityOptions.Builder()
355
+ .setTaskList("taskList1")
356
+ .build();
357
+ this.store1 = Workflow.newActivityStub(FileProcessingActivities.class, options1);
358
+
359
+ ActivityOptions options2 = new ActivityOptions.Builder()
360
+ .setTaskList("taskList2")
361
+ .build();
362
+ this.store2 = Workflow.newActivityStub(FileProcessingActivities.class, options2);
363
+ }
364
+
351
365
### Calling Activities Asynchronously
352
- Being able to call activity synchronously is great. But sometimes workflows need to perform certain operations in parallel.
366
+
367
+ Sometimes workflows need to perform certain operations in parallel.
353
368
` Workflow.async ` static method allows invoking any activity asynchronously. The call returns a ` Promise ` result immediately.
354
369
` Promise ` is similar to both Java ` Future ` and ` CompletionStage ` . The ` Promise ` ` get ` blocks until a result is available.
355
370
Also it exposes ` thenApply ` and ` handle ` methods. See ` Promise ` JavaDoc for technical details on differences with ` Future ` .
@@ -358,7 +373,7 @@ To convert a synchronous call
358
373
``` java
359
374
String localName = activities. download(surceBucket, sourceFile);
360
375
```
361
- to asynchronous style, pass the method reference to the ` Workflow.async ` followed by activity arguments.
376
+ to asynchronous style, the method reference is passed to the ` Workflow.async ` followed by activity arguments.
362
377
``` java
363
378
Promise<String > localNamePromise = Workflow . async(activities:: download, surceBucket, sourceFile);
364
379
```
@@ -414,7 +429,7 @@ Besides activities a workflow can also orchestrate other workflows.
414
429
415
430
` Workflow.newChildWorkflowStub ` returns a client side stub that implements a child workflow interface.
416
431
It takes a child workflow type and an optional child workflow options as arguments. Workflow options may be needed to override
417
- the timeouts and task list if they differ from the parent workflow ones.
432
+ the timeouts and task list if they differ from the defined in @ WorkflowMethod annotation or parent workflow ones.
418
433
419
434
The first call to the child workflow stub must always be to a method annotated with @WorkflowMethod . Similarly to activities a call
420
435
can be synchronous or asynchronous using ` Workflow.async ` . The synchronous call blocks until a child workflow completion. The asynchronous
@@ -484,20 +499,22 @@ public static class GreetingWorkflowImpl implements GreetingWorkflow {
484
499
Calling methods annotated with @QueryMethod is not allowed from within a workflow code.
485
500
### Workflow Implementation Constraints
486
501
487
- Cadence uses event sourcing to recover a state of the workflow object including its threads and local variables using
488
- [ event sourcing] ( https://docs.microsoft.com/en-us/azure/architecture/patterns/event-sourcing ) . In essence every time
489
- workflow state has to be restored its code is reexecuted from the beginning ignoring side effects (like activity invocations)
490
- which were already recorded in the workflow event history. Don't get confused, when writing workflow logic the replay is not visible,
502
+ Cadence uses [ event sourcing] ( https://docs.microsoft.com/en-us/azure/architecture/patterns/event-sourcing ) to recover
503
+ the state of a workflow object including its threads and local variable values.
504
+ . In essence every time a workflow state has to be restored its code is reexecuted from the beginning ignoring side
505
+ effects (like activity invocations) which were already recorded in the workflow event history.
506
+ Don't get confused, when writing workflow logic the replay is not visible,
491
507
so the code should be written as it executes only once. But this design still puts the following constraints on the workflow
492
508
implementation:
493
509
- Do not use any mutable global variables as multiple instances of workflows are executed in parallel.
494
510
- Do not call any non deterministic functions like non seeded random or UUID.randomUUID() directly form the workflow code.
495
511
Always do it in activities.
496
512
- Don’t perform any IO or service calls as they are not usually deterministic. Use activities for that.
497
513
- Only use ` Workflow.currentTimeMillis() ` to get current time inside a workflow.
498
- - Do not use native Java ` Thread ` class. Use ` Workflow.newThread ` to create workflow friendly ` WorkflowThread ` .
514
+ - Do not use native Java ` Thread ` or any other multi-threaded classes like ` ThreadPoolExecutor ` . Use ` Async.invoke `
515
+ to execute code asynchronously.
499
516
- Don't use any synchronization, locks and other standard Java blocking concurrency related classes besides provided
500
- by the Workflow class. There is no need in explicit synchronization as even multithreaded code inside a workflow is
517
+ by the Workflow class. There is no need in explicit synchronization as even multi-threaded code inside a workflow is
501
518
executed one thread at a time and under a global lock.
502
519
- Call ` WorkflowThread.sleep ` instead of ` Thread.sleep `
503
520
- Use ` Promise ` and ` CompletablePromise ` instead of ` Future ` and ` CompletableFuture ` .
0 commit comments