Skip to content

Commit cff1724

Browse files
committed
Add diagrams for stream coordinator
1 parent 9b255fc commit cff1724

File tree

1 file changed

+77
-0
lines changed

1 file changed

+77
-0
lines changed
Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# Stream Coordinator
2+
3+
## Single Active Consumer
4+
5+
### "Simple" SAC (Not Super Stream)
6+
7+
```mermaid
8+
sequenceDiagram
9+
participant C as Coordinator
10+
participant C1 as Connection 1
11+
participant C2 as Connection 2
12+
participant C3 as Connection 3
13+
Note over C,C3: Simple SAC (not super stream)
14+
C1->>C: register sub 1
15+
C-)C1: {sac, sub 1, active = true}
16+
activate C1
17+
C1->>C1: consumer update to client
18+
C2->>C: register sub 2
19+
C3->>C: register sub 3
20+
C1->>C: unregister sub 1
21+
deactivate C1
22+
C-)C2: {sac, sub 2, active = true}
23+
activate C2
24+
C1->>C1: consumer update to client
25+
deactivate C2
26+
```
27+
28+
### SAC with Super Stream Partition
29+
30+
```mermaid
31+
sequenceDiagram
32+
participant C as Coordinator
33+
participant C1 as Connection 1
34+
participant C2 as Connection 2
35+
participant C3 as Connection 3
36+
Note over C,C3: Super Stream SAC (partition = 1)
37+
C1->>C: register sub 1
38+
C-)C1: {sac, sub 1, active = true}
39+
activate C1
40+
C2->>C: register sub 2
41+
C-)C1: {sac, sub 1, active = false, step down = true}
42+
deactivate C1
43+
C1->>C1: consumer update to client
44+
C1->>C: active consumer in group
45+
C-)C2: {sac, sub 1, active = true}
46+
activate C2
47+
C2->>C2: consumer update to client
48+
C3->>C: register sub 3
49+
Note over C, C3: active consumer stays the same (partition % consumers = 1 % 3 = 1)
50+
deactivate C2
51+
```
52+
53+
### `noconnection` management
54+
55+
```mermaid
56+
flowchart TB
57+
A(monitor) --noconnection--> B(status = disconnected, set up timer)
58+
B -. timeout .-> C(status = forgotten)
59+
B -. nodeup .-> D(reissue monitors, send msg to connections)
60+
D -. down .-> E(handle connection down)
61+
D -. connection response .-> F(evaluate impacted groups)
62+
```
63+
64+
* composite status for consumers: `{connected, active}`, `{disconnected,active}`, etc.
65+
* `disconnected` status can prevent rebalancing in a group, e.g. `{disconnected, active}` (it is impossible to tell the active consumer to step down)
66+
* consumers in `forgotten` status are ignored during rebalancing
67+
* it may be necessary to reconcile a group if a `{forgotten, active}` consumer comes back in a group ("evaluate impacted groups" box above).
68+
This is unlikely though.
69+
70+
### Stale Node Detection
71+
72+
```mermaid
73+
flowchart TB
74+
A(RA) -- tick --> B(stale nodes = RA known nodes - cluster nodes)
75+
B -. no stale nodes .-> C(nothing to do)
76+
B -. stale nodes .-> D(remove connections from state)
77+
```

0 commit comments

Comments
 (0)