|
| 1 | +# Stream Coordinator |
| 2 | + |
| 3 | +## Single Active Consumer |
| 4 | + |
| 5 | +### "Simple" SAC (Not Super Stream) |
| 6 | + |
| 7 | +```mermaid |
| 8 | +sequenceDiagram |
| 9 | + participant C as Coordinator |
| 10 | + participant C1 as Connection 1 |
| 11 | + participant C2 as Connection 2 |
| 12 | + participant C3 as Connection 3 |
| 13 | + Note over C,C3: Simple SAC (not super stream) |
| 14 | + C1->>C: register sub 1 |
| 15 | + C-)C1: {sac, sub 1, active = true} |
| 16 | + activate C1 |
| 17 | + C1->>C1: consumer update to client |
| 18 | + C2->>C: register sub 2 |
| 19 | + C3->>C: register sub 3 |
| 20 | + C1->>C: unregister sub 1 |
| 21 | + deactivate C1 |
| 22 | + C-)C2: {sac, sub 2, active = true} |
| 23 | + activate C2 |
| 24 | + C1->>C1: consumer update to client |
| 25 | + deactivate C2 |
| 26 | +``` |
| 27 | + |
| 28 | +### SAC with Super Stream Partition |
| 29 | + |
| 30 | +```mermaid |
| 31 | +sequenceDiagram |
| 32 | + participant C as Coordinator |
| 33 | + participant C1 as Connection 1 |
| 34 | + participant C2 as Connection 2 |
| 35 | + participant C3 as Connection 3 |
| 36 | + Note over C,C3: Super Stream SAC (partition = 1) |
| 37 | + C1->>C: register sub 1 |
| 38 | + C-)C1: {sac, sub 1, active = true} |
| 39 | + activate C1 |
| 40 | + C2->>C: register sub 2 |
| 41 | + C-)C1: {sac, sub 1, active = false, step down = true} |
| 42 | + deactivate C1 |
| 43 | + C1->>C1: consumer update to client |
| 44 | + C1->>C: active consumer in group |
| 45 | + C-)C2: {sac, sub 1, active = true} |
| 46 | + activate C2 |
| 47 | + C2->>C2: consumer update to client |
| 48 | + C3->>C: register sub 3 |
| 49 | + Note over C, C3: active consumer stays the same (partition % consumers = 1 % 3 = 1) |
| 50 | + deactivate C2 |
| 51 | +``` |
| 52 | + |
| 53 | +### `noconnection` management |
| 54 | + |
| 55 | +```mermaid |
| 56 | +flowchart TB |
| 57 | + A(monitor) --noconnection--> B(status = disconnected, set up timer) |
| 58 | + B -. timeout .-> C(status = forgotten) |
| 59 | + B -. nodeup .-> D(reissue monitors, send msg to connections) |
| 60 | + D -. down .-> E(handle connection down) |
| 61 | + D -. connection response .-> F(evaluate impacted groups) |
| 62 | +``` |
| 63 | + |
| 64 | +* composite status for consumers: `{connected, active}`, `{disconnected,active}`, etc. |
| 65 | +* `disconnected` status can prevent rebalancing in a group, e.g. `{disconnected, active}` (it is impossible to tell the active consumer to step down) |
| 66 | +* consumers in `forgotten` status are ignored during rebalancing |
| 67 | +* it may be necessary to reconcile a group if a `{forgotten, active}` consumer comes back in a group ("evaluate impacted groups" box above). |
| 68 | +This is unlikely though. |
| 69 | + |
| 70 | +### Stale Node Detection |
| 71 | + |
| 72 | +```mermaid |
| 73 | +flowchart TB |
| 74 | + A(RA) -- tick --> B(stale nodes = RA known nodes - cluster nodes) |
| 75 | + B -. no stale nodes .-> C(nothing to do) |
| 76 | + B -. stale nodes .-> D(remove connections from state) |
| 77 | +``` |
0 commit comments