@@ -201,8 +201,6 @@ The Run Engine emits events using its `eventBus`. This is used for runs completi
201201
202202# RunEngine System Architecture
203203
204- The RunEngine is composed of several specialized systems that handle different aspects of task execution and management. Below is a diagram showing the relationships between these systems.
205-
206204``` mermaid
207205graph TD
208206 RE[RunEngine]
@@ -216,6 +214,7 @@ graph TD
216214 DRS[DelayedRunSystem]
217215 TS[TtlSystem]
218216 WFS[WaitingForWorkerSystem]
217+ RCS[ReleaseConcurrencySystem]
219218
220219 %% Core Dependencies
221220 RE --> DS
@@ -228,6 +227,7 @@ graph TD
228227 RE --> DRS
229228 RE --> TS
230229 RE --> WFS
230+ RE --> RCS
231231
232232 %% System Dependencies
233233 DS --> ESS
@@ -239,11 +239,13 @@ graph TD
239239
240240 WS --> ESS
241241 WS --> ES
242+ WS --> RCS
242243
243244 ES --> ESS
244245
245246 CS --> ESS
246247 CS --> ES
248+ CS --> RCS
247249
248250 DRS --> ES
249251
@@ -260,54 +262,34 @@ graph TD
260262 RL[RunLocker]
261263 EB[EventBus]
262264 WRK[Worker]
263- RCQ[ReleaseConcurrencyQueue]
264265 end
265266
266267 %% Resource Dependencies
267268 RE -.-> Resources
268- DS & RAS & ESS & WS & BS & ES & CS & DRS & TS & WFS -.-> Resources
269+ DS & RAS & ESS & WS & BS & ES & CS & DRS & TS & WFS & RCS -.-> Resources
269270```
270271
271272## System Responsibilities
272273
273- ### DequeueSystem
274-
275- - Handles dequeuing of tasks from master queues
276- - Manages resource allocation and constraints
277- - Handles task deployment verification
278-
279- ### RunAttemptSystem
280-
281- - Manages run attempt lifecycle
282- - Handles success/failure scenarios
283- - Manages retries and cancellations
284- - Coordinates with other systems for run completion
285-
286- ### ExecutionSnapshotSystem
287-
288- - Creates and manages execution snapshots
289- - Tracks run state and progress
290- - Manages heartbeats for active runs
291- - Maintains execution history
292-
293- ### WaitpointSystem
274+ ### Core Systems
294275
295- - Manages waitpoints for task synchronization
296- - Handles waitpoint completion
297- - Coordinates blocked runs
298- - Manages concurrency release
276+ - ** ExecutionSnapshotSystem (ESS) ** : Manages execution state tracking and history
277+ - ** EnqueueSystem (ES) ** : Handles run scheduling and queueing
278+ - ** RunAttemptSystem (RAS) ** : Controls run lifecycle and execution attempts
279+ - ** ReleaseConcurrencySystem (RCS) ** : Manages concurrency control and token management
299280
300- ### BatchSystem
281+ ### Queue Management
301282
302- - Manages batch operations
303- - Handles batch completion
304- - Coordinates batch-related task runs
283+ - ** DequeueSystem (DS) ** : Handles task dequeuing and resource allocation
284+ - ** DelayedRunSystem (DRS) ** : Manages delayed run scheduling
285+ - ** WaitingForWorkerSystem (WFS) ** : Coordinates runs waiting for worker availability
305286
306- ### EnqueueSystem
287+ ### State Management
307288
308- - Handles enqueueing of runs
309- - Manages run scheduling
310- - Coordinates with execution snapshots
289+ - ** CheckpointSystem (CS)** : Manages execution checkpoints and resumption
290+ - ** WaitpointSystem (WS)** : Coordinates task synchronization points
291+ - ** BatchSystem (BS)** : Handles batch operations and coordination
292+ - ** TtlSystem (TS)** : Manages run time-to-live and expiration
311293
312294## Shared Resources
313295
@@ -318,13 +300,27 @@ graph TD
318300- ** RunLocker** : Run locking mechanism
319301- ** EventBus** : Event communication
320302- ** Worker** : Background task execution
321- - ** ReleaseConcurrencyQueue** : Manages concurrency token release
322303
323304## Key Interactions
324305
325- 1 . ** RunEngine** orchestrates all systems and manages shared resources
326- 2 . ** DequeueSystem** works with ** RunAttemptSystem** for task execution
327- 3 . ** RunAttemptSystem** coordinates with ** WaitpointSystem** and ** BatchSystem**
328- 4 . ** WaitpointSystem** uses ** EnqueueSystem** for run scheduling
329- 5 . ** ExecutionSnapshotSystem** is used by all other systems to track state
330- 6 . All systems share common resources through the ` SystemResources ` interface
306+ 1 . ** Core Execution Flow**
307+
308+ - ExecutionSnapshotSystem provides state tracking for all systems
309+ - EnqueueSystem coordinates with multiple systems for scheduling
310+ - RunAttemptSystem manages execution lifecycle
311+ - ReleaseConcurrencySystem controls concurrent execution limits
312+
313+ 2 . ** Queue Management**
314+
315+ - DequeueSystem coordinates with RunAttemptSystem for execution
316+ - DelayedRunSystem and WaitingForWorkerSystem handle specialized queuing
317+
318+ 3 . ** State Coordination**
319+
320+ - CheckpointSystem works with ReleaseConcurrencySystem and EnqueueSystem
321+ - WaitpointSystem coordinates with ReleaseConcurrencySystem for execution control
322+ - TtlSystem manages expiration through WaitpointSystem
323+
324+ 4 . ** Resource Management**
325+ - All systems share common resources through SystemResources
326+ - ReleaseConcurrencySystem provides centralized concurrency control
0 commit comments