Skip to content

Commit 2752030

Browse files
committed
some small adjustments
1 parent c13bb1f commit 2752030

File tree

2 files changed

+42
-42
lines changed

2 files changed

+42
-42
lines changed

index.bs

Lines changed: 34 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ This streaming model ensures efficient memory usage and enables continuous trans
5050
The configuration of pipelines, processors, inputs, and outputs is expressed semantically using RDF.
5151
This use of **semantic configuration** promotes clarity, extensibility, and interoperability by describing the structure and behavior of the system in a machine-readable, standards-based format.
5252

53-
Processor components in RDF Connect are designed for **reusability**.
53+
Processor components in RDF-Connect are designed for **reusability**.
5454
A processor defined once can be reused across multiple pipelines and in varying contexts, reducing duplication and encouraging modular design practices.
5555
This modularity fosters rapid prototyping and easier maintenance of complex workflows.
5656

@@ -78,7 +78,7 @@ The pipeline configuration itself is expressed in RDF, making it semantically ex
7878
A *processor* is a modular, reusable software component that performs a discrete data processing task.
7979
In a typical scenario, a *processor* receives input data through a *reader*, transforms it according to its logic, and emits the result via a *writer*.
8080
However, processors are also flexible enough to support single-directional tasks, such as those that only produce or only consume data.
81-
Crucially, processors are implementation-agnostic --- they can be written in any programming language and integrated into pipelines via language-specific runners.
81+
Crucially, processors are implementation-agnostic they can be written in any programming language and integrated into pipelines via language-specific runners.
8282
This makes processors the building blocks of RDF-Connect’s cross-language interoperability.
8383

8484
## Runner
@@ -120,7 +120,7 @@ Shapes define required and optional configuration properties, which are transfor
120120
In RDF-Connect, such shapes are used to describe required parameters.
121121
They result in a well-typed JSON object that developers can rely on during implementation.
122122

123-
Shacl shape defining some required configuration for a processor
123+
SHACL shape defining some required configuration for a processor
124124

125125
```turtle
126126
[] a sh:NodeShape;
@@ -328,29 +328,29 @@ The orchestrator is also the message post office, allowing messages to be sent t
328328

329329
## Communication Protocol: Orchestrator ↔ Runner
330330

331-
RDF Connect uses a bidirectional communication protocol based on Protocol Buffers (protobuf) for interaction between the **Orchestrator** and **Runners**.
331+
RDF-Connect uses a bidirectional communication protocol based on Protocol Buffers (protobuf) for interaction between the **Orchestrator** and **Runners**.
332332
The orchestrator manages execution, while runners host and execute individual processors.
333333

334334
### Messages Sent to Runners
335335

336336
The orchestrator can send the following messages to the runner:
337337

338-
* Rpc.pipeline: Contains the complete pipeline configuration (in Turtle syntax).
339-
* Rpc.proc: Instructs the runner to register and setup a new processor.
340-
* Rpc.start: Signals that all added processors should begin execution.
341-
* Rpc.close: Instructs the runner to gracefully shut down and terminate all processors.
342-
* Rpc.msg: Delivers a message to a specific processor. Used for normal data transfer.
343-
* Rpc.streamMsg: Begins a streaming message transmission to a processor, typically for large or chunked payloads.
338+
* `RPC.pipeline`: Contains the complete pipeline configuration (in Turtle syntax).
339+
* `RPC.proc`: Instructs the runner to register and setup a new processor.
340+
* `RPC.start`: Signals that all added processors should begin execution.
341+
* `RPC.close`: Instructs the runner to gracefully shut down and terminate all processors.
342+
* `RPC.msg`: Delivers a message to a specific processor. Used for normal data transfer.
343+
* `RPC.streamMsg`: Begins a streaming message transmission to a processor, typically for large or chunked payloads.
344344

345345
### Messages Sent from Runners
346346

347347
The runner can send the following messages to the orchestrator:
348348

349-
* Rpc.identify: Indicates that the runner is ready and provides their unique identifier (IRI).
350-
* Rpc.init: Confirms that a previously registered **processor** has successfully started.
351-
* Rpc.close: Notifies that the runner is shutting down because all processors have stopped.
352-
* Rpc.msg: Sends a message from a processor to another processor via the orchestrator.
353-
* Rpc.streamMsg: Sends a streaming message from a processor. Used when the payload is too large for a single message.
349+
* `RPC.identify`: Indicates that the runner is ready and provides their unique identifier (IRI).
350+
* `RPC.init`: Confirms that a previously registered **processor** has successfully started.
351+
* `RPC.close`: Notifies that the runner is shutting down because all processors have stopped.
352+
* `RPC.msg`: Sends a message from a processor to another processor via the orchestrator.
353+
* `RPC.streamMsg`: Sends a streaming message from a processor. Used when the payload is too large for a single message.
354354

355355

356356
## Orchestrator
@@ -391,7 +391,7 @@ Each pair includes:
391391
<div class="example" title="Pipeline definition">
392392

393393
This example pipeline contains three processors divided over two runners.
394-
The orchestrator starts for this pipeline, two runners and provide them with the correct processor configurations.
394+
The orchestrator starts this pipeline with two runners and provides them with the correct processor configurations.
395395

396396
```turtle
397397
@prefix rdfc: <https://w3id.org/rdf-connect/ontology#>.
@@ -415,7 +415,7 @@ Each simple runner should specify two things:
415415
- how the runner should be started, linked with `rdfc:command`
416416

417417
<aside class="note">
418-
The property `rdfc:handlesSubjectsOf` plays a somewhat unconventional role. It allows RDF Connect to remain aligned with [PROV-O](https://www.w3.org/TR/prov-o/).
418+
The property `rdfc:handlesSubjectsOf` plays a somewhat unconventional role. It allows RDF-Connect to remain aligned with [PROV-O](https://www.w3.org/TR/prov-o/).
419419
A runner may link to a predicate such as `rdfc:jsImplementationOf`, and each JavaScript processor may then declare itself using:
420420
```turtle
421421
<FooBar> rdfc:jsImplementationOf rdfc:Processor.
@@ -449,7 +449,7 @@ Once all expected runners have successfully identified themselves, the orchestra
449449

450450
#### Starting processors
451451

452-
Once all runners have been successfully identified via Rpc.identify, the orchestrator proceeds to initialize the processors defined in the pipeline.
452+
Once all runners have been successfully identified via `RPC.identify`, the orchestrator proceeds to initialize the processors defined in the pipeline.
453453
This involves the extraction and transformation of processor configuration data into a format suitable for consumption by the associated runner.
454454

455455
**Processor Arguments**
@@ -516,11 +516,11 @@ Processor definitions are extracted using the same SHACL-based mechanism describ
516516

517517
**RPC message**
518518

519-
Once both the arguments and definition have been extracted for a processor instance, the orchestrator sends an `Rpc.proc` message to the appropriate runner, initiating the processor launch process.
519+
Once both the arguments and definition have been extracted for a processor instance, the orchestrator sends an `RPC.proc` message to the appropriate runner, initiating the processor launch process.
520520

521521
The orchestrator MUST keep an internal record of all processor instance that have been dispatched to a runner, and the runner’s acknowledgment that a processor was successfully launched, as indicated by an incoming `Rpc.init` message.
522522

523-
No processor may be assumed to be operational until its runner has responded with `Rpc.init`.
523+
No processor may be assumed to be operational until its runner has responded with `RPC.init`.
524524
When all processors are successfully initialized, the orchestrator can start the pipeline.
525525

526526

@@ -543,12 +543,12 @@ Importantly, channels support **many-to-many** communication: multiple processor
543543

544544
#### Normal messages
545545

546-
When a runner sends a `Rpc.msg` message to the orchestrator, the message includes a channel IRI indicating its logical destination.
546+
When a runner sends a `RPC.msg` message to the orchestrator, the message includes a channel IRI indicating its logical destination.
547547

548548
The orchestrator MUST:
549549
1. Resolve the set of processors that are declared to consume this channel.
550550
2. Determine which runner is responsible for each of those processors.
551-
3. Forward the message to each relevant runner using `Rpc.msg`.
551+
3. Forward the message to each relevant runner using `RPC.msg`.
552552

553553
These messages are sent as discrete units and fit within the allowed message size.
554554

@@ -565,8 +565,8 @@ This protocol enables large messages to be sent incrementally over a separate gR
565565
The process is as follows:
566566
1. **Sender** (runner) initiates a `sendStreamMessage` gRPC stream to the orchestrator.
567567
2. The **orchestrator** generates a unique stream identifier and sends it back on this stream.
568-
3. The **sender** then sends a `Rpc.streamMsg` over the normal bidirectional RPC stream, including the stream identifier and the channel IRI.
569-
4. The **orchestrator** resolves which processors receive messages on the given channel and forwards the stream identifier to their corresponding runners with `Rpc.streamMsg`.
568+
3. The **sender** then sends a `RPC.streamMsg` over the normal bidirectional RPC stream, including the stream identifier and the channel IRI.
569+
4. The **orchestrator** resolves which processors receive messages on the given channel and forwards the stream identifier to their corresponding runners with `RPC.streamMsg`.
570570
5. **Receiving runners** connect to the orchestrator using `receiveStreamMessage`, passing the received stream identifier.
571571
6. Once all participants are connected, the orchestrator acts as a relay: all incoming chunks from the sending stream are forwarded to each connected receiving stream.
572572

@@ -587,7 +587,7 @@ While minimal runners can be implemented with little overhead, they often become
587587

588588
These quality-of-life features include:
589589

590-
* Wrapping readers and writer in idiomatic objects
590+
* Wrapping readers and writers in idiomatic objects
591591
* Runners SHOULD also make it possible to let processors start up before acknowledging to the orchestrator that the processor is initialized.
592592

593593
This is useful for processor to execute longer running operations, like reading a file or consulting an external API.
@@ -613,12 +613,12 @@ The following sections detail the runner startup flow and describe the expected
613613

614614
A runner is defined with an instance of `rdfc:Runner`, which currently is instantiated by a command.
615615
It requires the command to actually start the runner with `rdfc:command`, and a link to the programming context (`rdfc:handlesSubjectsOf`).
616-
The object of `rdfc:handlesSubjectsOf`, links runners and processors to a context term. This often refers to the programming language.
616+
The object of `rdfc:handlesSubjectsOf` links runners and processors to a context term. This often refers to the programming language.
617617

618618
Each context term is related to a SHACL shape, this specifies the incoming data that the runner can use to start the processors.
619619

620620
<div class=example>
621-
For example, the NodeRunner, a runner that starts JavaScript processors with node, is defined as follows.
621+
The NodeRunner, for example, is a program that starts JavaScript processors with Node. It is defined as follows:
622622

623623
```turtle
624624
rdfc:NodeRunner a rdfc:Runner;
@@ -676,10 +676,10 @@ After initialization, the orchestrator may send multiple `RPC.proc` messages to
676676
Each message includes a processor IRI, a configuration object and an argument object.
677677
Both the configuration and arguments are provided as JSON-LD strings.
678678
The configuration object contains the arguments as defined by the context term following the section [Mapping SHACL to Configuration Structures](#mapping-shacl-to-configuration-structures).
679-
The arguments are constructed based on a SHACL shape defined for the processor type
679+
The arguments are constructed based on a SHACL shape defined for the processor type.
680680

681-
Runners are MAY to parse these JSON-LD payloads and transform known constructs into idiomatic equivalents.
682-
For example reader and writer objects are represented as JSON-LD values with: an @id field (containing the channel IRI), and an @type field indicating either `https://w3id.org/rdf-connect/ontology#Reader` or `https://w3id.org/rdf-connect/ontology#Writer`.
681+
Runners MAY parse these JSON-LD payloads and transform known constructs into idiomatic equivalents.
682+
For example, reader and writer objects are represented as JSON-LD values with: an `@id` field (containing the channel IRI), and an `@type` field indicating either `https://w3id.org/rdf-connect/ontology#Reader` or `https://w3id.org/rdf-connect/ontology#Writer`.
683683
Runners are RECOMMENDED to replace these values with appropriate typed objects in the target environment.
684684

685685
When a processor has been fully initialized, the runner MUST send an `RPC.init` message, indicating success or failure.
@@ -714,7 +714,7 @@ If the runner determines that the message size is small enough, it MAY convert a
714714

715715
#### Sending messages
716716

717-
Runners must also support outbound communication from processors.
717+
Runners MUST also support outbound communication from processors.
718718
While runners MAY omit support for certain advanced features (such as streaming output), a full implementation is strongly encouraged.
719719

720720
To send a normal message, the runner uses the `RPC.msg` method on the normal stream.
@@ -728,10 +728,10 @@ The message is considered complete when the runner closes the stream.
728728

729729
#### Channel Closure
730730

731-
Processors may indicate that a given channel is closed (i.e., no further messages will be sent).
732-
The runner MUST propagate this information to the orchestrator via an RPC.close message.
731+
Processors MAY indicate that a given channel is closed (i.e., no further messages will be sent).
732+
The runner MUST propagate this information to the orchestrator via an `RPC.close` message.
733733

734-
Similarly, when the orchestrator sends an RPC.close message for a channel, the runner MAY respond by closing or invalidating the corresponding data stream in the processor.
734+
Similarly, when the orchestrator sends an `RPC.close` message for a channel, the runner MAY respond by closing or invalidating the corresponding data stream in the processor.
735735

736736
## Processor
737737

scripts.js

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -32,12 +32,12 @@ flowchart TD
3232
end
3333
direction LR
3434
35-
P1 w1@-.-|writes| A@{ shape: processes, label: "Channel" }
36-
P3 w3@-.-|writes| A
37-
P4 w4@-.-|writes| A
38-
P2 r2@-.-|reads| A
39-
P3 r3@-.-|reads| A
40-
P4 r4@-.-|reads| A
35+
P1 w1@-.-|1: writes| A@{ shape: processes, label: "Channel" }
36+
P3 w3@-.-|5: writes| A
37+
P4 w4@-.-|3: writes| A
38+
P2 r2@-.-|6: reads| A
39+
P3 r3@-.-|4: reads| A
40+
P4 r4@-.-|2: reads| A
4141
classDef animateRead stroke-dasharray: 9,5,stroke-dashoffset: 900,animation: dash 25s linear infinite reverse;
4242
class r2,r3,r4 animateRead
4343
classDef animateWrite stroke-dasharray: 9,5,stroke-dashoffset: 900,animation: dash 25s linear infinite;
@@ -65,7 +65,7 @@ sequenceDiagram
6565
end
6666
loop For every runner
6767
O-->>R: RPC.Start: Start
68-
loop For every Processor
68+
loop For every processor
6969
R-->>P: Start
7070
end
7171
end
@@ -91,7 +91,7 @@ sequenceDiagram
9191
P1 -->> R1: Data chunks
9292
R1 -->> O: Data chunks over stream
9393
O -->> R2: Data chunks over stream
94-
R2 -->>P2: Data chnuks
94+
R2 -->>P2: Data chunks
9595
end
9696
`
9797
};

0 commit comments

Comments
 (0)