Skip to content

Commit 30f2fe0

Browse files
committed
spelling
1 parent a2274b3 commit 30f2fe0

File tree

1 file changed

+8
-12
lines changed

1 file changed

+8
-12
lines changed

docs/design/4940-reliable-loki-pipelines.md

Lines changed: 8 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -32,15 +32,15 @@ loki.process "logs" {
3232
loki.write "loki" {}
3333
```
3434

35-
`loki.source.file` will tail all files from targets and compete to send on the channel exposed by `loki.process`. Only one entry will be processed by each stage configured in `loki.process`. If a reload happens or if alloy is shutting down logs could be silently dropped.
35+
`loki.source.file` will tail all files from targets and compete to send on the channel exposed by `loki.process`. Only one entry will be processed by each stage configured in `loki.process`. If a reload happens or if Alloy is shutting down, logs could be silently dropped.
3636

3737
## Proposal 0: Do nothing
3838
This architecture works in most cases, it will be hard to use slow components such as `secretfilter` because a lot of the time it's too slow.
3939
It's also hard to use Alloy as a gateway for loki pipelines with e.g. `loki.source.api` due to the limitations listed above.
4040

4141
## Proposal 1: Chain function calls
4242

43-
Loki pipelines are the only one using channels for passing data between components. Prometheus, Pyroscope and otelcol are all using this pattern where each component just calls functions on the next.
43+
Loki pipelines are the only ones using channels for passing data between components. Prometheus, Pyroscope and otelcol are all using this pattern where each component just calls functions on the next.
4444

4545
They all have slightly different interfaces but basically work the same. Each component exports its own interface like Appender for Prometheus or Consumer for Otel.
4646

@@ -52,7 +52,7 @@ type Consumer interface {
5252
}
5353
```
5454

55-
Adopting this pattern for loki pipelines would change it from a channel based pipeline to a function based pipeline. This would give us two things:
55+
Adopting this pattern for loki pipelines would change it from a channel-based pipeline to a function-based pipeline. This would give us two things:
5656
1. Increased throughput because several sources such as many files or http requests can now call the next component in the pipeline at the same time.
5757
2. A way to return signals back to the source so we can handle things like giving a proper error response or determine if the position file should be updated.
5858

@@ -65,7 +65,7 @@ Pros:
6565
* A way to signal back to the source
6666
Cons:
6767
* We need to rewrite all loki components with this new pattern and make them safe to call in parallel.
68-
* We go from an iterator like pipeline to passing slices around. Every component would have to iterate over this slice and we need to make sure it's safe to mutate because of fanout.
68+
* We go from an iterator-like pipeline to passing slices around. Every component would have to iterate over this slice and we need to make sure it's safe to mutate because of fan-out.
6969

7070
## Proposal 2: Appendable
7171

@@ -88,14 +88,12 @@ type Appender interface {
8888
This approach would, like Proposal 1, solve the issues listed above with a function-based pipeline, but the pipeline would still be iterator-like (one entry at a time).
8989

9090
### How it works
91-
92-
Sources would obtain an `Appender` from the next component in the pipeline, then:
93-
1. Call `Append(entry)` for each entry in a batch
94-
2. Call `Commit()` to finalize the batch, or `Rollback()` to discard it
95-
3. Handle errors at any step
91+
Source components would:
92+
Obtain an `Appender` that can fan-out to all downstream components, then call `Append` for each entry.
93+
If every call to `Append` is successful, `Commit` should be called; otherwise `Rollback`.
9694

9795
Processing components would:
98-
Implement `Appendable` to return an `Appender` that runs processing for each entry and forwards it to next component.
96+
Implement `Appendable` to return an `Appender` that runs processing for each entry and fan-out to all downstream components.
9997

10098
Sink components would:
10199
Implement `Appendable` to return an `Appender` that buffers entries until either `Commit` or `Rollback` is called.
@@ -153,5 +151,3 @@ The following components need to be updated with this new interface and we need
153151
**Sink components** (need to implement `Consumer`):
154152
- `loki.write`
155153
- `loki.echo`
156-
157-

0 commit comments

Comments
 (0)