Skip to content

Commit 8037145

Browse files
sandersonsanderson
authored
Update the v3 storage diagram with missing components (#5558)
* update v3 storage diagram with missing components * remove orphaned style tag from svg * Apply suggestions from code review
1 parent 6944b6f commit 8037145

File tree

4 files changed

+164
-112
lines changed

4 files changed

+164
-112
lines changed

‎content/influxdb/cloud-dedicated/reference/internals/storage-engine.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,13 @@ queries, and is optimized to reduce storage cost.
2323

2424
- [Storage engine diagram](#storage-engine-diagram)
2525
- [Storage engine components](#storage-engine-components)
26+
- [Router](#router)
2627
- [Ingester](#ingester)
2728
- [Querier](#querier)
2829
- [Catalog](#catalog)
2930
- [Object store](#object-store)
3031
- [Compactor](#compactor)
32+
- [Garbage collector](#garbage-collector)
3133
- [Scaling strategies](#scaling-strategies)
3234
- [Vertical scaling](#vertical-scaling)
3335
- [Horizontal scaling](#horizontal-scaling)
@@ -38,11 +40,29 @@ queries, and is optimized to reduce storage cost.
3840

3941
## Storage engine components
4042

43+
- [Router](#router)
4144
- [Ingester](#ingester)
4245
- [Querier](#querier)
4346
- [Catalog](#catalog)
4447
- [Object store](#object-store)
4548
- [Compactor](#compactor)
49+
- [Garbage collector](#garbage-collector)
50+
51+
### Router
52+
53+
The Router (also known as the Ingest Router) parses incoming line
54+
protocol and then routes it to [Ingesters](#ingester).
55+
To ensure write durability, the Router replicates data to two or more of the
56+
available Ingesters.
57+
58+
##### Router scaling strategies
59+
60+
The Router can be scaled both [vertically](#vertical-scaling) and
61+
[horizontally](#horizontal-scaling).
62+
Horizontal scaling increases write throughput and is typically the most
63+
effective scaling strategy for the Router.
64+
Vertical scaling (specifically increased CPU) improves the Router's ability to
65+
parse incoming line protocol with lower latency.
4666

4767
### Ingester
4868

@@ -150,6 +170,20 @@ increasing the available CPU) is the most effective scaling strategy for the Com
150170
Horizontal scaling increases compaction throughput, but not as efficiently as
151171
vertical scaling.
152172

173+
### Garbage collector
174+
175+
The Garbage collector runs background jobs that evict expired or deleted data,
176+
remove obsolete compaction files, and reclaim space in both the [Catalog](#catalog) and the
177+
[Object store](#object-store).
178+
179+
##### Garbage collector scaling strategies
180+
181+
The Garbage collector is not designed for distributed load and should _not_ be
182+
scaled horizontally. The Garbage collector does not perform CPU- or
183+
memory-intensive work, so [vertical scaling](#vertical-scaling) should only be
184+
considered only if you observe very high CPU usage or if the container regularly
185+
runs out of memory.
186+
153187
---
154188

155189
## Scaling strategies

‎content/influxdb/clustered/admin/scale-cluster.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -262,13 +262,22 @@ to automatically scale your cluster as needed.
262262

263263
## Recommended scaling strategies per component
264264

265+
- [Router](#router)
265266
- [Ingester](#ingester)
266267
- [Querier](#querier)
267-
- [Router](#router)
268268
- [Compactor](#compactor)
269269
- [Catalog](#catalog)
270270
- [Object store](#object-store)
271271

272+
### Router
273+
274+
The Router can be scaled both [vertically](#vertical-scaling) and
275+
[horizontally](#horizontal-scaling).
276+
Horizontal scaling increases write throughput and is typically the most
277+
effective scaling strategy for the Router.
278+
Vertical scaling (specifically increased CPU) improves the Router's ability to
279+
parse incoming line protocol with lower latency.
280+
272281
### Ingester
273282

274283
The Ingester can be scaled both [vertically](#vertical-scaling) and
@@ -322,13 +331,6 @@ Horizontal scaling increases query throughput to handle more concurrent queries.
322331
Vertical scaling improves the Querier’s ability to process computationally
323332
intensive queries.
324333

325-
### Router
326-
327-
The Router can be scaled both [vertically](#vertical-scaling) and
328-
[horizontally](#horizontal-scaling).
329-
Horizontal scaling increases request throughput and is typically the most effective
330-
scaling strategy for the Router.
331-
332334
### Compactor
333335

334336
The Compactor can be scaled both [vertically](#vertical-scaling) and

‎content/influxdb/clustered/reference/internals/storage-engine.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ menu:
1111
parent: InfluxDB internals
1212
influxdb/clustered/tags: [storage, internals]
1313
related:
14+
- /influxdb/clustered/admin/scale-cluster/
1415
- /influxdb/clustered/admin/custom-partitions/
1516
---
1617

@@ -23,23 +24,43 @@ queries, and is optimized to reduce storage cost.
2324

2425
- [Storage engine diagram](#storage-engine-diagram)
2526
- [Storage engine components](#storage-engine-components)
27+
- [Router](#router)
2628
- [Ingester](#ingester)
2729
- [Querier](#querier)
2830
- [Catalog](#catalog)
2931
- [Object store](#object-store)
3032
- [Compactor](#compactor)
33+
- [Garbage collector](#garbage-collector)
3134

3235
## Storage engine diagram
3336

3437
{{< svg "/static/svgs/v3-storage-architecture.svg" >}}
3538

3639
## Storage engine components
3740

41+
- [Router](#router)
3842
- [Ingester](#ingester)
3943
- [Querier](#querier)
4044
- [Catalog](#catalog)
4145
- [Object store](#object-store)
4246
- [Compactor](#compactor)
47+
- [Garbage collector](#garbage-collector)
48+
49+
### Router
50+
51+
The Router (also known as the Ingest Router) parses incoming line
52+
protocol and then routes it to [Ingesters](#ingester).
53+
To ensure write durability, the Router replicates data to two or more of the
54+
available Ingesters.
55+
56+
##### Router scaling strategies
57+
58+
The Router can be scaled both [vertically](/influxdb/clustered/admin/scale-cluster/#vertical-scaling)
59+
and [horizontally](/influxdb/clustered/admin/scale-cluster/#horizontal-scaling).
60+
Horizontal scaling increases write throughput and is typically the most
61+
effective scaling strategy for the Router.
62+
Vertical scaling (specifically increased CPU) improves the Router's ability to
63+
parse incoming line protocol with lower latency.
4364

4465
### Ingester
4566

@@ -149,3 +170,18 @@ Because compaction is a compute-heavy process, vertical scaling (especially
149170
increasing the available CPU) is the most effective scaling strategy for the Compactor.
150171
Horizontal scaling increases compaction throughput, but not as efficiently as
151172
vertical scaling.
173+
174+
### Garbage collector
175+
176+
The Garbage collector runs background jobs that evict expired or deleted data,
177+
remove obsolete compaction files, and reclaim space in both the [Catalog](#catalog) and the
178+
[Object store](#object-store).
179+
180+
##### Garbage collector scaling strategies
181+
182+
183+
The Garbage collector is not designed for distributed load and should _not_ be
184+
scaled horizontally. The Garbage collector does not perform CPU- or
185+
memory-intensive work, so [vertical scaling](/influxdb/clustered/admin/scale-cluster/#vertical-scaling)
186+
should only be considered only if you observe very high CPU usage or
187+
if the container regularly runs out of memory.

‎static/svgs/v3-storage-architecture.svg

Lines changed: 84 additions & 104 deletions
Loading

0 commit comments

Comments
 (0)