Skip to content

Commit e8171ff

Browse files
authored
Fixed mapping and slightly re-arranged the ingest lag headings
1 parent 5f53e25 commit e8171ff

File tree

2 files changed

+11
-16
lines changed

2 files changed

+11
-16
lines changed

manage-data/ingest/transform-enrich/ingest-lag.md

Lines changed: 11 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,4 @@
11
---
2-
mapped_pages:
3-
- https://www.elastic.co/docs/manage-data/ingest/transform-enrich/calculate-ingest-lag.html
42
applies_to:
53
stack: ga
64
serverless: ga
@@ -99,16 +97,6 @@ As discussed above `@timestamp` is set to the timestamp from within the collecte
9997
10098
Regardless of the chosen architecture, add a `remove` processor at the end of the pipeline to drop the `_tmp` field. The raw timestamps from the various processing steps are not needed, as the latency in seconds should be sufficient. For additional pipeline architectures, refer to [Ingest architectures](../ingest-reference-architectures.md).
10199

102-
## Logstash
103-
104-
When Logstash is added to the architecture we must add a timestamp, this can only be done by using Ruby and the simplest form is this:
105-
106-
```
107-
ruby {
108-
code => "event.set('[_tmp][logstash_seen]', Time.now());"
109-
}
110-
```
111-
112100
### Elastic Agent to Elasticsearch
113101

114102
Use `@timestamp` and `event.ingested` to calculate the difference. This will give you the following document. The `event.ingestion.latency` is in seconds.
@@ -156,8 +144,17 @@ POST _ingest/pipeline/_simulate
156144
}
157145
}
158146
```
147+
### Logstash
148+
149+
When Logstash is added to the architecture we must add a timestamp, this can only be done by using Ruby and the simplest form is this:
150+
151+
```
152+
ruby {
153+
code => "event.set('[_tmp][logstash_seen]', Time.now());"
154+
}
155+
```
159156

160-
### Elastic Agent => Logstash => Elasticsearch
157+
#### Elastic Agent => Logstash => Elasticsearch
161158

162159
Elastic Agent populates the `@timestamp` field, but Logstash doesn't add any timestamp by default. Add a temporary timestamp, for example by setting `_tmp.logstash_seen`. With this, you can calculate the following latency values:
163160

@@ -236,7 +233,7 @@ POST _ingest/pipeline/_simulate
236233
}
237234
```
238235

239-
### Elastic Agent => Logstash => Kafka => Logstash => Elasticsearch
236+
#### Elastic Agent => Logstash => Kafka => Logstash => Elasticsearch
240237

241238
As with the previous scenario, adding an additional hop introduces another point where latency can occur. The recommendation is to add another temporary timestamp field. For more details, refer to the previous section.
242239

manage-data/ingest/transform-enrich/readable-maintainable-ingest-pipelines.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,4 @@
11
---
2-
mapped_pages:
3-
- https://www.elastic.co/docs/manage-data/ingest/transform-enrich/common-mistakes.html
42
applies_to:
53
stack: ga
64
serverless: ga

0 commit comments

Comments
 (0)