Add a terminate ingest processor
#114157
Merged
Add a terminate ingest processor
#114157
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This processor simply causes any remaining processors in the pipeline
to be skipped. It will normally be executed conditionally using the
`if` option. (If this pipeline is being called from another pipeline,
the calling pipeline is *not* terminated.)
For example, this:
```
POST /_ingest/pipeline/_simulate
{
"pipeline":
{
"description": "Appends just 'before' to the steps field if the number field
is present, or both 'before' and 'after' if not",
"processors": [
{
"append": {
"field": "steps",
"value": "before"
}
},
{
"terminate": {
"if": "ctx.error != null"
}
},
{
"append": {
"field": "steps",
"value": "after"
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "doc1",
"_source": {
"name": "okay",
"steps": []
}
},
{
"_index": "index",
"_id": "doc2",
"_source": {
"name": "bad",
"error": "oh no",
"steps": []
}
}
]
}
```
returns something like this:
```
{
"docs": [
{
"doc": {
"_index": "index",
"_version": "-3",
"_id": "doc1",
"_source": {
"name": "okay",
"steps": [
"before",
"after"
]
},
"_ingest": {
"timestamp": "2024-10-04T16:25:20.448881Z"
}
}
},
{
"doc": {
"_index": "index",
"_version": "-3",
"_id": "doc2",
"_source": {
"name": "bad",
"error": "oh no",
"steps": [
"before"
]
},
"_ingest": {
"timestamp": "2024-10-04T16:25:20.448932Z"
}
}
}
]
}
```
|
Documentation preview: |
PeteGillinElastic
added
:Data Management/Ingest Node
|
Hi @PeteGillinElastic, I've created a changelog YAML for you. |
|
Hi @PeteGillinElastic, I've updated the changelog YAML for you. |
1 similar comment
|
Hi @PeteGillinElastic, I've updated the changelog YAML for you. |
PeteGillinElastic
force-pushed
the
fix/110218
branch
from
df8c436 to
4574150
Compare
elasticsearchmachine
added
the
Team:Data Management
|
Pinging @elastic/es-data-management (Team:Data Management) |
|
@dakrone Are you the best person to review this, since we were talking about the reroute mechanism earlier? |
dakrone
approved these changes
Oct 8, 2024
💚 Backport successful
|
PeteGillinElastic
added a commit
to PeteGillinElastic/elasticsearch
that referenced
this pull request
Oct 8, 2024
This processor simply causes any remaining processors in the pipeline
to be skipped. It will normally be executed conditionally using the
`if` option. (If this pipeline is being called from another pipeline,
the calling pipeline is *not* terminated.)
For example, this:
```
POST /_ingest/pipeline/_simulate
{
"pipeline":
{
"description": "Appends just 'before' to the steps field if the number field
is present, or both 'before' and 'after' if not",
"processors": [
{
"append": {
"field": "steps",
"value": "before"
}
},
{
"terminate": {
"if": "ctx.error != null"
}
},
{
"append": {
"field": "steps",
"value": "after"
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "doc1",
"_source": {
"name": "okay",
"steps": []
}
},
{
"_index": "index",
"_id": "doc2",
"_source": {
"name": "bad",
"error": "oh no",
"steps": []
}
}
]
}
```
returns something like this:
```
{
"docs": [
{
"doc": {
"_index": "index",
"_version": "-3",
"_id": "doc1",
"_source": {
"name": "okay",
"steps": [
"before",
"after"
]
},
"_ingest": {
"timestamp": "2024-10-04T16:25:20.448881Z"
}
}
},
{
"doc": {
"_index": "index",
"_version": "-3",
"_id": "doc2",
"_source": {
"name": "bad",
"error": "oh no",
"steps": [
"before"
]
},
"_ingest": {
"timestamp": "2024-10-04T16:25:20.448932Z"
}
}
}
]
}
```
PeteGillinElastic
added a commit
that referenced
this pull request
Oct 9, 2024
This processor simply causes any remaining processors in the pipeline
to be skipped. It will normally be executed conditionally using the
`if` option. (If this pipeline is being called from another pipeline,
the calling pipeline is *not* terminated.)
For example, this:
```
POST /_ingest/pipeline/_simulate
{
"pipeline":
{
"description": "Appends just 'before' to the steps field if the number field
is present, or both 'before' and 'after' if not",
"processors": [
{
"append": {
"field": "steps",
"value": "before"
}
},
{
"terminate": {
"if": "ctx.error != null"
}
},
{
"append": {
"field": "steps",
"value": "after"
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "doc1",
"_source": {
"name": "okay",
"steps": []
}
},
{
"_index": "index",
"_id": "doc2",
"_source": {
"name": "bad",
"error": "oh no",
"steps": []
}
}
]
}
```
returns something like this:
```
{
"docs": [
{
"doc": {
"_index": "index",
"_version": "-3",
"_id": "doc1",
"_source": {
"name": "okay",
"steps": [
"before",
"after"
]
},
"_ingest": {
"timestamp": "2024-10-04T16:25:20.448881Z"
}
}
},
{
"doc": {
"_index": "index",
"_version": "-3",
"_id": "doc2",
"_source": {
"name": "bad",
"error": "oh no",
"steps": [
"before"
]
},
"_ingest": {
"timestamp": "2024-10-04T16:25:20.448932Z"
}
}
}
]
}
```
PeteGillinElastic
added a commit
to PeteGillinElastic/elasticsearch
that referenced
this pull request
Oct 9, 2024
A docs page for the `terminate` processor was added in elastic#114157, but the change to include it in the outer processor reference page was omitted. This change corrects that oversight.
PeteGillinElastic
added a commit
that referenced
this pull request
Oct 10, 2024
A docs page for the `terminate` processor was added in #114157, but the change to include it in the outer processor reference page was omitted. This change corrects that oversight.
PeteGillinElastic
added a commit
to PeteGillinElastic/elasticsearch
that referenced
this pull request
Oct 10, 2024
A docs page for the `terminate` processor was added in elastic#114157, but the change to include it in the outer processor reference page was omitted. This change corrects that oversight.
elasticsearchmachine
pushed a commit
that referenced
this pull request
Oct 10, 2024
A docs page for the `terminate` processor was added in #114157, but the change to include it in the outer processor reference page was omitted. This change corrects that oversight.
matthewabbott
pushed a commit
to matthewabbott/elasticsearch
that referenced
this pull request
Oct 10, 2024
This processor simply causes any remaining processors in the pipeline
to be skipped. It will normally be executed conditionally using the
`if` option. (If this pipeline is being called from another pipeline,
the calling pipeline is *not* terminated.)
For example, this:
```
POST /_ingest/pipeline/_simulate
{
"pipeline":
{
"description": "Appends just 'before' to the steps field if the number field
is present, or both 'before' and 'after' if not",
"processors": [
{
"append": {
"field": "steps",
"value": "before"
}
},
{
"terminate": {
"if": "ctx.error != null"
}
},
{
"append": {
"field": "steps",
"value": "after"
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "doc1",
"_source": {
"name": "okay",
"steps": []
}
},
{
"_index": "index",
"_id": "doc2",
"_source": {
"name": "bad",
"error": "oh no",
"steps": []
}
}
]
}
```
returns something like this:
```
{
"docs": [
{
"doc": {
"_index": "index",
"_version": "-3",
"_id": "doc1",
"_source": {
"name": "okay",
"steps": [
"before",
"after"
]
},
"_ingest": {
"timestamp": "2024-10-04T16:25:20.448881Z"
}
}
},
{
"doc": {
"_index": "index",
"_version": "-3",
"_id": "doc2",
"_source": {
"name": "bad",
"error": "oh no",
"steps": [
"before"
]
},
"_ingest": {
"timestamp": "2024-10-04T16:25:20.448932Z"
}
}
}
]
}
```
matthewabbott
pushed a commit
to matthewabbott/elasticsearch
that referenced
this pull request
Oct 10, 2024
A docs page for the `terminate` processor was added in elastic#114157, but the change to include it in the outer processor reference page was omitted. This change corrects that oversight.
davidkyle
pushed a commit
to davidkyle/elasticsearch
that referenced
this pull request
Oct 13, 2024
This processor simply causes any remaining processors in the pipeline
to be skipped. It will normally be executed conditionally using the
`if` option. (If this pipeline is being called from another pipeline,
the calling pipeline is *not* terminated.)
For example, this:
```
POST /_ingest/pipeline/_simulate
{
"pipeline":
{
"description": "Appends just 'before' to the steps field if the number field
is present, or both 'before' and 'after' if not",
"processors": [
{
"append": {
"field": "steps",
"value": "before"
}
},
{
"terminate": {
"if": "ctx.error != null"
}
},
{
"append": {
"field": "steps",
"value": "after"
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "doc1",
"_source": {
"name": "okay",
"steps": []
}
},
{
"_index": "index",
"_id": "doc2",
"_source": {
"name": "bad",
"error": "oh no",
"steps": []
}
}
]
}
```
returns something like this:
```
{
"docs": [
{
"doc": {
"_index": "index",
"_version": "-3",
"_id": "doc1",
"_source": {
"name": "okay",
"steps": [
"before",
"after"
]
},
"_ingest": {
"timestamp": "2024-10-04T16:25:20.448881Z"
}
}
},
{
"doc": {
"_index": "index",
"_version": "-3",
"_id": "doc2",
"_source": {
"name": "bad",
"error": "oh no",
"steps": [
"before"
]
},
"_ingest": {
"timestamp": "2024-10-04T16:25:20.448932Z"
}
}
}
]
}
```
davidkyle
pushed a commit
to davidkyle/elasticsearch
that referenced
this pull request
Oct 13, 2024
A docs page for the `terminate` processor was added in elastic#114157, but the change to include it in the outer processor reference page was omitted. This change corrects that oversight.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This processor simply causes any remaining processors in the pipeline to be skipped. It will normally be executed conditionally using the
ifoption. (If this pipeline is being called from another pipeline, the calling pipeline is not terminated.)For example, this:
returns something like this:
closes #110218