normalize_for_stream: Incompatible with flexible field access pattern #134116
Description
When using normalize_for_stream together with field_access_pattern: flexible, it doesn't behave right:
POST _ingest/pipeline/_simulate
{
"docs": [
{
"_source": {
"@timestamp": "2024-01-01T00:00:00.000Z",
"message": "test",
"log": {
"level": "info",
"logger": "nginx"
}
}
}
],
"pipeline": {
"processors": [
{
"normalize_for_stream": {}
}
]
}
}
returns
{
"attributes": {
"log.logger": "nginx"
},
"@timestamp": "2024-01-01T00:00:00.000Z",
"body": {
"text": "test"
},
"resource": {
"attributes": {}
},
"severity_text": "info"
}
which is correct. However, setting "field_access_pattern": "flexible" will result in this:
{
"attributes": {
"log.logger": "nginx",
"body.text": "test"
},
"@timestamp": "2024-01-01T00:00:00.000Z",
"resource": {
"attributes": {}
},
"severity_text": "info"
}
POST _ingest/pipeline/_simulate
{
"docs": [
{
"_source": {
"@timestamp": "2024-01-01T00:00:00.000Z",
"message": """{
"@timestamp": "2024-01-01T00:00:00.000Z",
"log.level": "info",
"log.logger": "nginx",
"message": "test"
}"""
}
}
],
"pipeline": {
"processors": [
{
"normalize_for_stream": {}
}
],
"field_access_pattern": "flexible"
}
}
will return
{
"type": "illegal_argument_exception",
"reason": "field [log] not present as part of path [log]"
}
normalize_for_stream shouldn't change behavior no matter the current field_access_mode