Skip to content

Conversation

@masseyke
Copy link
Member

@masseyke masseyke commented Mar 17, 2025

The mapping for the deprecation logger defines data_stream.type, data_stream.dataset, and data_stream.namespace as constant_keyword fields, with data_stream.type being restricted to being logs. These fields can have different values for different data streams though. Assuming its type is logs, the first entry written doesn't cause a problem because the mapping is not strict and gets updated to have the constant be whatever was inserted. But if the deprecation logger then writes a different value to one of the constant_keyword fields, we get an error like this (seen on an actual cluster in the process of upgrading):

[instance-0000000005] Bulk write of deprecation logs encountered some failures: [[4VK3j5UBamJ0EqvgUJmJ org.elasticsearch.index.mapper.DocumentParsingException: [1:87] failed to parse field [data_stream.dataset] of type [constant_keyword] in document with id '4VK3j5UBamJ0EqvgUJmJ'. Preview of field's value: 'elasticsearch.deprecation']]

@masseyke masseyke added v9.0.0 v8.19.0 :Core/Infra/Logging Log management and logging utilities v8.18.0 v8.18.1 auto-backport Automatically create backport pull requests when merged labels Mar 17, 2025
@masseyke masseyke marked this pull request as ready for review March 17, 2025 18:37
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@elasticsearchmachine elasticsearchmachine added the Team:Core/Infra Meta label for core/infra team label Mar 17, 2025
@masseyke masseyke added the >bug label Mar 17, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @masseyke, I've created a changelog YAML for you.

@masseyke masseyke requested a review from a team March 18, 2025 13:56
@masseyke
Copy link
Member Author

I was mistaken about the problem here. The type, dataset, and namespace don't change (they are the type/dataset/namespace of the deprecation logger data stream, not metadata about whatever is being logged). This was probably caused by the deprecation logger dataset name being changed in #117933.

@masseyke masseyke closed this Mar 20, 2025
@masseyke masseyke deleted the fix/deprecation-logger-mapping branch March 20, 2025 22:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged >bug :Core/Infra/Logging Log management and logging utilities Team:Core/Infra Meta label for core/infra team v8.18.0 v8.18.1 v8.19.0 v9.0.0 v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants