Skip to content

Conversation

@JonasKunz
Copy link
Contributor

@JonasKunz JonasKunz commented Nov 5, 2025

Adds a very basic CSV test for exponential histograms.
The dataset consists of some pathological dummy values alongside with a few real response time histograms.

Adapts the CSV test infrastructure to work with exponential histograms and hopefully doesn't break any bwc / release tests. Please take a closer look at that, as I'm really new to the ESQL testing world.

We don't have a ES test node feature for support of the exponential_histogram field type in elasticsearch, as that is hidden behind a feature flag aswell. But I think using the ES|QL capability should be a good enough surrogate, as we are planning on lifting those at the same time (or at least closely one after another).

@elasticsearchmachine elasticsearchmachine added v9.3.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Nov 5, 2025
@JonasKunz JonasKunz marked this pull request as ready for review November 5, 2025 15:18
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Nov 5, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

Copy link
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine.

JSON in these is pretty gnarly. For date_ranges we're doing the rust-like 2025-01-01..2026-01-01. We'll do that for ip and numeric ranges too. But these complex things already have json parsing code and it's compelling to reuse it. A local minima, maybe.

Maybe we should do a """ to support un-escaped ". It'd be easier to embed json, I guess.

@JonasKunz
Copy link
Contributor Author

JSON in these is pretty gnarly

Indeed, but I think querying raw exponential histograms will rarely happen in the CSV tests. We'll rather look at the aggregations (e.g. PERCENTILE) instead.

Maybe we should do a """ to support un-escaped ". It'd be easier to embed json, I guess.

Is this some CSV standard or are you thinking of a custom implementation? I'm only aware of the "" to generate single quotes.

We could also make our CSV-test matching more lenient and omit the quotes completely IINM.

@JonasKunz JonasKunz merged commit 87e82d1 into elastic:main Nov 7, 2025
34 checks passed
@JonasKunz JonasKunz deleted the exp-histo-csv-tests branch November 7, 2025 09:00
Kubik42 pushed a commit to Kubik42/elasticsearch that referenced this pull request Nov 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL external-contributor Pull request authored by a developer outside the Elasticsearch team >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants