ES|QL PresentOverTime function #134355
ES|QL PresentOverTime function #134355
Conversation
Add a new ES|QL function that checks for the presence of a field in the output result. Presence means that the input expression yields any non-null value. Part of elastic#131069
Add unit tests and documentation for the PRESENT function. Part of elastic#131069
…nto feature/esql-present-function
Clean-up of the PRESENT function. Part of elastic#131069
# Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java
- Change intermediate state for using boolean - Add unit tests for PresentAggregatorFunctionTests and PresentGroupingAggregatorFunctionTests Part of elastic#131069
- Add VerifierTests Part of elastic#131069
- Add union_types csv tests Part of elastic#131069
- Fix unit tests Part of elastic#131069
- Comment out TestLogging on CsvTests - Add missing DataTypes to the function Part of elastic#131069
- Improve documentation Part of elastic#131069
- Optimize AggregatorFunctions Part of elastic#131069
- Fix Rest Tests Part of elastic#131069
Add PRESENT_OVER_TIME ES|QL function. Part of elastic#131069
Add PRESENT_OVER_TIME ES|QL function. Part of elastic#131069
Optimize AggregatorFunction Part of elastic#131069
…t-over-time-function
Optimize PresentGroupingAggregatorFunction Part of elastic#131069
…t-over-time-function
Add PresentErrorTests Part of elastic#131069
…t-over-time-function
Add docs Part of elastic#131069
…t-over-time-function
# Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/aggregate/AggregateWritables.java
Add docs Part of elastic#131069
…t-over-time-function # Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/aggregate/AggregateWritables.java
# Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/EsqlFunctionRegistry.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/aggregate/AggregateWritables.java
Fix tests Part of elastic#131069
leontyevdv
added
>enhancement
Team:Analytics
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
🔍 Preview links for changed docs |
ℹ️ Important: Docs version tagging👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version. We use applies_to tags to mark version-specific features and changes. Expand for a quick overviewWhen to use applies_to tags:✅ At the page level to indicate which products/deployments the content applies to (mandatory) What NOT to do:❌ Don't remove or replace information that applies to an older version 🤔 Need help?
|
Fix docs Part of elastic#131069
Add tests that demonstrate and test conversion of boolean values to integers. Part of elastic#131069
There was a problem hiding this comment.
I've left some small comments, but looks good. Thanks @leontyevdv
| tag = "present-by" | ||
| ), | ||
| @Example( | ||
| description = "To check for the presence and return 1 when it's true and 0 when it's false", |
There was a problem hiding this comment.
I think we need to update this doc?
There was a problem hiding this comment.
This is meant to demonstrate how easy it is to convert boolean to integer in order to comply with PromQL in the future. We discussed these tests with @kkrik-es today and I added them and added one as an example to the documentation. Perhaps adding these examples was unnecessary? WDYT
Here is how it looks:
FROM employees
| WHERE emp_no == 10020
| STATS is_present = TO_INTEGER(PRESENT(languages))
;
is_present:integer
0
;
There was a problem hiding this comment.
Ah, I missed that. Thanks for explaining.
| * Similar to {@link Present}, but it is used to check the presence of values over a time series in the given field. | ||
| */ | ||
| public class PresentOverTime extends TimeSeriesAggregateFunction { | ||
| public static final NamedWriteableRegistry.Entry ENTRY = new NamedWriteableRegistry.Entry( |
There was a problem hiding this comment.
Can we add a serialization test or remove serialization support for this class? I don't think we serialize this class.
| if (aggFunc instanceof Avg || aggFunc instanceof AvgOverTime) { | ||
| return AggregateMetricDoubleBlockBuilder.Metric.COUNT; | ||
| } | ||
| if (aggFunc instanceof Present || aggFunc instanceof PresentOverTime) { |
There was a problem hiding this comment.
Great, this function now supports downsampled data :)
# Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java # x-pack/plugin/src/yamlRestTest/resources/rest-api-spec/test/esql/60_usage.yml
Fix tests Part of elastic#131069
3 participants
Add a new ES|QL PRESENT_OVER_TIME function that is similar to PRESENT, but is used to check the presence of values over a time series in the given field.
This has to be merged after #133986
Part of #131069