Skip to content

Conversation

timgrein
Copy link
Contributor

@timgrein timgrein commented Aug 12, 2025

Addresses ESQL: Add decay functions

This PR introduces the decay function for numeric, spatial and temporal data types. For better intuition on decay functions you can read up on the detailed example on the decay function in the ES docs.

Key differences to the existing decay function:

  • ESQL decay also works with dates using nanosecond precision (type: date_nanos)
  • ESQL decay also works on cartesian points (using euclidean distance)

The function supports the following data types for the main input (value): int, double, long, geo_point, cartesian_point, datetime, date_nanos.

Numeric (int) example:

Creating the index:

curl --location --request PUT 'http://localhost:9200/decay_integer_linear_index' \
--header 'Content-Type: application/json' \
--data '{
  "mappings": {
    "properties": {
      "value": {
        "type": "integer"
      }
    }
  }
}'

Indexing sample data:

curl --location 'http://localhost:9200/_bulk' \
--header 'Content-Type: application/json' \
--data '{ "index": { "_index": "decay_integer_linear_index", "_id": "1" } }
{ "value": "0" }
{ "index": { "_index": "decay_integer_linear_index", "_id": "2" } }
{ "value": "10" }
{ "index": { "_index": "decay_integer_linear_index", "_id": "3" } }
{ "value": "50" }
{ "index": { "_index": "decay_integer_linear_index", "_id": "4" } }
{ "value": "100" }
{ "index": { "_index": "decay_integer_linear_index", "_id": "5" } }
{ "value": "2500" }
'

Querying using the Query DSL:

curl --location 'http://localhost:9200/decay_integer_linear_index/_search' \
--header 'Content-Type: application/json' \
--data '{
  "query": {
    "function_score": {
      "linear": {
        "value": {
          "origin": "0",
          "scale": "10",
          "offset": "5",
          "decay": 0.5
        }
      }
    }
  }
}'

Querying using ES|QL:

curl --location 'http://localhost:9200/_query?format=txt' \
--header 'Content-Type: application/json' \
--data '{
    "query": "FROM decay_integer_linear_index | KEEP value | EVAL score = decay(value, 0, 10, {\"offset\": 5, \"decay\": 0.5, \"type\": \"linear\"})"
}'

Spatial (GeoPoint) example:

Creating the index:

curl --location --request PUT 'http://localhost:9200/decay_geo_point_linear_index' \
--header 'Content-Type: application/json' \
--data '{
  "mappings": {
    "properties": {
      "value": {
        "type": "geo_point"
      }
    }
  }
}'

Indexing sample data:

curl --location 'http://localhost:9200/_bulk' \
--header 'Content-Type: application/json' \
--data '{ "index": { "_index": "decay_geo_point_linear_index", "_id": "1" } }
{ "value": "POINT (0 0)" }
{ "index": { "_index": "decay_geo_point_linear_index", "_id": "2" } }
{ "value": "POINT (1 1)" }
{ "index": { "_index": "decay_geo_point_linear_index", "_id": "3" } }
{ "value": "POINT (180 90)" }
{ "index": { "_index": "decay_geo_point_linear_index", "_id": "4" } }
{ "value": "POINT (-180 -90)" }
{ "index": { "_index": "decay_geo_point_linear_index", "_id": "5" } }
{ "value": "POINT (12.3 45.6)" }
'

Querying using the Query DSL:

curl --location 'http://localhost:9200/decay_geo_point_linear_index/_search' \
--header 'Content-Type: application/json' \
--data '{
  "query": {
    "function_score": {
      "linear": {
        "value": {
          "origin": "POINT (1 1)",
          "scale": "10000km",
          "offset": "10km",
          "decay": 0.33
        }
      }
    }
  }
}'

Querying using ES|QL:

curl --location 'http://localhost:9200/_query?format=txt' \
--header 'Content-Type: application/json' \
--data '{
    "query": "FROM decay_geo_point_linear_index | KEEP value | EVAL score = decay(TO_GEOPOINT(value), TO_GEOPOINT(\"POINT (1 1)\"), \"10000km\", {\"offset\": \"10km\", \"decay\": 0.33, \"type\": \"linear\"})"
}'

Temporal (Datetime) example:

Creating the index:

curl --location --request PUT 'http://localhost:9200/decay_datetime_linear_index' \
--header 'Content-Type: application/json' \
--data '{
  "mappings": {
    "properties": {
      "value": {
        "type": "date"
      }
    }
  }
}'

Indexing sample data:

curl --location 'http://localhost:9200/_bulk' \
--header 'Content-Type: application/json' \
--data '{ "index": { "_index": "decay_datetime_linear_index", "_id": "1" } }
{ "value": "2025-08-20" }
{ "index": { "_index": "decay_datetime_linear_index", "_id": "2" } }
{ "value": "2020-08-20" }
{ "index": { "_index": "decay_datetime_linear_index", "_id": "3" } }
{ "value": "2000-01-01" }
{ "index": { "_index": "decay_datetime_linear_index", "_id": "4" } }
{ "value": "1970-08-20" }
{ "index": { "_index": "decay_datetime_linear_index", "_id": "5" } }
{ "value": "1900-12-12" }
'

Querying using the Query DSL:

curl --location 'http://localhost:9200/decay_datetime_linear_index/_search' \
--header 'Content-Type: application/json' \
--data '{
  "query": {
    "function_score": {
      "linear": {
        "value": {
          "origin": "2000-01-01",
          "scale": "10000d",
          "offset": "10d",
          "decay": 0.33
        }
      }
    }
  }
}'

Querying using ES|QL:

curl --location 'http://localhost:9200/_query?format=txt' \
--header 'Content-Type: application/json' \
--data '{
    "query": "FROM decay_datetime_linear_index | KEEP value | EVAL score = decay(TO_DATETIME(value), TO_DATETIME(\"2000-01-01T00:00:00Z\"), 240000 hours, {\"offset\": 240 hours, \"decay\": 0.33, \"type\": \"linear\"})"
}'

@elasticsearchmachine elasticsearchmachine added v9.2.0 needs:triage Requires assignment of a team area label labels Aug 12, 2025
@timgrein timgrein marked this pull request as draft August 13, 2025 08:21
Copy link
Contributor

@ioanatia ioanatia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work!
I checked the last review from @carlosdelest and it looks like you addressed all comments.

Copy link
Contributor

@leemthompo leemthompo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs look good for this first iteration 👍

@timgrein timgrein merged commit 88f6798 into elastic:main Sep 9, 2025
33 checks passed
elasticsearchmachine pushed a commit that referenced this pull request Sep 9, 2025
…SET and INLINESTATS commands) (#134358)

Three PRs have just been merged that cause failures in `test-release`,
and a third PR changes ES|QL railroad diagrams for another recently
released change to `KNN` (also a snapshot-only feature).

* #132729 - `DecayTests` has over 300 failures
* #134029 - `ParsingTests` had six new failures
* #124725 - Failures in `VerifierTests` and `OptimizerVerificationTests`

This PR attempts to fix all in one, because if we do this in separate
PRs they will fail due to the failures cause by the other PRs.
rjernst pushed a commit to rjernst/elasticsearch that referenced this pull request Sep 9, 2025
Kubik42 pushed a commit to Kubik42/elasticsearch that referenced this pull request Sep 9, 2025
Kubik42 pushed a commit to Kubik42/elasticsearch that referenced this pull request Sep 9, 2025
…SET and INLINESTATS commands) (elastic#134358)

Three PRs have just been merged that cause failures in `test-release`,
and a third PR changes ES|QL railroad diagrams for another recently
released change to `KNN` (also a snapshot-only feature).

* elastic#132729 - `DecayTests` has over 300 failures
* elastic#134029 - `ParsingTests` had six new failures
* elastic#124725 - Failures in `VerifierTests` and `OptimizerVerificationTests`

This PR attempts to fix all in one, because if we do this in separate
PRs they will fail due to the failures cause by the other PRs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>non-issue :Search Relevance/ES|QL Search functionality in ES|QL Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants