Skip to content

ESQL: Add decay function #121526

@ioanatia

Description

@ioanatia

We already support the following decay functions in painless for the script_score query:

double decayNumericLinear(double origin, double scale, double offset, double decay, double docValue)
double decayNumericExp(double origin, double scale, double offset, double decay, double docValue)
double decayNumericGauss(double origin, double scale, double offset, double decay, double docValue)

double decayGeoLinear(String originStr, String scaleStr, String offsetStr, double decay, GeoPoint docValue)
double decayGeoExp(String originStr, String scaleStr, String offsetStr, double decay, GeoPoint docValue)
double decayGeoGauss(String originStr, String scaleStr, String offsetStr, double decay, GeoPoint docValue)

double decayDateLinear(String originStr, String scaleStr, String offsetStr, double decay, JodaCompatibleZonedDateTime docValueDate)
double decayDateExp(String originStr, String scaleStr, String offsetStr, double decay, JodaCompatibleZonedDateTime docValueDate)
double decayDateGauss(String originStr, String scaleStr, String offsetStr, double decay, JodaCompatibleZonedDateTime docValueDate)

Decay functions are used to penalize scores based on recency (for date types) or distance (numeric and geo types) and this is a useful feature we want to add to ES|QL.

We do not want to add 9 different decay functions.

We will have one decay function, that default to linear decay:

decay(value, origin, scale) // defaults to linear
decay(value, origin, scale,  { "offset": 0.5 , "decay": 0, "type": "exp"}) // use a different type

For parameters:

Value: numeric, spatial, date types - required
Origin: numeric, spatial, date types - required
Scale: numeric, spatial, date types - required

Options:

Offset: numeric, spatial, date types - defaults to 0.5 (same as https://www.elastic.co/docs/reference/query-languages/query-dsl/query-dsl-function-score-query#function-decay)
Decay: numeric - default to 0 (same as https://www.elastic.co/docs/reference/query-languages/query-dsl/query-dsl-function-score-query#function-decay)
Type: can be "linear" (default), "exp" or `"gauss"

The return type should be double.

How to apply score decay

It should be as simple as:

FROM index METADATA _score
| ....
| EVAL final_score = _score - decay(value, origin, scale)

Metadata

Metadata

Assignees

Labels

:Search Relevance/SearchCatch all for Search Relevance>featureES|QL-uiImpacts ES|QL UITeam:Search RelevanceMeta label for the Search Relevance team in Elasticsearchpriority:normalA label for assessing bug priority to be used by ES engineers

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions