Adding Contains ESQL String function

[mjmbischoff]

Adding contains ES:QL function, this function checks if a substring is contained within the given string, returning a boolean. This is equivalent with using locate and checkling for !=0

[github-actions]

🔍 Preview links for changed docs

• docs/reference/query-languages/esql/kibana/docs/functions/contains.md
• docs/reference/query-languages/esql/functions-operators/string-functions.md

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[elasticsearchmachine]

Hi @mjmbischoff, I've created a changelog YAML for you.

[mjmbischoff]

If merged, can rewrite the following:

FROM main_data_index
| LOOKUP JOIN my_lookup_index ON destination.ip
| WHERE LOCATE(CONCAT("#",MV_CONCAT(destination.denied_ports::STRING, "#"),"#"), CONCAT("#",destination.port::STRING,"#"))!=0
| KEEP message, destination.*

FROM main_data_index
| LOOKUP JOIN my_lookup_index ON destination.ip
| WHERE CONTAINS(CONCAT("#",MV_CONCAT(destination.denied_ports::STRING, "#"),"#"), CONCAT("#",destination.port::STRING,"#"))
| KEEP message, destination.*

But a nicer way to check multi valued fields for a value is still needed: #120782

[mjmbischoff]

#98545 (comment) should probably be updated.

[nik9000]

If merged, can rewrite the following:

terrible code
into
terrible code

I'm so sorry we haven't written a better way yet.

[nik9000]

Could you drop the null checks in the eval method please? They are confusing because those things can't be null because of the @Evaluator annotation. Makes a reader have to go and look up stuff in the generated code.

[nik9000]

I think it's probably faster to do it by hand without converting to a String first. But this is fine for now.

[mjmbischoff]

I think so as well, but having been bitten by character encoding subtleties before.. 😅

[nik9000]

Right. ESQL deals in utf-8 naturally like Elasticsearch and Lucene mostly do. It's probably fine to do the contains on raw strings because utf-8 is like that. But it's worth making super sure before you do it. So it can wait.

[nik9000]

Looks great - I asked for two things that should happen before merging (the capability and the null check) and one thing you can do, or it can wait for a while (replace utf8ToString with a loop checking).

[mjmbischoff]

While I think we need a 'passAlongNulls'/'dontHandleNulls' option for the generator; Here I'm perfectly fine with the handling, as all string functions seem to return null if any of the arguments are null. (For some it's undocumented - should probably be fixed)

[nik9000]

Here I'm perfectly fine with the handling, as all string functions seem to return null if any of the arguments are null.

95% of our functions have "any null input returns null" semantics which is very SQL. Very in the spirit of "null means unknown" which SQL likes. In Elasticsearch null often means "empty list" more than "unknown". Which is one of the really sneaky things about all this.

But in this case I think "any null input returns null" is natural and good.

[nik9000]

For some it's undocumented - should probably be fixed

If you have a list or want to add the docs, that'd be awesome.

[mjmbischoff]

For some it's undocumented - should probably be fixed

If you have a list or want to add the docs, that'd be awesome.

I've now added it to my (long) TODO list. 😅

[elasticsearchmachine]

💔 Backport failed

The backport operation could not be completed due to the following error:

There are no branches to backport to. Aborting.

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 133016