Add MV_CONTAINS function

[mjmbischoff]

Adding the MV_CONTAINS_ALL function.

The function checks if the multivalue expression that is supplied as the first argument resolves to all values yielded by the multivalue expression that is the second argument. Effectively establishing if the first argument is a super set of the second argument which would then be the subset. This allows for a contains element function that also has the flexibility to work with multivalue arguments. Since we compare each entry duplicates collapse as we simply match against the same entry on the other side.

This avoids the need to abuse string functions to check if something is one of the multi values.:

FROM main_data_index
| LOOKUP JOIN my_lookup_index ON destination.ip
| WHERE CONTAINS(CONCAT("#",MV_CONCAT(destination.denied_ports::STRING, "#"),"#"), CONCAT("#",destination.port::STRING,"#"))
| KEEP message, destination.*

FROM main_data_index
| LOOKUP JOIN my_lookup_index ON destination.ip
| WHERE MV_CONTAINS(destination.denied_ports, destination.port)
| KEEP message, destination.*

the function allows for multivalue so the following should also work:

MV_CONTAINS(roles, ["read", "write"])

---

TODO

• Regenerate docs
• Rename MV_CONTAINS_ALL to MV_CONTAINS (pending decision - jumped the gun but should check for consensus)
• Introduce as PREVIEW or straight to GA?

[github-actions]

🔍 Preview links for changed docs

• docs/reference/query-languages/esql/kibana/docs/functions/mv_contains.md
• docs/reference/query-languages/esql/functions-operators/mv-functions.md

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[elasticsearchmachine]

Hi @mjmbischoff, I've created a changelog YAML for you.

[elasticsearchmachine]

Hi @mjmbischoff, I've updated the changelog YAML for you.

[mjmbischoff]

If we do want to introduce infix operators for this, and want the operator to be IN, then it would be correct for it to have the opposite parameter order from the MV_CONTAINS_ALL function.

Agree. If we add an infix operator will create a class for it as invert the operator order, but that will be a followup PR.

[mjmbischoff]

Had to do quite a bit(too much?) to follow the contract of always returning a boolean vs null if any argument is null, which the test classes seem to expect.

[ivancea]

It looks good! It's a shame you had to create a custom evaluator; I guess we'll have to invest some extra time there, as MV functions can usually handle nulls

[elasticsearchmachine]

💔 Backport failed

The backport operation could not be completed due to the following error:

There are no branches to backport to. Aborting.

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 133099

[DaveCTurner]

This commit introduced compile errors, I have reverted it from main.