Support Fields API in conditional ingest processors

[eyalkoren]

This PR separates the Field APIs into write and read and exposes only the read APIs for use within ingest conditional scripts.
Things to note:

• Immutability: all read APIs, including iterators, must not be able to mutate the source map
• Performance: since the conditional script factory now needs to create script instances with the ctx map, we cannot reuse a single script instance; instead, we precompile the factory and create a script instance in each invocation

[eyalkoren]

@elasticmachine test this please

[jdconrad]

This is a good start! I left some feedback for a few items that need to be looked at.

[eyalkoren]

@jdconrad I had a bit of a mess and had to force push, please review carefully. I went over it and it looks fine, but since it is a small one, better go over everything in details.

[eyalkoren]

The CI failures seem somewhat arbitrary lately, making it hard to tell what needs further investigation and what simply needs another build attempt.

@jdconrad your assistance in this regard would be appreciated. The consistent failures are around the Serverless checks and some bwc tests. If you have any pointers on where to look or what to fix - that'd be great.

[ldematte]

Tests in BWC look unrelated -- there are a lot of ESQL tests failing there. A rebase should help.
The serverless failure instead seems to be related -- the test expects a ScriptException but gets a UnsupportedOperationException. @jdconrad we might need your input here -- seems the exception is failing to get wrapped?

[jdconrad]

I will take another pass through this today.

[jdconrad]

LGTM! Thanks for the iterations.

[jdconrad]

I've reproduced the exception in serverless locally, and what I have found is it's a serialization issue. If the script exception is not serialized then it gets sent directly back as x-content. However, if it is serialized, it gets wrapped in not serializable exception wrapper which is then sent back as a proxy for the script exception in x-content. This is happening because the unsupported operation exception isn't serializable which is thrown when the unmodifiable collection is written to.

In serverless it is being serialized and in non-serverless is isn't. I assume this is due to the way the clusters for the tests are configued, but I haven't looked too closely into this.

For now I would suggest that we check to make sure an exception comes back with the catch_bad_request parameter, but not check what the type of exception is. In the future, we may want to have Painless handle this specific exception in a different way or have elasticsearch exception add unsupported operation exception to its custom serialization list.

[eyalkoren]

For sure this inconsistency should not hold this PR from being merged, so I will adjust my assertion.
However, I am not sure we should just forget about it...

I assume this is due to the way the clusters for the tests are configued, but I haven't looked too closely into this.

If it's a matter of test clusters setup, fixing it is nice-to-have (really nice 🙂 ).
If it's a systematic inconsistency between serverless and non-serverless - I think this should be considered a bug. For example, someone could specifically catch a ScriptException and base a behavior on that, thus worsening inconsistency.

[jdconrad]

This can potentially happen in just normal elasticsearch as well, but would require the requests to be serialized as part of the test cases. Either way this change still LGTM!