Add option for Append Processor to skip/allow empty values

[limotova]

If one is using templates to generate values, this option can be helpful to avoid adding values that resolve into empty strings

Fixes #104813

[github-actions]

Documentation preview:

• ✨ Changed pages

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[joegallo]

I'd like to have this mirror the set processor's ignore_empty_value option, rather than creating a new option name with no precedent -- it's functionally the same as that option, right (albeit with the opposite sense)?

[romulets]

This feature is actually pretty useful. I'm my use-case, I'm appending a lot of fields. To keep checking if they are not null on context is pretty repetitive. Right now we have a painless script that removes all empty values. But would be awesome to have this out of the box.

[elasticsearchmachine]

Hi @limotova, I've created a changelog YAML for you.

[github-actions]

It looks like this PR modifies one or more .asciidoc files. These files are being migrated to Markdown, and any changes merged now will be lost. See the migration guide for details.

[joegallo]

Oops, my one commit comment got truncated because I wasn't careful about syntax, that was intended to read:

Rewrite another setFieldValue arity, too

This one is a bit trickier, so I've lampshaded the surprising part
with a comment, and also there's a newly added test on main via
#134217

[joegallo]

I merged main into this PR again, so that brings #134217 and #134319 in. They all play nicely together on my machine but we'll see what CI has to say about it.

[joegallo]

A todo list for myself of what I'd still like to see here:

• I need to resurrect the docs changes in markdown rather than asciidoc, including the example(s) I promised earlier in Add copy_from option to the Append processor #132003 (comment)
• I need to add a test only feature, and also some rest tests
• I think it makes sense to add more test coverage of ignore_empty_values with copy_from (that could probably be done with the previously mentioned rest tests)

[limotova]

I can't approve my own PR haha but the changes look reasonable to me, thank you!

[github-actions]

🔍 Preview links for changed docs

• docs/reference/enrich-processor/append-processor.md

[github-actions]

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

• Check out the cumulative docs guidelines
• Reach out in the #docs Slack channel