Support optional values for Environment Provider

[strawgate]

With #2261 if a customer provides an env var that doesn't exist, the input does not get included in the rendered configuration. This is a fantastic feature.

What would be really great is if we offered an alternative syntax for the env provider which made the env var optional during rendering:

${env.foo?}
${env.?foo}
${?env.foo}
$?{env.foo}
${optional.env.foo}

If the optional env var is not available, it would be lovely if only that setting didn't render. For example in the fleet policy

hosts: 
  - ${env.mysql}
interval: ${optional.env.mysql_interval}

if mysql exists but mysql_interval does not, agent would render

hosts:
  - ${env.mysql}

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[blakerouse]

This is already supported:

${env.foo|''}

If env.foo doesn't exist then '' will be used.

You can even chain them together.

${env.var1|env.var2|env.var3|''}

It will first try for var1, then var2, then var3, and then use the empty string if non are set.

Doesn't look like this is explained anywhere in the documentation. I looked around and couldn't find it, probably should be added on this page here - https://www.elastic.co/guide/en/fleet/current/providers.html

[swiatekm]

It's explained here, under the variables and conditions section.

[nimarezainia]

@blakerouse @swiatekm how would a Fleet based user apply these variables? I believe if the definition is in the elastic-agent.yml file, when agent is installed, it will apply. However perhaps not a variable that can be added later. I am asking in context of this: #3582 (comment)

... where ideally per-policy tagging can have a variable as the tag

[blakerouse]

@nimarezainia It can be applied any time. If you update a setting in an integration to use a variable then it will be replaced in the running Elastic Agent, when that new policy is applied.

[blakerouse]

@swiatekm Thanks for finding that in the documentation. I was not able to find it. I think we can close this issue then as it is fully supported and documented.

[strawgate]

Amazing news! will test this out soon

[nimarezainia]

the docs are under the standalone agent. @kilfoyle I wonder we need to consider moving these somewhere more generic and prominent, document the application to integrations also.

[kilfoyle]

@nimarezainia Certainly. I think the best would be to add a prominent note at least in the Elastic Agent Install docs, to help people find the dynamic config info. Here's a PR: elastic/ingest-docs#1731