Skip to content

Commit 714b1ae

Browse files
jack-bergjack-berg
authored
Define syntax for escaping declarative config env var substitution (#4375)
Resolves #3914. An environment variable substitution reference which starts with `$$` (i.e. `$${API_KEY}`) is escaped. Implementations should strip the leading `$` character and not perform substitution (i.e. `$${API_KEY} => ${API_KEY}`). See java implementation here: open-telemetry/opentelemetry-java#7033
1 parent 3fcb74b commit 714b1ae

File tree

2 files changed

+61
-41
lines changed

2 files changed

+61
-41
lines changed

‎CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,10 @@ release.
3232

3333
### SDK Configuration
3434

35+
- Define syntax for escaping declarative configuration environment variable
36+
references.
37+
([#4375](https://github.com/open-telemetry/opentelemetry-specification/pull/4375))
38+
3539
### Common
3640

3741
### Supplementary Guidelines

‎specification/configuration/data-model.md‎

Lines changed: 57 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,29 @@ more non line break characters (i.e. any character except `\n`). If a referenced
7575
environment variable is not defined and does not have a `DEFAULT_VALUE`, it MUST
7676
be replaced with an empty value.
7777

78+
The `$` character is an escape sequence, such that `$$` in the input is
79+
translated to a single `$` in the output. The resolved `$` from an escape
80+
sequence MUST NOT be considered when matching input against the environment
81+
variable substitution regular expression. For example, `$${API_KEY}` resolves
82+
to `${API_KEY}`, and the value of the `API_KEY` environment variable is NOT
83+
substituted. See table below for more examples. In practice, this implies that
84+
parsers consume the input from left to right, iteratively identifying the next
85+
escape sequence, and matching the content since the prior escape sequence
86+
against the environment variable substitution regular expression.
87+
88+
For example, the pseudocode for processing input `$${FOO} ${BAR} $${BAZ}`
89+
where `FOO=a, BAR=b, BAZ=c` would resemble:
90+
91+
* Identify escape sequence `$$` at index 0. Perform substitution
92+
against `input.substring(0, 0)=""` => `""` and append to output. Append `$`
93+
to output. Current output: `"$"`.
94+
* Identify escape sequence `$$` at index 15. Perform substitution
95+
against `input.substring(0+2, 15)="{FOO} ${BAR} "` => `"{FOO} b "` and append
96+
to output. Append `$` to output. Current output: `"${FOO} b $"`.
97+
* Reach end of input without escape sequence. Perform substitution
98+
against `input.substring(15+2, input.length)="{BAZ}"` => `"{BAZ}"` and append
99+
to output. Return output: `"${FOO} b ${BAZ}"`.
100+
78101
When parsing a configuration file that contains a reference not matching
79102
the references regular expression but does match the following PCRE2
80103
regular expression, the parser MUST return an empty result (no partial
@@ -94,55 +117,48 @@ example, see references to `INVALID_MAP_VALUE` environment variable below.
94117
It MUST NOT be possible to inject environment variable by environment variables.
95118
For example, see references to `DO_NOT_REPLACE_ME` environment variable below.
96119

97-
For example, consider the following environment variables,
98-
and [YAML](#yaml-file-format) configuration file:
120+
The table below demonstrates environment variable substitution behavior for a
121+
variety of inputs. Examples assume environment variables are set as follows:
99122

100123
```shell
101124
export STRING_VALUE="value"
102125
export BOOL_VALUE="true"
103126
export INT_VALUE="1"
104127
export FLOAT_VALUE="1.1"
105-
export HEX_VALUE="0xdeadbeef" # A valid integer value written in hexadecimal
128+
export HEX_VALUE="0xdeadbeef" # A valid integer value (i.e. 3735928559) written in hexadecimal
106129
export INVALID_MAP_VALUE="value\nkey:value" # An invalid attempt to inject a map key into the YAML
107130
export DO_NOT_REPLACE_ME="Never use this value" # An unused environment variable
108131
export REPLACE_ME='${DO_NOT_REPLACE_ME}' # A valid replacement text, used verbatim, not replaced with "Never use this value"
132+
export VALUE_WITH_ESCAPE='value$$' # A valid replacement text, used verbatim, not replaced with "Never use this value"
109133
```
110134

111-
```yaml
112-
string_key: ${STRING_VALUE} # Valid reference to STRING_VALUE
113-
env_string_key: ${env:STRING_VALUE} # Valid reference to STRING_VALUE
114-
other_string_key: "${STRING_VALUE}" # Valid reference to STRING_VALUE inside double quotes
115-
another_string_key: "${BOOL_VALUE}" # Valid reference to BOOL_VALUE inside double quotes
116-
string_key_with_quoted_hex_value: "${HEX_VALUE}" # Valid reference to HEX_VALUE inside double quotes
117-
yet_another_string_key: ${INVALID_MAP_VALUE} # Valid reference to INVALID_MAP_VALUE, but YAML structure from INVALID_MAP_VALUE MUST NOT be injected
118-
bool_key: ${BOOL_VALUE} # Valid reference to BOOL_VALUE
119-
int_key: ${INT_VALUE} # Valid reference to INT_VALUE
120-
int_key_with_unquoted_hex_value: ${HEX_VALUE} # Valid reference to HEX_VALUE without quotes
121-
float_key: ${FLOAT_VALUE} # Valid reference to FLOAT_VALUE
122-
combo_string_key: foo ${STRING_VALUE} ${FLOAT_VALUE} # Valid reference to STRING_VALUE and FLOAT_VALUE
123-
string_key_with_default: ${UNDEFINED_KEY:-fallback} # UNDEFINED_KEY is not defined but a default value is included
124-
undefined_key: ${UNDEFINED_KEY} # Invalid reference, UNDEFINED_KEY is not defined and is replaced with ""
125-
${STRING_VALUE}: value # Invalid reference, substitution is not valid in mapping keys and reference is ignored
126-
recursive_key: ${REPLACE_ME} # Valid reference to REPLACE_ME
127-
# invalid_identifier_key: ${STRING_VALUE:?error} # If uncommented, this is an invalid identifier, it would fail to parse
128-
```
129-
130-
Environment variable substitution results in the following YAML:
131-
132-
```yaml
133-
string_key: value # Interpreted as type string, tag URI tag:yaml.org,2002:str
134-
env_string_key: value # Interpreted as type string, tag URI tag:yaml.org,2002:str
135-
other_string_key: "value" # Interpreted as type string, tag URI tag:yaml.org,2002:str
136-
another_string_key: "true" # Interpreted as type string, tag URI tag:yaml.org,2002:str
137-
string_key_with_quoted_hex_value: "0xdeadbeef" # Interpreted as type string, tag URI tag:yaml.org,2002:str
138-
yet_another_string_key: "value\nkey:value" # Interpreted as type string, tag URI tag:yaml.org,2002:str
139-
bool_key: true # Interpreted as type bool, tag URI tag:yaml.org,2002:bool
140-
int_key: 1 # Interpreted as type int, tag URI tag:yaml.org,2002:int
141-
int_key_with_unquoted_hex_value: 3735928559 # Interpreted as type int, tag URI tag:yaml.org,2002:int
142-
float_key: 1.1 # Interpreted as type float, tag URI tag:yaml.org,2002:float
143-
combo_string_key: foo value 1.1 # Interpreted as type string, tag URI tag:yaml.org,2002:str
144-
string_key_with_default: fallback # Interpreted as type string, tag URI tag:yaml.org,2002:str
145-
undefined_key: # Interpreted as type null, tag URI tag:yaml.org,2002:null
146-
${STRING_VALUE}: value # Interpreted as type string, tag URI tag:yaml.org,2002:str
147-
recursive_key: ${DO_NOT_REPLACE_ME} # Interpreted as type string, tag URI tag:yaml.org,2002:str
148-
```
135+
| YAML - input | YAML - post substitution | Resolved Tag URI | Notes |
136+
|--------------------------------------------|-------------------------------------|---------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|
137+
| `key: ${STRING_VALUE}` | `key: value` | `tag:yaml.org,2002:str` | YAML parser resolves to string |
138+
| `key: ${BOOL_VALUE}` | `key: true` | `tag:yaml.org,2002:bool` | YAML parser resolves to true |
139+
| `key: ${INT_VALUE}` | `key: 1` | `tag:yaml.org,2002:int` | YAML parser resolves to int |
140+
| `key: ${FLOAT_VALUE}` | `key: 1.1` | `tag:yaml.org,2002:float` | YAML parser resolves to float |
141+
| `key: ${HEX_VALUE}` | `key: 0xdeadbeef` | `tag:yaml.org,2002:int` | YAML parser resolves to int `3735928559` |
142+
| `key: "${STRING_VALUE}"` | `key: "value"` | `tag:yaml.org,2002:str` | Double quoted to force coercion to string `"value"` |
143+
| `key: "${BOOL_VALUE}"` | `key: "true"` | `tag:yaml.org,2002:str` | Double quoted to force coercion to string `"true"` |
144+
| `key: "${INT_VALUE}"` | `key: "1"` | `tag:yaml.org,2002:str` | Double quoted to force coercion to string `"1"` |
145+
| `key: "${FLOAT_VALUE}"` | `key: "1.1"` | `tag:yaml.org,2002:str` | Double quoted to force coercion to string `"1.1"` |
146+
| `key: "${HEX_VALUE}"` | `key: "0xdeadbeef"` | `tag:yaml.org,2002:str` | Double quoted to force coercion to string `"0xdeadbeef"` |
147+
| `key: ${env:STRING_VALUE}` | `key: value` | `tag:yaml.org,2002:str` | Alternative `env:` syntax |
148+
| `key: ${INVALID_MAP_VALUE}` | `key: value\nkey:value` | `tag:yaml.org,2002:str` | Map structure resolves to string and _not_ expanded |
149+
| `key: foo ${STRING_VALUE} ${FLOAT_VALUE}` | `key: foo value 1.1` | `tag:yaml.org,2002:str` | Multiple references are injected and resolved to string |
150+
| `key: ${UNDEFINED_KEY}` | `key:` | `tag:yaml.org,2002:null` | Undefined env var is replaced with `""` and resolves to null |
151+
| `key: ${UNDEFINED_KEY:-fallback}` | `key: fallback` | `tag:yaml.org,2002:str` | Undefined env var results in substitution of default value `fallback` |
152+
| `${STRING_VALUE}: value` | `${STRING_VALUE}: value` | `tag:yaml.org,2002:str` | Usage of substitution syntax in keys is ignored |
153+
| `key: ${REPLACE_ME}` | `key: ${DO_NOT_REPLACE_ME}` | `tag:yaml.org,2002:str` | Value of env var `REPLACE_ME` is `${DO_NOT_REPLACE_ME}`, and is _not_ substituted recursively |
154+
| `key: ${UNDEFINED_KEY:-${STRING_VALUE}}` | `${STRING_VALUE}` | `tag:yaml.org,2002:str` | Undefined env var results in substitution of default value `${STRING_VALUE}`, and is _not_ substituted recursively |
155+
| `key: ${STRING_VALUE:?error}` | n/a | n/a | Invalid substitution reference produces parse error |
156+
| `key: $${STRING_VALUE}` | `key: ${STRING_VALUE}` | `tag:yaml.org,2002:str` | `$$` escape sequence is replaced with `$`, `{STRING_VALUE}` does not match substitution syntax |
157+
| `key: $$${STRING_VALUE}` | `key: $value` | `tag:yaml.org,2002:str` | `$$` escape sequence is replaced with `$`, `${STRING_VALUE}` is replaced with `value` |
158+
| `key: $$$${STRING_VALUE}` | `key: $${STRING_VALUE}` | `tag:yaml.org,2002:str` | `$$` escape sequence is replaced with `$`, `$$` escape sequence is replaced with `$`, `{STRING_VALUE}` does not match substitution syntax |
159+
| `key: $${STRING_VALUE:-fallback}` | `${STRING_VALUE:-fallback}` | `tag:yaml.org,2002:str` | `$$` escape sequence is replaced with `$`, `{STRING_VALUE:-fallback}` does not match substitution syntax |
160+
| `key: $${STRING_VALUE:-${STRING_VALUE}}` | `${STRING_VALUE:-value}` | `tag:yaml.org,2002:str` | `$$` escape sequence is replaced with `$`, leaving `{STRING_VALUE:-${STRING_VALUE}}`, `${STRING_VALUE}` is replaced with `value` |
161+
| `key: ${UNDEFINED_KEY:-$${UNDEFINED_KEY}}` | `${STRING_VALUE:-${UNDEFINED_KEY}}` | `tag:yaml.org,2002:str` | `$$` escape sequence is replaced with `$`, leaving `${UNDEFINED_KEY:-` before and `{UNDEFINED_KEY}}` after which do not match substitution syntax |
162+
| `key: ${VALUE_WITH_ESCAPE}` | `value$$` | `tag:yaml.org,2002:str` | Value of env var `VALUE_WITH_ESCAPE` is `value$$`, which is substituted without escaping |
163+
| `key: a $$ b` | `key: a $ b` | `tag:yaml.org,2002:str` | `$$` escape sequence is replaced with `$` |
164+
| `key: a $ b` | `key: a $ b` | `tag:yaml.org,2002:str` | No escape sequence, no substitution references, value is left unchanged |

0 commit comments

Comments
 (0)