How to force string value with variable substitution?

[artem-nefedov]

I'm using variable substitution to replace values in the manifests applied by Kustomization (using substituteFrom).
In my case, I have a value that contains only numbers, but I need it applied as a string.

If I do foo: '${myvar}' or foo: "${myvar}", it does not work, and instead I get an integer value: foo: 123.
If I do foo: '"${myvar}"', it also does not work, and instead I get a string value with literal quotes inside: foo: '"123"'.

How do I get foo: "123"?

[kingdonb]

foo: "\"${foo}\""

This is super awkward, but I believe this is how it's done. There was a thread on the CNCF slack #flux about this, credit to Ben A for finding the workaround. https://cloud-native.slack.com/archives/CLAJ40HV3/p1623953195127500?thread_ts=1623949464.122200&cid=CLAJ40HV3

[rmartine-ias]

This workaround no longer works. See here for our solution: #1839 (comment)

[kingdonb]

This is not ideal as a solution, but you're right, the old solution does not work (since already maybe 10 minor versions ago) and it does not work in 2.0.0-rc.1, the current latest version.

There is an answer for this, this is an unfortunate limitation of using envsubst. I think I understand the issue as being that the YAML gets parsed and quotes get consumed, before envsubst happens. Templating can be done with a pre-parse rather than post-YAML-parsing solution, with a templating solution like one in Weave GitOps Enterprise as example:

• https://docs.gitops.weave.works/docs/gitops-templates/supported-langs/

Note the "templating" kind, which has lots of advantages over envsubst.

Then you have access to Go templating functions, you can do advanced things like parsing a semver into a struct, or simple things like ensure the value gets quoted when it's substituted.

This doesn't solve it for OSS Flux users, but it is a solution that might work, depending on your use case. I'm glad you found a workaround for now.

[fgeck]

Is there any workaround when the variable substitution is not done in a helm template, but e.g. for a standard cert-manager certificate resource?
Example where CERTIFICATE_DOMAIN is a wildcard like *.myfancydomain.com :

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: "${CERTIFICATE_NAME}"
  namespace: edge
spec:
  secretName: "${CERTIFICATE_NAME}"
  issuerRef:
    name: my-cert
    kind: ClusterIssuer
  dnsNames:
    - "${CERTIFICATE_DOMAIN}"

[stefanprodan]

To force the conversion of variables to string see the docs here: https://fluxcd.io/flux/components/kustomize/kustomizations/#post-build-substitution-of-numbers-and-booleans

[kaden-l-nelson]

This seems really heavy handed for something that objectively should be simple. 😅

Is there a better way?

[solidDoWant]

Once fluxcd/pkg#948 and/or fluxcd/pkg#947 are resolved (flux bugs), you can do this:

some-key: ${A_VARIABLE/#/!<tag:yaml.org,2002:str> }

This uses Flux's bash variable substitution to match a prefix of "" (empty string). This replaces the empty string (which is just prepending the variable value) with !<tag:yaml.org,2002:str> , which is a YAML reference tag that tells the YAML parser that the value of string.

Given a variable SOME_VARIABLE=1234, this gets processed as follows (as I understand it):

1. Flux parses the manifest into a data structure within the program. Internally, this sets some-key to ${A_VARIABLE/#/!<tag:yaml.org,2002:str> }. Because Flux is parsing this, any quotes are stripped (which is where the discussed quoting issue appears).
2. Flux replaces the renders the variables. The value ${A_VARIABLE/#/!<tag:yaml.org,2002:str> } turns into !<tag:yaml.org,2002:str> 1234, so some-key is set to !<tag:yaml.org,2002:str> 1234.
3. Flux parses the rendered manifest to send to the API server. Because the value is prepended by the !<tag:yaml.org,2002:str> reference string, the parser treats the value 1234 as a string, rather than a number.

[ringerc]

I have a refinement for this approach that works when a default value is also desired; see #1795 (comment)

The above does not, because flux doesn't support zsh-style nested substititions like ${${VARNAME:=default}/find/replace}

[ringerc]

As a (gruesome) workaround, "protect" a YAML type-tag from earlier passes with ${DELIBERATELY_UNDEFINED:=!!str }, so it is injected as a prefix to the real value at the same time as the flux substitution is applied. Unlike using double quotes, this won't mangle the value if the input is actually already a string.

Handily this also works to protect truth-ish values by ensuring they also get properly quoted.

Example input:

kind: fake
metadata:
  name: fake
value: '${DELIBERATELY_UNDEFINED:=!!str }${MYVARIABLE:=10000000}'

Output of flux-envsubst < example.yaml | yq '.'

kind: fake
metadata:
  name: fake
value: "10000000"

Output of MYVARIABLE=100 flux-envsubst example.yaml < example.yaml| yq '.'

kind: fake
metadata:
  name: fake
value: "100"

Output of MYVARIABLE="string valued" flux-envsubst < example.yaml | yq '.':

kind: fake
metadata:
  name: fake
value: string valued

Note that the string value is injected correctly, without unwanted double-double quotes.

Note that this trick will emit an unwanted log message like:

variable "DELIBERATELY_UNDEFINED" might be missing

that there isn't much to be done about.

This is related to the way yaml libs like that used by kustomize strip quoting that's not syntactically required at that time; see kubernetes-sigs/kustomize#4845

This is still kind of a code-crime, it's a really ugly workaround for using a text-based substitution that's not fully yaml-aware. Using a fully yaml-aware tool like ytt probably wouldn't have this issue. But the workaround works, and while tools like kustomize are so opposed to supporting a richer value-tranformation and substitution system, post-processing steps like this are likely to stay with us.