Use split_doc with other documents in the same stream

[mathewpeterson]

Hello,

I am trying to handle an edge case when working with "lists" resources (ConfigMapList, RoleBindingList, RoleList) that are generated by a Helm chart. Since these are not real KRDs, I would like to split all items from the list into their own document so that I can save them to disk.

Typically, this is very easy with the split_doc operator since it works exactly as expected. However the issue I am running into is when there are other non "list" documents in the YAML stream as I need those, too.

Say I have a stream of documents like:

# all.yaml
apiVersion: v1
kind: ConfigMapList
items:
  - apiVersion: v1
    kind: ConfigMap
    metadata:
      name: one
    data:
      one: one
  - apiVersion: v1
    kind: ConfigMap
    metadata:
      name: two
    data:
      two: two
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: three
data:
  three: three

I can execute the following which only gives me documents that were split from the list and does not include the third, non-list document:

yq 'select(.kind == "ConfigMapList") | .items[] | split_doc ' all.yaml 

Is it possible to get the following as a result? It would be fine for my use-case if this also included the original ConfigMapList document as I could filter that out later.

apiVersion: v1
kind: ConfigMap
metadata:
  name: one
data:
  one: one
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: two
data:
  two: two
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: three
data:
  three: three

I understand that I could execute multiple invocations of yq if I store the output on disk but I am hoping that I can do this in a single yq call so I can pass the output directly into another utility.

[mikefarah]

Yep:

yq '(select(.kind == "ConfigMapList") | .items[]) // . | split_doc' file.yaml

Yourcan use the '//` alternative operator (https://mikefarah.gitbook.io/yq/operators/alternative-default-value) to provide an alternative value when we don't find a node with Kind ConfigMapList

[mikefarah]

Actually scratch that, that won't work properly if the ConfigMapList has empty items, use this instead:

yq '(select(.kind == "ConfigMapList") // {"items": [.]}) | .items[] | split_doc' 

Idea behind this is to always have an items array, then proceed with the pipeline