How to parse escaped nginx log line with escaped double quotes using VRL

[prabhatsharma]

This is my source data:

{
  "_timestamp": 1688641704414631,
  "kubernetes_annotations_kubernetes_io_psp": "eks.privileged",
  "kubernetes_container_hash": "registry.k8s.io/ingress-nginx/controller@sha256:4ba73c697770664c1e00e9f968de14e08f606ff961c76e5d7033a4a9c593c629",
  "kubernetes_container_image": "registry.k8s.io/ingress-nginx/controller@sha256:4ba73c697770664c1e00e9f968de14e08f606ff961c76e5d7033a4a9c593c629",
  "kubernetes_container_name": "controller",
  "kubernetes_labels_app_kubernetes_io_component": "controller",
  "kubernetes_labels_app_kubernetes_io_instance": "ingress-nginx",
  "kubernetes_labels_app_kubernetes_io_name": "ingress-nginx",
  "kubernetes_labels_pod_template_hash": "7d97444d5",
  "kubernetes_namespace_name": "ingress-nginx",
  "kubernetes_pod_name": "ingress-nginx-controller-7d97444d5-l9rfd",
  "log": "10.2.81.141 - alice [06/Jul/2023:11:08:24 +0000] \"POST /api/path1/prometheus/api/v1/write HTTP/1.1\" 200 0 \"-\" \"Prometheus/2.15.2+ds\" 468 0.005 [zinc-cp1-zinc-cp-4082] [] 9.9.176.197:4082 0 0.005 200 0ca2f386ce3b27951f36fb1f54af41f5\n",
  "stream": "stdout"
}

log field is standard nginx log line.

How do I parse this using VRL with parse_nginx_log?

[jszwedko]

You should be able to do something like parse_nginx_log(.log, format: "combined").

[prabhatsharma]

I have tried parse_nginx_log(.log, format: "combined") earlier. It fails

Probably because of the escaped double quotes

[dsmith3197]

I was able to successfully parse the example log line in the docs with escaped double quotes. For your case, you'll want to use the ingress_upstreaminfo format (example).

[dsmith3197]

I noticed that ingress_upstreaminfo is not in the public docs. Opened #17893 to fix this.