SO 2.4.120 Elastic Fleet Agent Google Workspace pipeline_error field [id] not pre

[chris-ci]

Status Update on Mar 5, 2025

Just circiling back too close the loop on this issue. I had reached out to support and it is an issue within the current version of elastic, It looks like the root issue is actually an Elastic bug, as described here and the next update of SO should fix the problem.

Leaving issue open until verifying the next release of SO resolves this issue.

Original Post

I am attempting to injest data from our google workspace environment and have followed all the steps from the elastic agent doc but am unable to determine why the pipeline is erroring out.

I am looking at the elastic discovery results for one of the records and I also have the save_original turned on wtihin in the pipeline and the json data doesn't look correct as I just see 2 keys etag and kind.

Below is the JSON of the doc in elastic below and any help in getting this to work is greatly appriciated.

{
  "_index": ".ds-logs-google_workspace.login-default-2025.02.18-000001",
  "_id": "REDACT",
  "_version": 1,
  "_score": 0,
  "_source": {
    "input": {
      "type": "httpjson"
    },
    "agent": {
      "name": "FleetServer-REDACT",
      "id": "REDACT",
      "type": "filebeat",
      "ephemeral_id": "REDACT",
      "version": "8.14.3"
    },
    "@timestamp": "2025-02-24T14:19:27.583Z",
    "ecs": {
      "version": "8.11.0"
    },
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "google_workspace.login"
    },
    "elastic_agent": {
      "id": "REDACT",
      "version": "8.14.3",
      "snapshot": false
    },
    "json": {
      "kind": "admin#reports#activities",
      "etag": "\"REDACT\""
    },
    "event": {
      "agent_id_status": "verified",
      "ingested": "2025-02-24T14:19:37Z",
      "original": "{\"etag\":\"\\\"REDACT/REDACT\\\"\",\"kind\":\"admin#reports#activities\"}",
      "created": "2025-02-24T14:19:27.583Z",
      "kind": "pipeline_error",
      "dataset": "google_workspace.login"
    },
    "error": {
      "message": [
        "field [id] not present as part of path [json.id.time]"
      ]
    },
    "tags": [
      "preserve_original_event",
      "forwarded",
      "google_workspace-login"
    ]
  },
  "fields": {
    "elastic_agent.version": [
      "8.14.3"
    ],
    "agent.type": [
      "filebeat"
    ],
    "event.module": [
      "google_workspace"
    ],
    "agent.name": [
      "FleetServer-soc"
    ],
    "json.kind": [
      "admin#reports#activities"
    ],
    "elastic_agent.snapshot": [
      false
    ],
    "event.agent_id_status": [
      "verified"
    ],
    "event.kind": [
      "pipeline_error"
    ],
    "event.original": [
      "{\"etag\":\"\\\"REDACT/REDACT\\\"\",\"kind\":\"admin#reports#activities\"}"
    ],
    "elastic_agent.id": [
      "REDACT"
    ],
    "data_stream.namespace": [
      "default"
    ],
    "json.etag": [
      "\"REDACT/REDACT\""
    ],
    "input.type": [
      "httpjson"
    ],
    "data_stream.type": [
      "logs"
    ],
    "tags": [
      "preserve_original_event",
      "forwarded",
      "google_workspace-login"
    ],
    "event.ingested": [
      "2025-02-24T14:19:37.000Z"
    ],
    "@timestamp": [
      "2025-02-24T14:19:27.583Z"
    ],
    "agent.id": [
      "REDACT"
    ],
    "ecs.version": [
      "8.11.0"
    ],
    "error.message": [
      "field [id] not present as part of path [json.id.time]"
    ],
    "data_stream.dataset": [
      "google_workspace.login"
    ],
    "event.created": [
      "2025-02-24T14:19:27.583Z"
    ],
    "agent.ephemeral_id": [
      "REDACT"
    ],
    "agent.version": [
      "8.14.3"
    ],
    "event.dataset": [
      "google_workspace.login"
    ]
  }
}

[reyesj2]

2.4.130 was just released, which includes an upgrade to ES 8.17.3
#14376

[chris-ci]

I just performed an update to the latest version of SO and verified ES 8.17.3 is within the agent field in the record an am still having the same issue. I have replied back to my support case with SO.