elastic-agent FIM returning index errors

[cukal]

Version

2.4.30

Installation Method

Security Onion ISO image

Description

installation

Installation Type

Standalone

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

5

RAM

48

Storage for /

300

Storage for /nsm

6TB

Network Traffic Collection

tap

Network Traffic Speeds

1Gbps to 10Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

Yes, there are additional clues in /opt/so/log/ (please provide detail below)

Detail

Using Fleet elastic-agent I copied the existing endpoints-initial policy and added File Integrity Monitoring and pushed it to 10 CentOS7 nodes. Nodes are healthy but no FIM events are generated.
/opt/so/log/logstash/logstash.log mentions this error:

"type" => "illegal_argument_exception",
"reason" => "[constant_keyword] field [event.module] only accepts values that are 
equal to the value defined in the mappings [file_integrity], but got [fim]"

Full logline:

{
	: status => 400,
	: action => ["create", {
			: _id => nil,
			: _index => "logs-fim.event-default",
			: routing => nil
		}, {
			"service" => {
				"type" => "file_integrity"
			},
			"@timestamp" => 2023 - 12 - 13T16: 04: 29.608Z,
			"type" => "redis-input",
			"@version" => "1",
			"elastic_agent" => {
				"snapshot" => false,
				"version" => "8.10.4",
				"id" => "0eafbaad-7077-4de8-88d4-dfc4038ac0b5"
			},
			"data_stream" => {
				"namespace" => "default",
				"type" => "logs",
				"dataset" => "fim.event"
			},
			"agent" => {
				"version" => "8.10.4",
				"id" => "0eafbaad-7077-4de8-88d4-dfc4038ac0b5",
				"type" => "auditbeat",
				"ephemeral_id" => "66c8e7bb-bc8e-480c-bd1c-23713f5f0cf2",
				"name" => "servers.redacted.company"
			},
			"tags" => ["fim-event", "elastic-agent", "input-so.redacted.company", "beats_input_raw_event"],
			"ecs" => {
				"version" => "8.0.0"
			},
			"host" => {
				"mac" => ["00-50-56-81-CE-2D"],
				"hostname" => "servers.redacted.company",
				"id" => "e5c6de3ead5c4e5a806ff764ab1b6c61",
				"ip" => ["172.23.71.100"],
				"name" => "servers.redacted.company",
				"architecture" => "x86_64",
				"os" => {
					"version" => "7 (Core)",
					"kernel" => "3.10.0-1062.4.3.el7.x86_64",
					"type" => "linux",
					"name" => "CentOS Linux",
					"family" => "redhat",
					"codename" => "Core",
					"platform" => "centos"
				},
				"containerized" => false
			},
			"file" => {
				"group" => "root",
				"hash" => {
					"sha1" => "da39a3ee5e6b4b0d3255bfef95601890afd80709"
				},
				"type" => "file",
				"inode" => "67882012",
				"uid" => "0",
				"owner" => "root",
				"ctime" => "2023-12-13T16:04:29.607Z",
				"gid" => "0",
				"mode" => "0644",
				"path" => "/usr/bin/test1",
				"mtime" => "2023-12-13T16:04:29.607Z",
				"size" => 0
			},
			"metadata" => {
				"raw_index" => "logs-fim.event-default",
				"version" => "8.10.4",
				"input_id" => "audit/file_integrity-fim-46538f9e-9fe9-45f7-bdc7-c055065000a8",
				"type" => "_doc",
				"stream_id" => "audit/file_integrity-fim.event-46538f9e-9fe9-45f7-bdc7-c055065000a8",
				"input" => {
					"beats" => {
						"host" => {
							"ip" => "172.23.71.100"
						}
					}
				},
				"beat" => "auditbeat"
			},
			"event" => {
				"action" => ["attributes_modified"],
				"type" => ["change"],
				"module" => "file_integrity",
				"dataset" => "fim.event",
				"kind" => "event",
				"category" => ["file"]
			}
		}
	],
	: response => {
		"create" => {
			"_index" => ".ds-logs-fim.event-default-2023.12.13-000001",
			"_id" => "osnsY4wBi8CElh4VMv2A",
			"status" => 400,
			"error" => {
				"type" => "document_parsing_exception",
				"reason" => "[1:1696] failed to parse field [event.module] of type [constant_keyword] in document with id 'osnsY4wBi8CElh4VMv2A'. Preview of field's value: 'fim'",
				"caused_by" => {
					"type" => "illegal_argument_exception",
					"reason" => "[constant_keyword] field [event.module] only accepts values that are equal to the value defined in the mappings [file_integrity], but got [fim]"
				}
			}
		}
	}
}

Guidelines

• I have read the above statement and can confirm my post is relevant to Security Onion 2.4.

[cukal]

Vanilla install. Add FIM integration to elastic agent.

Agent picks up file event:

{
  "log.level": "debug",
  "@timestamp": "2023-12-14T12:17:48.632+0100",
  "message": "Received fsnotify event",
  "component": {
    "binary": "auditbeat",
    "dataset": "elastic_agent.auditbeat",
    "id": "audit/file_integrity-default",
    "type": "audit/file_integrity"
  },
  "log": {
    "source": "audit/file_integrity-default"
  },
  "event_flags": "REMOVE",
  "service.name": "auditbeat",
  "file_path": "/usr/bin/test1",
  "ecs.version": "1.6.0",
  "log.logger": "file_integrity",
  "log.origin": {
    "file.line": 143,
    "file.name": "file_integrity/eventreader_fsnotify.go"
  }
}

But SO can't handle it:

: response => {
		"create" => {
			"_index" => ".ds-logs-fim.event-default-2023.12.14-000001",
			"_id" => "V1EMaIwBqTbtxYmhGOBV",
			"status" => 400,
			"error" => {
				"type" => "document_parsing_exception",
				"reason" => "[1:1446] failed to parse field [event.module] of type [constant_keyword] in document with id 'V1EMaIwBqTbtxYmhGOBV'. Preview of field's value: 'fim'",
				"caused_by" => {
					"type" => "illegal_argument_exception",
					"reason" => "[constant_keyword] field [event.module] only accepts values that are equal to the value defined in the mappings [file_integrity], but got [fim]"
				}
			}
		}
	}

Guess it's broken.

[keis3cker]

There's an open issue for FIM.
Its included in the next milestone to be fixed.

[cukal]

@keis3cker Thanks for your reply. Is there a workaround or would you perhaps have a link to the reported issue?

[dougburks]

Here's the issue for FIM:
#11847