Field is of the wrong type in Elastic

[soad20000]

Version

2.4.30

Installation Method

Security Onion ISO image

Description

Configuration

Installation Type

Standalone

Location

On-prem with Internet access

Hardware Specs

Exceeds minimum requirements

Status

Salt Status

No errors when calling high state

Logs

Detail

Hello,

I am having an issue with fields not being mapped correctly. The issue began this morning, or at least that's when I noticed it. Yesterday I added a new log path to the system integration. I wanted to start ingesting logs from /var/log/firewalld so I added that path to the integration, deployed it to my agents, and made a custom pipeline that uses Grok to parse the log entries. The new fields that were created are named differently, and have a "firewalld" prefix so that they don't get confused with other fields. That worked out fine, and I managed to see the data that I wanted and it was being mapped correctly.

This morning however I noticed that now the "default" fields of source.ip, destination.ip, source.port, and destination.port are throwing the error in the images above. I checked their mapping and I noticed that the mapping for the syslog index maps ip and port to keywords, while the fortigate index maps source.ip as IP and source.port as long. Does it have something to do with the change I implemented yesterday? How should I go about fixing this?

[dougburks]

It does seem related to the change you made. I'd try reverting that change and then see what the fields look like on the following day.

[soad20000]

There was a mapping conflict between different indices. A few of the dashboards were using documents from two different indices, the fortigate index and the syslog index. I checked which fields had mapping conflicts in Advanced Settings -> Data Views, and created a runtime field for each one of those fields/indices.

PUT index_name/_mapping { "runtime": { "source.ip": { "type": "ip" }, "source.port": { "type": "long" }, } }