RITA number_format_exception "unable to convert to integer"

[ng1991]

Hi All,

I seem to be facing a similar issue to this which the issue created off the back of it is set to resolved (#8137).

I recently got RITA inputs working and in the manager GUI I can see the information for DNS and connections but nothing for beacons.

After doing some investigation I found the following error in logstash:

[2022-12-28T11:55:48,540][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"so-rita", :routing=>nil, :pipeline=>"rita.beacon"}, {"message"=>"0.419,10.136.2.76,185.201.0.18,309,55167,17046740,0.463,0.409,0.228,0.572,0", "ecs"=>{"version"=>"8.0.0"}, "@version"=>"1", "host"=>{"name"=>"HOSTANME"}, "module"=>"rita", "metadata"=>{"version"=>"8.4.3", "beat"=>"filebeat", "pipeline"=>"rita.beacon", "type"=>"_doc", "ip_address"=>"10.0.125.6", "raw_index"=>"so-rita"}, "dataset"=>"beacon", "category"=>"network", "type"=>"redis-input", "tags"=>["beats_input_codec_plain_applied"], "agent"=>{"version"=>"8.4.3", "ephemeral_id"=>"bc67bf6e-aa51-4b5b-a6ae-c4b27f953705", "id"=>"9c2f040d-8eab-402f-9248-d11eabd36447", "name"=>"HOSTNAME", "type"=>"filebeat"}, "log"=>{"file"=>{"path"=>"/nsm/rita/beacons.csv"}, "offset"=>274361}, "@timestamp"=>2022-12-28T11:55:46.433Z}], :response=>{"index"=>{"_index"=>"so-rita", "_id"=>nil, "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"unable to convert [0.463] to integer", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"For input string: \"0.463\""}}}}}
The error seems to be with converting to integer unable to convert [0.463] to integer I have tried to amend the ingest to float but also noticed the fields the csv is mapping to seems to be for Beacons FQDN not Beacons which is actually what is created and the fields should be more.

"csv": { "field": "message", "target_fields": [ "beacon.score", "source.ip", "destination.ip", "network.connections", "network.average_bytes", "beacon.total_bytes", "beacon.ts.score", "beacon.ds.score", "beacon.dur.score", "beacon.hist.score", "beacon.top.interval", ] }

Rather than

{ "csv": { "field": "message", "target_fields": [ "beacon.score", "source.ip", "destination.ip", "network.connections", "network.average_bytes", "beacon.interval.range", "beacon.size.range", "beacon.interval.top", "beacon.size.top", "beacon.interval.top_count", "beacon.size.top_count", "beacon.interval.skew", "beacon.size.skew", "beacon.interval.dispersion", "beacon.size.dispersion", "network.bytes" ] } }

Anyone else facing these issues or have any ideas how I can get the beacons data indexing.

This is twhat I see in the GUI by dataset

SO Version 2.3.180
On prem setup with 2 search nodes

[weslambert]

This was converted to an issue here:

#9503

[ng1991]

Hi @weslambert

Thanks again for picking this up I had a look at the issue you created and can see a fix has been implemented how do I actually apply the fix, do I need to apply a patch?

[ng1991]

It is working now I amended the output as per your changed file in the default directory and now see the beacon data

Thanks again for your help

[weslambert]

Good to hear 👍 !