No alerts from elastic fleet logs

[danielbidwell]

Version

2.4.111

Installation Method

Cloud image (Amazon, Azure, Google)

Description

configuration

Installation Type

Standalone

Location

cloud

Hardware Specs

Meets minimum requirements

CPU

8

RAM

12gb

Storage for /

252G

Storage for /nsm

252g

Network Traffic Collection

tap

Network Traffic Speeds

Less than 1Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

I am trying to aggregate the logs of 8 other servers via the elasticagent. All 8 servers show up in the elastic fleet list as healthy with ongoing recent contact. Under the dashboards I see that there are log entries that have been imported from the servers, but nothing shows up from these log entries in the alerts report like they used to.

I have followed the help suggestions in #12274
to ZEEK to SURICATA and to change the default policy for each node to so-grid-nodes_general.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[cm-ops]

If you run elastic-agent inspect do you see the suricata alerts integration?

[danielbidwell]

I see json stanzas as follows but no log files: - data_stream: namespace: so id: logfile-logs-d0652a8c-866f-468b-82e4-1f1d8f913f7d meta: package: name: log version: 2.3.1 name: suricata-logs package_policy_id: d0652a8c-866f-468b-82e4-1f1d8f913f7d revision: 1 streams: - data_stream: dataset: suricata id: logfile-log.logs-d0652a8c-866f-468b-82e4-1f1d8f913f7d ignore_older: 72h paths: - /nsm/suricata/eve*.json pipeline: suricata.common processors: - add_fields: fields: category: network module: suricata target: event type: logfile use_output: so-manager_logstash and - data_stream: namespace: so id: logfile-logs-e600408e-8b2a-48d8-bed3-153f45bdb161 meta: package: name: log version: 2.3.1 name: import-suricata-logs package_policy_id: e600408e-8b2a-48d8-bed3-153f45bdb161 revision: 1 streams: - data_stream: dataset: import id: logfile-log.logs-e600408e-8b2a-48d8-bed3-153f45bdb161 ignore_older: 72h paths: - /nsm/import/*/suricata/eve*.json pipeline: suricata.common processors: - add_fields: fields: category: network imported: true module: suricata target: event - dissect: field: log.file.path target_prefix: "" tokenizer: /nsm/import/%{import.id}/suricata/%{import.file} type: logfile use_output: so-manager_logstash - data_stream: namespace: so id: logfile-logs-e600408e-8b2a-48d8-bed3-153f45bdb161 meta: package: name: log version: 2.3.1 name: import-suricata-logs package_policy_id: e600408e-8b2a-48d8-bed3-153f45bdb161 revision: 1 streams: - data_stream: dataset: import id: logfile-log.logs-e600408e-8b2a-48d8-bed3-153f45bdb161 ignore_older: 72h paths: - /nsm/import/*/suricata/eve*.json pipeline: suricata.common processors: - add_fields: fields: category: network imported: true module: suricata target: event - dissect: field: log.file.path target_prefix: "" tokenizer: /nsm/import/%{import.id}/suricata/%{import.file} type: logfile use_output: so-manager_logstash and - data_stream: namespace: so id: logfile-logs-e600408e-8b2a-48d8-bed3-153f45bdb161 meta: package: name: log version: 2.3.1 name: import-suricata-logs package_policy_id: e600408e-8b2a-48d8-bed3-153f45bdb161 revision: 1 streams: - data_stream: dataset: import id: logfile-log.logs-e600408e-8b2a-48d8-bed3-153f45bdb161 ignore_older: 72h paths: - /nsm/import/*/suricata/eve*.json pipeline: suricata.common processors: - add_fields: fields: category: network imported: true module: suricata target: event - dissect: field: log.file.path target_prefix: "" tokenizer: /nsm/import/%{import.id}/suricata/%{import.file} type: logfile use_output: so-manager_logstash and - data_stream: namespace: so id: logfile-logs-d0652a8c-866f-468b-82e4-1f1d8f913f7d meta: package: name: log version: 2.3.1 name: suricata-logs package_policy_id: d0652a8c-866f-468b-82e4-1f1d8f913f7d revision: 1 streams: and - data_stream: namespace: so id: logfile-logs-e600408e-8b2a-48d8-bed3-153f45bdb161 meta: package: name: log version: 2.3.1 name: import-suricata-logs package_policy_id: e600408e-8b2a-48d8-bed3-153f45bdb161 revision: 1 streams: - data_stream: dataset: import id: logfile-log.logs-e600408e-8b2a-48d8-bed3-153f45bdb161 ignore_older: 72h paths: - /nsm/import/*/suricata/eve*.json pipeline: suricata.common processors: - add_fields: fields: category: network imported: true module: suricata target: event - dissect: field: log.file.path target_prefix: "" tokenizer: /nsm/import/%{import.id}/suricata/%{import.file} type: logfile use_output: so-manager_logstash - data_stream: namespace: so id: logfile-logs-e600408e-8b2a-48d8-bed3-153f45bdb161 meta: package: name: log version: 2.3.1 name: import-suricata-logs package_policy_id: e600408e-8b2a-48d8-bed3-153f45bdb161 revision: 1 streams:

…

On Mon, 2025-01-27 at 07:58 -0800, Chris Morgret wrote: If you run elastic-agent inspect do you see the suricata alerts integration? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: <Security-Onion-Solutions/securityonion/repo- ***@***.***>

-- Daniel R. Bidwell | ***@***.*** TikvahShalomConsulting,LLC If two always agree, one of them is unnecessary. Karma is getting what you deserve, mercy is not getting what you deserve grace is getting what you do not deserve. In theory, theory and practice are the same. In practice, they are not.

[cm-ops]

Let me know what this shows - sudo so-index-list

[danielbidwell]

sudo so-index-list health status index uuid pri rep docs.count docs.deleted store.size pri.store.size dataset.size green open .ds-.fleet-actions-results-2025.01.22-000001 nkiQ3orwSuaIN-xpDjYGXA 1 0 8 0 36.9kb 36.9kb 36.9kb green open .ds-logs-detections.alerts-so-2024.12.26-000001 o_6zfR01RXSRSwPQKwsNbA 1 0 1 0 65.9kb 65.9kb 65.9kb green open .ds-logs-detections.alerts-so-2024.12.27-000002 iKsM4oGIRker_NinnGuubA 1 0 1 0 65.9kb 65.9kb 65.9kb green open .ds-logs-detections.alerts-so-2025.01.02-000003 _teUKn1WQT-SqSbWq4fS_Q 1 0 1 0 64.3kb 64.3kb 64.3kb green open .ds-logs-detections.alerts-so-2025.01.14-000004 GtY3fZFKRSefKuqxCSyKZg 1 0 0 0 249b 249b 249b green open .ds-logs-elastic_agent-default-2025.01.18-000001 yv4B84GzSymjW7xqP8zd3A 1 0 10462 0 13.8mb 13.8mb 13.8mb green open .ds-logs-elastic_agent.endpoint_security-default-2025.01.18-000001 tqbf4sWjRD-STwBVHHgEeQ 1 0 500466 0 163.6mb 163.6mb 163.6mb green open .ds-logs-elastic_agent.filebeat-default-2025.01.18-000001 KZ2TkNxDQeyytdo1X_LqGw 1 0 129749 0 136.2mb 136.2mb 136.2mb green open .ds-logs-elastic_agent.fleet_server-default-2025.01.18-000001 trPgNUMPQ_27ryMmSJiHHw 1 0 353603 0 140.4mb 140.4mb 140.4mb green open .ds-logs-elastic_agent.osquerybeat-default-2025.01.22-000001 7jcoY-CSTW-QUhfQqZEVXw 1 0 146 0 465.4kb 465.4kb 465.4kb green open .ds-logs-elasticsearch.server-default-2025.01.18-000001 9NqaVmkRTnGC0ApB7Xkx6A 1 0 7391 0 7.9mb 7.9mb 7.9mb green open .ds-logs-endpoint.events.file-default-2025.01.14-000002 GRP7SOrWTN-UoAQbfSafsQ 1 0 186004479 0 50gb 50gb 50gb green open .ds-logs-endpoint.events.file-default-2025.01.19-000004 Z8y_4XuiS1eMFTMz3V5WOw 1 0 118627377 0 31.5gb 31.5gb 31.5gb green open .ds-logs-kratos-so-2024.12.23-000001 0ETcqoL6SsKi4k__jeI7jg 1 0 299762 0 767.2mb 767.2mb 767.2mb green open .ds-logs-kratos-so-2025.01.22-000002 QQpKRqFnQ2qhESHSdbin1Q 1 0 15739 0 40.9mb 40.9mb 40.9mb green open .ds-logs-soc-so-2024.12.23-000001 5zqv6iRdQTeELxs6vRgx_Q 1 0 8926898 0 5.4gb 5.4gb 5.4gb green open .ds-logs-soc-so-2025.01.22-000002 nxJvhvwJRG-wnc6K_nfHAw 1 0 15171903 0 6.6gb 6.6gb 6.6gb green open .ds-logs-strelka-so-2025.01.22-000001 x3mwyjxDRdWg823Wf-qPzQ 1 0 7 0 265.5kb 265.5kb 265.5kb green open .ds-logs-suricata.alerts-so-2025.01.22-000001 hNukzTMMRT2PJMRVGV4yFg 1 0 133 0 1mb 1mb 1mb green open .ds-logs-suricata.alerts-so-2025.01.23-000002 jfJPbFaPT02ND5i81doQDQ 1 0 0 0 249b 249b 249b green open .ds-logs-syslog-so-2025.01.22-000001 OQOwdcgvS1mRT6nVYcefwQ 1 0 241470 0 105.4mb 105.4mb 105.4mb green open .ds-logs-system.auth-default-2025.01.18-000001 mnoOLX21S-KQPJFzU9pp-g 1 0 146547 0 85.1mb 85.1mb 85.1mb green open .ds-logs-system.syslog-default-2025.01.18-000001 K9YAWvmlTpyy8QbYL1jf_g 1 0 4877220 0 2.5gb 2.5gb 2.5gb green open .ds-logs-zeek-so-2024.12.23-000001 2lRbIeBdTgWK4vPTwUqU2A 2 0 26259 0 33.2mb 33.2mb 33.2mb green open .ds-logs-zeek-so-2025.01.22-000002 N5TCznOPR2OgT6KLq3Vrkg 2 0 2819 0 4.4mb 4.4mb 4.4mb green open .ds-metrics-endpoint.metadata-default-2025.01.09-000001 H45MDqElRoeHCrbdDORQ1Q 1 0 356 0 564.1kb 564.1kb 564.1kb green open .ds-metrics-endpoint.metrics-default-2025.01.09-000001 N6lUUqoQTwOElTm46qds2Q 1 0 356 0 5mb 5mb 5mb green open .ds-metrics-endpoint.policy-default-2025.01.09-000001 8fAhGwCiT8mQWuvlyqBgvw 1 0 271 0 1.5mb 1.5mb 1.5mb green open .internal.alerts-default.alerts-default-000001 Bvb7_jl8T3OrRi-5jES2Pg 1 0 0 0 249b 249b 249b green open .internal.alerts-ml.anomaly-detection-health.alerts-default-000001 aDi2g0zARl6-gKRmR6973A 1 0 0 0 249b 249b 249b green open .internal.alerts-ml.anomaly-detection.alerts-default-000001 7al2DGNWRISqGVIDR3W0yA 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.apm.alerts-default-000001 gfqwrDRRRWSGBF7Slvu7yA 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.logs.alerts-default-000001 8SPCgntTTHmBlGt0x9sq2A 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.metrics.alerts-default-000001 Ot0qJUL2Q5yfMXPfNSQrIA 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.slo.alerts-default-000001 fkj5i2xwSge24AdBLBPz7g 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.threshold.alerts-default-000001 Z8w8wa1UR12wzfJ1jYtTsg 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.uptime.alerts-default-000001 CjJBdjsXTIuckdo_z0hL-g 1 0 0 0 249b 249b 249b green open .internal.alerts-security.alerts-default-000001 Uc4Pn7DSTXiCVHFILiPQFg 1 0 0 0 249b 249b 249b green open .internal.alerts-stack.alerts-default-000001 3edKCQngS-2MYz25Y4H5Ag 1 0 0 0 249b 249b 249b green open .internal.alerts-transform.health.alerts-default-000001 vvzS1yanQo6g5IcTg_uY3Q 1 0 0 0 249b 249b 249b green open .kibana-observability-ai-assistant-conversations-000001 32QKOdMDQDGTFCfLb3HHvg 1 0 0 0 249b 249b 249b green open .logs-osquery_manager.action.responses-default pQVCYmiRR2mnRAnZUu0_rA 1 0 0 0 249b 249b 249b green open .logs-osquery_manager.actions-default hcJ-uK6AQLSaT8RBx7r63w 1 0 0 0 249b 249b 249b green open elastalert x3SpZ0bSQn2bdb2qr1xk0A 1 0 3 0 185.5kb 185.5kb 185.5kb green open elastalert_error b5cU8ukpTTO8ZjlWCPUlkw 1 0 483280 0 347.2mb 347.2mb 347.2mb green open elastalert_past 4bc7Me5kSbCOADLBPzjhPw 1 0 0 0 249b 249b 249b green open elastalert_silence uaT3_RJlRBCENE8wZhKYQQ 1 0 0 0 249b 249b 249b green open elastalert_status _Qfh6qPsQ7idqfkxAiIH2A 1 0 1089467 0 181.7mb 181.7mb 181.7mb green open logs-ti_abusech_latest.dest_malware-2 K2rEKgNnToqei0zEJSAgKA 1 0 0 0 249b 249b 249b green open logs-ti_abusech_latest.dest_malwarebazaar-2 QfBcZRwMRRefQbkrvBk83w 1 0 0 0 249b 249b 249b green open logs-ti_abusech_latest.dest_threatfox-2 mtGK766ySGO06ixOAGmlDQ 1 0 0 0 249b 249b 249b green open logs-ti_abusech_latest.dest_url-2 WVeDvtaBQNGzMafd-I6FRg 1 0 0 0 249b 249b 249b green open logs-ti_anomali_latest.threatstream-2 Ui52fRzJRiq_Q897HmKqaw 1 0 0 0 249b 249b 249b green open logs-ti_cybersixgill_latest.dest_threat-2 pHanjIClQIijoAshLlfT3Q 1 0 0 0 249b 249b 249b green open logs-ti_misp_latest.dest_threat_attributes-2 2If9A9ZlQeKFNXzDAwjLxw 1 0 0 0 249b 249b 249b green open logs-ti_otx_latest.dest_pulses_subscribed-1 lkAv9WgkQa2qHaEkYP8cGQ 1 0 0 0 249b 249b 249b green open logs-ti_recordedfuture_latest.threat-2 kt2cfwfgR2GC4_cWg0AvAA 1 0 0 0 249b 249b 249b green open logs-ti_threatq_latest.dest_threat-2 TzwCHOctSAqPk0Iw-pHsWw 1 0 0 0 249b 249b 249b green open metrics-endpoint.metadata_current_default dnfI3PhiTvWNbppg1uIXTA 1 0 8 0 185kb 185kb 185kb green open so-detection JQfHs50nQy-Ps-iV3P4Lsg 1 0 61508 13707 44.9mb 44.9mb 44.9mb green open so-detectionhistory cP56NExLT_6G-jOdr3atMw 1 0 458120 0 514.1mb 514.1mb 514.1mb

…

On Wed, 2025-01-29 at 04:20 -0800, Chris Morgret wrote: Let me know what this shows - sudo so-index-list — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: <Security-Onion-Solutions/securityonion/repo- ***@***.***>

-- Daniel R. Bidwell | ***@***.*** TikvahShalomConsulting,LLC If two always agree, one of them is unnecessary. Karma is getting what you deserve, mercy is not getting what you deserve grace is getting what you do not deserve. In theory, theory and practice are the same. In practice, they are not.

[cm-ops]

Thanks, based on the above, are you ingesting network data or just syslog?

[danielbidwell]

We just want to ingest syslog data.

…

On Thu, 2025-01-30 at 08:20 -0800, Chris Morgret wrote: Thanks, based on the above, are you ingesting network data or just syslog? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: <Security-Onion-Solutions/securityonion/repo- ***@***.***>

-- Daniel R. Bidwell | ***@***.*** TikvahShalomConsulting,LLC If two always agree, one of them is unnecessary. Karma is getting what you deserve, mercy is not getting what you deserve grace is getting what you do not deserve. In theory, theory and practice are the same. In practice, they are not.

[cm-ops]

If you aren't ingesting network traffic via SPAN/TAP there is nothing for Suricata to evaluate against the signatures. You won't see any alerts.

If you have Sigma rules set up to alert on your syslog traffic, you would see an alert for sigma and the rule you set up.