Unexpected "Suricata: Rule Mismatch" involving modified emerging-all.rules file in airgapped environment

[branchnetconsulting]

Version

2.4.170

Installation Method

Security Onion ISO image

Description

other (please provide detail below)

Installation Type

Standalone

Location

airgap

Hardware Specs

Meets minimum requirements

CPU

varies

RAM

varies

Storage for /

varies

Storage for /nsm

varies

Network Traffic Collection

other (please provide detail below)

Network Traffic Speeds

Less than 1Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

I have numerous independent Security Onion 2.4 deployments that are subscribed to the ETPRO rule feed. I also have a common shared set of enable, disable, and modify rules and additional rules that are to be folded into the ETPRO ruleset before delivery to the individual NSM grids. My expectation is that after receiving our revised version of the latest ETPRO ruleset, that each NSM will then additionally apply any uniquely site-specific Detections measures that are defined at each given site.

I am doing this by using the airgap method so that I'm allowed to use my own custom orchestration to deliver our revised emerging-all.rules file to /nsm/rules/suricata/
(https://docs.securityonion.net/en/2.4/airgap.html#rule-updates).

I generate the revised emerging-all.rules file on a non-NSM server where I have a vanilla install of the latest suricata 7.x and suricata-update in place. From that server suricata-update downloads the latest from ETPRO and then applies my shared enable, disable, modify, and local rules like this:

suricata-update --no-test --output /var/lib/suricata/etpro/rules --disable-conf disablesid.bnc --enable-conf enablesid.bnc --modify-conf modifysid.bnc --local /var/lib/suricata/bnc.rules
cat /var/lib/suricata/etpro/rules/suricata.rules > /var/lib/suricata/etpro/emerging-all.rules

From there the resulting emerging-all.rules file is distributed to each NSM manager's /nsm/rules/suricata/ directory.

This appears to be working. I find clear evidence that all my different types of changes are in place when I inspect /etc/suricata/rules/all.rules while shelled into the so-suricata container.

However, I am concerned because the Detections module keeps throwing an "enabledButNotDeployed" complaint about rules that I did indeed disable with suricata-update on my centralized server.

Please help me understand why the Suricata rule integrity check would consider these problematic, when it is complaining about rule numbers that were already disabled at the point so-rule-update ingested them on the NSMs? Also, I don't see other integrity check complaints about my custom modify measures or about my added rules.

This may just be a cosmetic issue, but I would like to confirm I don't need to worry about

• the NSMs eventually reverting my changes somehow
• the Detections module being disrupted from applying site-specific rule changes
• NSM managers being disrupted from propagating changed rules out to forward nodes

If I can actually tweak something so that I don't see "Suricata: Rule Mismatch" at all, that would be ideal. Otherwise I'd like some assurance that my approach is not breaking needed functionality.

Thanks,
Kevin Branch

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[defensivedepth]

Please post a sanitized version of the enabledButNotDeployed log you are seeing.

[branchnetconsulting]

Here is the expanded messages field from the error record

{
  "fields": {
    "deployedButNotEnabled": [],
    "deployedButNotEnabledCount": 0,
    "detectionEngine": "suricata",
    "enabledButNotDeployed": [
      "2019239",
      "2103430",
      "2103267",
      "2009968",
      "2102944",
      "2012710",
      "2025090",
      "2103414",
      "2101384",
      "2801480",
      "2028814",
      "2012044",
      "2000015",
      "2100252",
      "2018377",
      "2016302",
      "2811700",
      "2024057",
      "2062285",
      "2100639"
    ],
    "enabledButNotDeployedCount": 2081,
    "intCheckId": "904eb11b-afef-4d56-8d02-ebf6f1243622",
    "syncId": "30f32024-abe6-4aa7-8f23-15e74f5eaee3"
  },
  "level": "warn",
  "timestamp": "2025-09-02T17:52:52.447569194Z",
  "message": "integrity check failed"
}

and here is the entire record

@timestamp	2025-09-02T17:52:52.828Z
@version	1
agent.ephemeral_id	5580c855-b6ca-44ed-9903-b2df9890327d
agent.id	48cefa7d-c244-45da-a625-69f27c23086a
agent.name	xyz-nsm
agent.type	filebeat
agent.version	8.17.3
container.id	sensoroni-server.log
data_stream.dataset	soc
data_stream.namespace	so
data_stream.type	logs
ecs.version	8.0.0
elastic_agent.id	48cefa7d-c244-45da-a625-69f27c23086a
elastic_agent.snapshot	false
elastic_agent.version	8.17.3
event.action	integrity check failed
event.category	host
event.dataset	soc.server
event.module	soc
host.architecture	x86_64
host.containerized	false
host.hostname	xyz-nsm
host.id	cd5d56b0fc754253aaea5b13961c4514
host.ip	[
  "172.16.241.11",
  "fe80::d28e:79ff:fef0:bff6",
  "10.11.1.169",
  "172.17.1.1",
  "172.17.0.1"
]
host.mac	[
  "00-0B-F7-54-15-C0",
  ...]
host.name	xyz-nsm
host.os.family	redhat
host.os.kernel	5.15.0-311.185.9.el9uek.x86_64
host.os.name	Oracle Linux Server
host.os.platform	ol
host.os.type	linux
host.os.version	9.6
input.type	log
log.file.path	/opt/so/log/soc/sensoroni-server.log
log.level	warn
log.offset	848001954
message	{"fields":{"deployedButNotEnabled":[],"deployedButNotEnabledCount":0,"detectionEngine":"suricata","enabledButNotDeployed":["2019239","2103430","2103267","2009968","2102944","2012710","2025090","2103414","2101384","2801480","2028814","2012044","2000015","2100252","2018377","2016302","2811700","2024057","2062285","2100639"],"enabledButNotDeployedCount":2081,"intCheckId":"904eb11b-afef-4d56-8d02-ebf6f1243622","syncId":"30f32024-abe6-4aa7-8f23-15e74f5eaee3"},"level":"warn","timestamp":"2025-09-02T17:52:52.447569194Z","message":"integrity check failed"}
metadata.beat	filebeat
metadata.input.beats.host.ip	172.17.1.1
metadata.input_id	logfile-logs-a1f296f2-4e16-4a67-ba90-f3a33acb3f19
metadata.pipeline	common
metadata.raw_index	logs-soc-so
metadata.stream_id	logfile-log.logs-a1f296f2-4e16-4a67-ba90-f3a33acb3f19
metadata.type	_doc
metadata.version	8.17.3
soc.fields.deployedButNotEnabled	[]
soc.fields.deployedButNotEnabledCount	0
soc.fields.detectionEngine	suricata
soc.fields.enabledButNotDeployed	[
  "2019239",
  "2103430",
  "2103267",
  "2009968",
  "2102944",
  "2012710",
  "2025090",
  "2103414",
  "2101384",
  "2801480",
  "2028814",
  "2012044",
  "2000015",
  "2100252",
  "2018377",
  "2016302",
  "2811700",
  "2024057",
  "2062285",
  "2100639"
]
soc.fields.enabledButNotDeployedCount	2081
soc.fields.intCheckId	904eb11b-afef-4d56-8d02-ebf6f1243622
soc.fields.syncId	30f32024-abe6-4aa7-8f23-15e74f5eaee3
soc.timestamp	2025-09-02T17:52:52.447569194Z
tags	[
  "so-soc",
  "elastic-agent",
  "input-xyz-nsm",
  "beats_input_codec_plain_applied",
  "server"
]
soc_id	IaSPC5kB2JQYVLCgbvxH
soc_score	11.604553
soc_type	
soc_timestamp	2025-09-02T17:52:52.828Z
soc_source	xyz-nsm:.ds-logs-soc-so-2025.08.15-000001

[branchnetconsulting]

And of all the referenced rule IDs that I check, they correspond to rules that I am myself am intentionally disabling in the ruleset before the NSM downloads it from my own server.

[defensivedepth]

The integrity checker compares the enable|disable status of a rule in Elasticsearch (which is what the UI displays) vs. the actual all.rules file - this is to make sure that what you are seeing in the web interface matches reality.

Can you just doublecheck a couple of those referenced rule ids - are they showing as enabled in the web UI?

[branchnetconsulting]

I do see directly in the so-detection index for the rules that I out-of-band disabled that so_detection.isEnabled is set to true, even though the revised emerging-all.rules file that the NSM downloads from my internal server, definitely has those rules commented out. Also looking up the same disabled rules, I find them commented out in the all.rules file inside the so-suricata container. I would assume that downloading a new emerging-all.rules file (airgap style) would cause the enabled/disabled status for all the rules in that file to be used to update the records in so-detection. Do I need to do something to flush out the suricata rule history in so-detection so there is no confusion about the state of things between before and after I switch from direct downloads of ETPRO from the Internet to internal downloads of a revised copy of ETPRO from my server?