detect: investigation on single-pkt flows inspection (5180) - v4#14752
detect: investigation on single-pkt flows inspection (5180) - v4#14752jufajardini wants to merge 4 commits intoOISF:mainfrom
Conversation
During initialization, the engine reports how many rules were loaded, as well as which types. Pkt-only or stream-pkt rules would cause a "hole" in such stats, as they're not counted.
Commit 497394e removed inspection of app-proto txs for packets without an established TCP connection. But this meant that the first packet seen in a session pick mid-stream could go without inspection (previous bug 5510 seemed to point towards this behavior, too). If a flow has more packets, the stream will be inspected as part of the upcoming packets and this would go unnoticed. In a single-packet flow, however, the inspection for the packed would be skipped. Although this might not affect alerts -- as they could be processed as part of the flow timeout logic, the actual traffic could be evaded in IPS, in case of a drop rule. From the above, the most visible scenario is when there is only one packet on the flow, as then the engine doesn't have "more time" to pick-up real-packets to inspect for that given flow. But certain tests show that this can also happen for more than one packet scenarios: there will be one less drop event, or traffic from a packet that should have been already dropped will be logged. This led to the possibility of a real packet not being blocked, in IPS, or matched against rules, as the corresponding portion of the stream was only inspected later, as part of the stream/flow-timeout logic. To ensure that we correctly flag the first packet seen for a given mid-stream session, we must check for the session state *after* we have dealt with TCP flags and state. Related to Bug OISF#5510 As part of Bug OISF#5180
Since we're doing this further down in the same function, in a branch that will also cover this case. Related to Bug OISF#5180
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #14752 +/- ##
=======================================
Coverage 82.15% 82.15%
=======================================
Files 1003 1003
Lines 263643 263657 +14
=======================================
+ Hits 216586 216602 +16
+ Misses 47057 47055 -2
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
|
Information: QA ran without warnings. Pipeline = 29442 |
| cnt_packet++; | ||
| } else if (s->type == SIG_TYPE_PKT_STREAM) { | ||
| SCLogDebug("Signature %" PRIu32 " is considered \"Packet-stream inspecting\"", s->id); | ||
| cnt_packet_stream++; |
There was a problem hiding this comment.
Should we add a final else DEBUG_VALIDATE_BUG_ON("rule not in any category") ?
There was a problem hiding this comment.
I'd have to understand better which rule types we want to have listed here, as I noticed that not all of them were listed -- so I just added two that seemed reasonable to have listed. If we add them all, then I agree with you.
|
|
||
| /* if ssn was set in this run (e.g. midstream cases), reflect TCP state on the packet */ | ||
| if (ssn->state >= TCP_ESTABLISHED) { | ||
| p->flags |= PKT_STREAM_EST; |
There was a problem hiding this comment.
Should we add a check that this is midstream ? (debug validation or regular check)
There was a problem hiding this comment.
This already existed regardless of being a midstream check, so I'm not sure that's the only case this could happen... I'd have to study more to be sure.
|
Looks mostly cool with the super-long commit message for the 3-line move :-) |
Link to ticket: https://redmine.openinfosecfoundation.org/issues/
https://redmine.openinfosecfoundation.org/issues/5180
Previous PR: #14747
Describe changes:
To consider: what to do about what was mentioned on #14747 description? (should something be done?)
Provide values to any of the below to override the defaults.
SV_BRANCH=OISF/suricata-verify#2901