matching: multi match support

[securitym0nkey]

Make sure these boxes are signed before submitting your Pull Request
-- thank you.

• I have read the contributing guide lines at
  https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Contributing
• I have signed the Open Information Security Foundation
  contribution agreement at
  https://suricata-ids.org/about/contribution-agreement/
• I have updated the user guide (in doc/userguide/) to reflect the
  changes made (if applicable)

This is an implementation of redmine 2509

Describe changes:

This is a resubmit of pr #37. After reworking to fit the 2023 code base.

It adds support to specify multiple conditions to match a rule.

Syntax:
multi:<match_condition1>;<match_condition2>;...<match_conditionN>;

Examples:
Match all rules including the term "nmap" but just from the "emerging-scan.rules" file.
multi:filename:rule/emerging-scan.rules; re:nmap;

Match all rules with a recent cve reference and a perimeter deployment
multi:re:cve-202[23];metadata: deployment perimeter;

[jasonish]

Sorry for taking so long to get to this. One issue I have is not all valid regex's are supported. For example:

re: sid:\s*2100498;\s*rev:7;

works...

but:

multi: re: sid:\s*2100498;\s*rev:7;

appears to work, but actually fails to properly load as it will only take up to the ;, then silently fail to load the remainder. So I think that would need to be addressed with some test cases.

[jasonish]

#331 (comment)

[securitym0nkey]

That's a good point. ; is definitely not a good delimiter. So what could be a better one? Almost every character might be part of a valid regex, right? Though other delimiters might be less likely to be used in someones regex.

My quick idea would be to use a [TAB] as a delimiter. A good regex would use \t instead of a real [TAB]. But i'm not really satisfied with that solution myself. Any thoughts on what would be acceptable.

Certainly its a good idea to throw some warning or error if a child matcher is not a valid one. I think that would happen with with your example.

[jasonish]

That's a good point. ; is definitely not a good delimiter. So what could be a better one? Almost every character might be part of a valid regex, right? Though other delimiters might be less likely to be used in someones regex.

My quick idea would be to use a [TAB] as a delimiter. A good regex would use \t instead of a real [TAB]. But i'm not really satisfied with that solution myself. Any thoughts on what would be acceptable.

Certainly its a good idea to throw some warning or error if a child matcher is not a valid one. I think that would happen with with your example.

Yeah, I'm not sure either. It was not created with more parameters on a line than one in mind... Proper quoting could work, but also a breaking change as we don't currently require quoting now.

[securitym0nkey]

After some thoughts i would suggest to use && [space][andsign][andsign][space] as the delimiter. I can't come up with a situation where this causes a conflict with other matchers except when the regular expression as exactly && in it. I can't think of a situation where one would write such regex. Obviously it needs to be documented as invalid when using the multi matcher. The workaround would be to encode && as \x26\x26 within the regex.

So your example combined with some other matchers would look something like this.

multi: re: sid:\s*2100498;\s*rev:7; && multi:filename:rule/emerging-scan.rules && metadata: deployment perimeter

Why i like this solution:

• no quoting, no breaking change
• && indicates that the multiple matchers are logical AND
• even the edge case where the delimiter is part of the regex can be addressed

If that's an acceptable solution i would go ahead and implement it.