Merge pull request #690 from crowdsecurity/appsec-alerts_and_scenarios

buixor · web-flow · commit c9a5f5d294d5 · 2024-12-03T15:36:45.000+01:00
[AppSec] Document generated events
diff --git a/crowdsec-docs/docs/appsec/alerts_and_scenarios.md b/crowdsec-docs/docs/appsec/alerts_and_scenarios.md
@@ -0,0 +1,155 @@
+---
+id: alerts_and_scenarios
+title: AppSec Alerts & Scenarios
+sidebar_position: 5
+---
+
+## Generated Events Layout
+
+HTTP requests that trigger _In-Band_ or _Out-Of-Band_ AppSec/WAF rules generate events. These events can trigger scenarios that react by banning or alerting when rules are matched.
+
+The [`crowdsecurity/appsec-logs` parser](https://app.crowdsec.net/hub/author/crowdsecurity/configurations/appsec-logs) is designed as a general-purpose tool to convert events into a format that is easier to process with scenarios.
+
+
+The generated event looks like this:
+
+ - `evt.Meta.service` is set to `appsec`
+ - `evt.Meta.log_type`:
+    - `appsec-block` for blocked requests (_In-Band_ rule matched, for example)
+    - `appsec-info` for requests that triggered _Out-Of-Band_ rule (not blocked)
+ - `evt.Meta.source_ip` is set to the source (client) IP
+ - `evt.Meta.target_host` is set to the FQDN if present (`Host` header in the HTTP request)
+ - `evt.Meta.target_uri` is set to the complete URI of the HTTP request
+ - `evt.Meta.rule_name` is set to the name of the triggered rule
+ - `evt.Meta.remediation_cmpt_ip` is set to the IP of the Remediation Component (Bouncer) that sent the HTTP request.
+
+:::info
+The [`crowdsecurity/appsec-logs` parser](https://app.crowdsec.net/hub/author/crowdsecurity/configurations/appsec-logs) is already part of the generic AppSec/WAF collections and doesn't have to be manually installed.
+:::
+
+
+## Creating Scenario Based on AppSec/WAF Events
+
+### Triggering on _In-Band_ Rules
+
+A simple yexample is the [`crowdsecurity/appsec-vpatch` scenario](https://app.crowdsec.net/hub/author/crowdsecurity/configurations/appsec-vpatch) that will ban IPs triggering two distinct _In-Band_ rules:
+
+```yaml title="/etc/crowdsec/scenarios/appsec-vpatch.yaml"
+type: leaky
+name: crowdsecurity/appsec-vpatch
+filter: "evt.Meta.log_type == 'appsec-block'"
+distinct: evt.Meta.rule_name
+leakspeed: "60s"
+capacity: 1
+groupby: evt.Meta.source_ip
+...
+```
+
+:::info
+The [`crowdsecurity/appsec-vpatch` scenario](https://app.crowdsec.net/hub/author/crowdsecurity/configurations/appsec-vpatch) is already part of the generic AppSec/WAF collections, and doesn't have to be manually installed.
+:::
+
+### Triggering on Out-Of-Band Rules
+
+Let's try to solve an imaginary scenario:
+
+> We aim to prevent users from enumerating certain URLs (specifically, those that begin with `/foobar/*`) when a particular HTTP header is present (`something: *test*`). However, we want to impose this restriction only on users attempting to access two or more distinct `/foobar/*` URLs while this header is set.
+
+:::info
+Keep in mind that _Out-Of-Band_ rules will  generate an event instead of blocking the HTTP Request.
+:::
+
+#### The AppSec/WAF Rule
+
+This is our AppSec/WAF rule: 
+
+```yaml title="/etc/crowdsec/appsec-rules/foobar-access.yaml"
+name: crowdsecurity/foobar-access
+description: "Detect access to foobar files with the something header set"
+rules:
+  - zones:
+    - URI
+    transform:
+    - lowercase
+    match:
+      type: startsWith
+      value: /foobar/
+  - zones:
+    - HEADERS
+    variables:
+    - something
+    transform:
+    - lowercase
+    match:
+      type: contains
+      value: test
+```
+
+Let ensure it's loaded as an _Out-Of-Band_ rule, first by creating a new appsec-config:
+
+```yaml title="/etc/crowdsec/appsec-configs/appsec-oob.yaml"
+name: crowdsecurity/appsec-oob
+default_remediation: ban
+#Let's add our rule as an out-of-band rule
+outofband_rules:
+ - crowdsecurity/foobar-access
+```
+
+And then make sure this appsec-config is loaded:
+
+```yaml title="/etc/crowdsec/acquis.d/appsec.yaml"
+appsec_configs:
+ - crowdsecurity/appsec-default
+ - crowdsecurity/appsec-oob
+labels:
+  type: appsec
+listen_addr: 127.0.0.1:7422
+source: appsec
+```
+
+#### The Scenario
+
+We can now create a scenario that will trigger when a single IPs triggers this rule on distinct URLs:
+
+```yaml title="/etc/crowdsec/scenarios/foobar-enum.yaml"
+type: leaky
+format: 3.0
+name: crowdsecurity/foobar-enum
+description: "Ban IPs repeateadly triggering out of band rules"
+filter: "evt.Meta.log_type == 'appsec-info' && evt.Meta.rule_name == 'crowdsecurity/foobar-access'"
+distinct: evt.Meta.target_uri
+leakspeed: "60s"
+capacity: 1
+groupby: evt.Meta.source_ip
+blackhole: 1m
+labels:
+  remediation: true
+```
+
+:::info
+The `filter` ensures only _Out-Of-Band_ events generated by our scenario are picked up, while the `capacity: 1` and `distinct: evt.Meta.target_uri` will ensure that the IP has to trigger the rule on at least 2 distinct URLs to trigger the scenario.
+:::
+
+#### Testing
+
+Let's now test our setup:
+
+```bash
+$ curl -I localhost/foobar/1 -H 'something: test'
+HTTP/1.1 404 Not Found
+
+$ curl -I localhost/foobar/2 -H 'something: test'
+HTTP/1.1 404 Not Found
+
+$ curl -I localhost/foobar/3 -H 'something: test'
+HTTP/1.1 403 Forbidden
+```
+
+And CrowdSec logs will show:
+
+```
+INFO[2024-12-02T15:28:16+01:00] Ip ::1 performed 'crowdsecurity/foobar-enum' (2 events over 4.780233613s) at 2024-12-02 14:28:16.858419797 +0000 UTC 
+INFO[2024-12-02T15:28:17+01:00] (test/crowdsec) crowdsecurity/foobar-enum by ip ::1 (/0) : 4h ban on Ip ::1 
+```
+
+As expected, the first two requests were processed without being blocked. The second one triggered the scenario, resulting in the third request being blocked by the bouncer.
diff --git a/crowdsec-docs/docs/appsec/benchmark.md b/crowdsec-docs/docs/appsec/benchmark.md
@@ -33,8 +33,8 @@ The benchmarks cover the following tests:
 Each test has been run with multiple cases:
 
 - Application Security Component enabled but without any rules
-- Application Security Component enabled with 100 vpatch rules (in inband)
-- Application Security Component enabled with all the CRS (in inband)
+- Application Security Component enabled with 100 vpatch rules (in in-band)
+- Application Security Component enabled with all the CRS (in in-band)
 
 On the system, we deployed:
 
diff --git a/crowdsec-docs/docs/appsec/configuration.md b/crowdsec-docs/docs/appsec/configuration.md
@@ -10,7 +10,7 @@ Configuring the AppSec Component usually requires the use of multiple files:
 
  - [AppSec rules](/appsec/rules_syntax.md) allow you to write a signature to detect and/or block malevolent requests. [You can find more information about the syntax here](/appsec/rules_syntax.md)
  - [acquisition configuration](/data_sources/appsec.md) indicates which port is the AppSec Component listening on, and which AppSec configuration it will use.
- - AppSec configuration tells which rules are loaded in inband (blocking) and out-of-band (non-blocking)
+ - AppSec configuration tells which rules are loaded in in-band (blocking) and out-of-band (non-blocking)
   phases. [it as well allows you to tweak the behavior of the component via the powerful expr bindings](/appsec/rules_syntax.md)
 
 
@@ -40,11 +40,11 @@ A supplementary list of rules can be loaded during the out-of-band phase. These
 
 ### `inband_rules`
 
-An optional list of rules to be loaded in inband phase. In band rules are blocking and evaluated before answering the remediation component. Useful for virtual patching, rules with no/low false positives.
+An optional list of rules to be loaded in in-band phase. In band rules are blocking and evaluated before answering the remediation component. Useful for virtual patching, rules with no/low false positives.
 
 ### `default_remediation`
 
-An optional remediation for inband rules, defaults to `ban`. If set to `allow`, remediation component won't block the request (even if it matched rules). Any other value (including `captcha`) is passed as-is back to the remediation component.
+An optional remediation for in-band rules, defaults to `ban`. If set to `allow`, remediation component won't block the request (even if it matched rules). Any other value (including `captcha`) is passed as-is back to the remediation component.
 
 ### `default_pass_action`
 
@@ -84,6 +84,6 @@ See the [dedicated doc](/docs/appsec/hooks.md#on_match)
 
 ### `inband_options` and `outofband_options`
 
-Subset of options that can be applied to the inband/outofband rules:
+Subset of options that can be applied to the in-band/out-of-band rules:
  - `disable_body_inspection` : boolean, allows to disable HTTP body inspection
  - `request_body_in_memory_limit` : a number of byes indicating the maximum body size to be loaded in memory
diff --git a/crowdsec-docs/docs/appsec/installation.md b/crowdsec-docs/docs/appsec/installation.md
@@ -24,7 +24,7 @@ To have a functional AppSec Component, you need:
 Before diving into the practical steps, it's crucial to familiarize yourself with the core configuration aspects of the AppSec Component:
 
 - **acquisition configuration**: Specifies how to acquire the AppSec Component stream of data
-- **AppSec Component configuration**: Tells which rules are loaded in inband (blocking) and out-of-band (non-blocking)
+- **AppSec Component configuration**: Tells which rules are loaded in in-band (blocking) and out-of-band (non-blocking)
   phases, [and allows tweaking the behavior of the component via the powerful expr bindings](/appsec/hooks.md). <!--@sbl we need anchor for the on_whatever and expr helpers -->
 - **rules** allow writing a [signature to detect and/or block malevolent requests](/appsec/rules_syntax.md).
 
diff --git a/crowdsec-docs/sidebars.js b/crowdsec-docs/sidebars.js
@@ -1,3 +1,4 @@
+
 module.exports = {
     // By default, Docusaurus generates a sidebar from the docs folder structure
     //tutorialSidebar: [{type: 'autogenerated', dirName: '.'}],
@@ -760,6 +761,7 @@ module.exports = {
                 { type: "doc", id: "appsec/create_rules" },
             ],
         },
+        { type: "doc", id: "appsec/alerts_and_scenarios", label: "Alerts & Scenarios" },
         { type: "doc", id: "appsec/installation" },
         { type: "doc", id: "appsec/protocol", label: "Communication Protocol" },
         { type: "doc", id: "appsec/benchmark", label: "Benchmark" },
