Skip to content

Commit dbf88cd

Browse files
LaurenceJJonesLaurenceJJones
authored
Merge branch 'main' into native-haproxy-redirect-optimization
2 parents 5ba490b + aca2a0e commit dbf88cd

File tree

22 files changed

+2647
-1240
lines changed

22 files changed

+2647
-1240
lines changed

‎crowdsec-docs/docs/appsec/benchmark.md‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ sidebar_position: 80
1515
1616
-->
1717

18+
# Basic Benchmark
19+
1820
The Application Security Component benchmarks have been run on a AWS EC2 Instance `t2.medium` (2vCPU/4GiB RAM).
1921

2022
All the benchmarks have been run with only one `routine` configured for the Application Security Component.
@@ -63,3 +65,111 @@ On the system, we deployed:
6365
6466
![15 concurrent connections / 1000 requests](/img/appsec/bench/big_post_appsec_one_routine_15_1000.png)
6567
-->
68+
69+
# Stress Test
70+
71+
This test was run on a `c5a.4xlarge` EC2 instance (16CPU/32GiB RAM).
72+
73+
Tested versions are:
74+
75+
- Openresty `v1.27.1.2`
76+
- CrowdSec `v1.7.0`
77+
- cs-openresty-bouncer `v1.1.2`
78+
79+
Openresty was configured to not log anything and forward requests to a Go backend that always return 200, in order to improve raw throughput and not be limited by disk access.
80+
81+
Crowdsec WAF was configured with 16 routines to make use of as much CPU as possible.
82+
83+
All tests were simulating 400 concurrent users, making requests as quickly as possible during 1 minute.
84+
85+
Except for the baseline, all values in the tables are shown as a delta from the baseline performance.
86+
87+
## Baseline
88+
89+
This test was run without loading the Openresty bouncer to get a baseline throughput of the system.
90+
91+
### GET Requests
92+
93+
| Metric | Value |
94+
| --------------------- | -------- |
95+
| Average Response Time | 23.55ms |
96+
| Minimum Response Time | 21.24ms |
97+
| Median Response Time | 23.18ms |
98+
| Maximum Response Time | 255.16ms |
99+
| P90 Response Time | 24.72ms |
100+
101+
### 10% POST Requests
102+
103+
| Metric | Value |
104+
| --------------------- | -------- |
105+
| Average Response Time | 25.08ms |
106+
| Minimum Response Time | 21.29ms |
107+
| Median Response Time | 23.95ms |
108+
| Maximum Response Time | 331.08ms |
109+
| P90 Response Time | 30.95ms |
110+
111+
## Virtual Patching Rules
112+
113+
### GET Requests - 10% malicious - InBand
114+
115+
| Metric | Delta |
116+
| --------------------- | -------- |
117+
| Average Response Time | +4.94ms |
118+
| Minimum Response Time | +0.93ms |
119+
| Median Response Time | +3.48ms |
120+
| Maximum Response Time | +6.83ms |
121+
| P90 Response Time | +10.13ms |
122+
123+
### Realistic Traffic - 70% GET - 25% POST - 5% malicious - Inband
124+
125+
| Metric | Delta |
126+
| --------------------- | ------- |
127+
| Average Response Time | +4.03ms |
128+
| Minimum Response Time | +0.71ms |
129+
| Median Response Time | +2.36ms |
130+
| Maximum Response Time | +6.79ms |
131+
| P90 Response Time | +8.07ms |
132+
133+
## CRS
134+
135+
### GET Requests - 10% malicious - InBand
136+
137+
| Metric | Delta |
138+
| --------------------- | -------- |
139+
| Average Response Time | +32.85ms |
140+
| Minimum Response Time | +2.21ms |
141+
| Median Response Time | +27.47ms |
142+
| Maximum Response Time | -64.45ms |
143+
| P90 Response Time | +58.19ms |
144+
145+
### POST Requests - 10% malicious - InBand
146+
147+
| Metric | Delta |
148+
| --------------------- | --------- |
149+
| Average Response Time | +58.49ms |
150+
| Minimum Response Time | +3.18ms |
151+
| Median Response Time | +54.1ms |
152+
| Maximum Response Time | -106.76ms |
153+
| P90 Response Time | +83.01ms |
154+
155+
### Realistic Traffic - 70% GET - 25% POST - 5% malicious - Inband
156+
157+
| Metric | Delta |
158+
| --------------------- | -------- |
159+
| Average Response Time | +32.54ms |
160+
| Minimum Response Time | +1.87ms |
161+
| Median Response Time | +28.36ms |
162+
| Maximum Response Time | -68.34ms |
163+
| P90 Response Time | +53.83ms |
164+
165+
## Virtual Patching Inband + CRS Out-of-band
166+
167+
### Realistic Traffic - 70% GET - 25% POST - 5% malicious
168+
169+
| Metric | Delta |
170+
| --------------------- | --------- |
171+
| Average Response Time | +30.5ms |
172+
| Minimum Response Time | +1.56ms |
173+
| Median Response Time | +26.26ms |
174+
| Maximum Response Time | -101.66ms |
175+
| P90 Response Time | +51.18ms |

‎crowdsec-docs/docs/appsec/create_rules.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
id: create_rules
3-
title: Rules Creation & Testing
3+
title: Creation & Testing
44
sidebar_position: 3
55
---
66

‎crowdsec-docs/docs/appsec/rules_deploy.md‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
---
2+
id: rules_deploy
3+
title: Deployment
4+
sidebar_position: 81
5+
---
6+
7+
# WAF Rules Deployment
8+
9+
This walkthrough assumes you already wrote and validated a custom AppSec (WAF) rule. We will deploy a concrete example so you can mirror the exact commands on your host.
10+
11+
## Example Rule We Will Deploy
12+
13+
The example blocks any `GET` request whose `user_id` query argument contains non-numeric characters. While you iterate locally, keep it in a working directory as `./block-nonnumeric-user-id.yaml`:
14+
15+
```yaml title="./block-nonnumeric-user-id.yaml"
16+
name: custom/block-nonnumeric-user-id
17+
description: Block GET requests with a non-numeric user_id parameter.
18+
rules:
19+
- and:
20+
- zones:
21+
- METHOD
22+
match:
23+
type: equals
24+
value: GET
25+
- zones:
26+
- ARGS
27+
variables:
28+
- user_id
29+
match:
30+
type: regex
31+
value: "[^0-9]"
32+
labels:
33+
type: exploit
34+
service: http
35+
confidence: 2
36+
spoofable: 0
37+
behavior: "http:exploit"
38+
label: "Non numeric user id"
39+
```
40+
41+
Once the rule behaves as expected, the remaining steps package it for CrowdSec, wire it into the acquisition pipeline, and test it end to end.
42+
43+
## Step 1 — Stage the Rule File
44+
45+
CrowdSec loads AppSec rules from `/etc/crowdsec/appsec-rules/`. Copy your YAML rule into that directory (create a `custom/` subfolder to keep things tidy if you manage several rules):
46+
47+
```bash
48+
sudo install -d -m 750 /etc/crowdsec/appsec-rules/custom
49+
sudo install -m 640 ./block-nonnumeric-user-id.yaml \
50+
/etc/crowdsec/appsec-rules/custom/block-nonnumeric-user-id.yaml
51+
```
52+
53+
Make sure the `name` inside the rule file matches the file name convention you plan to reference (in our example `custom/block-nonnumeric-user-id`).
54+
55+
:::tip
56+
If you run CrowdSec in a container, copy the file into the volume that is mounted at `/etc/crowdsec/appsec-rules/` inside the container.
57+
:::
58+
59+
## Step 2 — Create an AppSec Configuration
60+
61+
An AppSec configuration lists which rules to load and how to handle matches. Create a new file under `/etc/crowdsec/appsec-configs/` that targets your custom rule:
62+
63+
```yaml title="/etc/crowdsec/appsec-configs/custom-block-nonnumeric-user-id.yaml"
64+
name: custom/block-nonnumeric-user-id
65+
default_remediation: ban
66+
inband_rules:
67+
- custom/block-nonnumeric-user-id
68+
# Add outofband_rules or hooks here if needed
69+
```
70+
71+
Key points:
72+
- `name` is how you will reference this configuration from the acquisition file and in logs.
73+
- `inband_rules` (and/or `outofband_rules`) accept glob patterns, so you can load multiple rules with a single entry such as `custom/block-*`.
74+
- During the reload step CrowdSec validates the syntax; if anything is off, the reload fails and the service logs the parsing error.
75+
76+
## Step 3 — Reference the Configuration in the Acquisition File
77+
78+
The AppSec acquisition file (`/etc/crowdsec/acquis.d/appsec.yaml`) controls which configurations are active for the WAF component. Add your configuration to the `appsec_configs` list. Order matters: later entries override conflicting defaults such as `default_remediation`.
79+
80+
```yaml title="/etc/crowdsec/acquis.d/appsec.yaml"
81+
appsec_configs:
82+
- crowdsecurity/appsec-default
83+
- custom/block-nonnumeric-user-id
84+
labels:
85+
type: appsec
86+
listen_addr: 127.0.0.1:7422
87+
source: appsec
88+
```
89+
90+
If you only want to run your custom configuration, remove other entries and keep the list with a single item.
91+
92+
## Step 4 — Reload CrowdSec and Validate the Load
93+
94+
Apply the changes by reloading the CrowdSec service:
95+
96+
```bash
97+
sudo systemctl reload crowdsec
98+
```
99+
100+
If your init system does not support reload, perform a restart instead. Then verify the rule and configuration are active:
101+
102+
```bash
103+
sudo cscli appsec-rules list | grep block-nonnumeric-user-id
104+
sudo cscli appsec-configs list | grep block-nonnumeric-user-id
105+
```
106+
107+
The rule should appear as `enabled`, and the configuration should show up in the list. CrowdSec logs confirm the configuration was loaded without errors.
108+
109+
## Step 5 — Functional Test with `curl`
110+
111+
Trigger the behaviour your rule is meant to catch to ensure it blocks as expected. For the example rule, send a request with a non-numeric `user_id` value:
112+
113+
```bash
114+
curl -i 'http://127.0.0.1/profile?user_id=abc123'
115+
```
116+
117+
A successful block returns an HTTP status such as `403 Forbidden`, and CrowdSec logs a matching alert:
118+
119+
```bash
120+
sudo cscli alerts list -s custom/block-nonnumeric-user-id
121+
```
122+
123+
If the request is not blocked, double-check that the rule `name` matches the pattern in your AppSec configuration, that the acquisition file lists your configuration, and that the CrowdSec service picked up the changes.
124+
125+
## Next Steps
126+
127+
- Add automated regression tests with `cscli hubtest` so future updates do not break the rule.
128+
- Version-control your custom rule and configuration files to keep track of changes.

‎crowdsec-docs/docs/appsec/rules_syntax.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
id: rules_syntax
3-
title: Rules syntax
3+
title: Syntax
44
sidebar_position: 8
55
---
66

@@ -135,6 +135,7 @@ Match provides the pattern to match the target against, including optional trans
135135
- `trim` : remove leading and trailing spaces
136136
- `normalizepath` : normalize the path (remove double slashes, etc)
137137
- `htmlEntitydecode` : decode HTML entities
138+
- `count` : number of times the _target_ appear
138139

139140
```yaml
140141
# we want the query parameter foo to be equal to 'toto'

0 commit comments

Comments
 (0)