chore: updates

Author: LaurenceJJones

crowdsec-docs/components.json

@@ -0,0 +1,20 @@
+{
+    "$schema": "https://ui.shadcn.com/schema.json",
+    "tailwind": {
+      "config": "tailwind.config.js",
+      "css": "src/css/custom.css",
+      "baseColor": "neutral",
+      "prefix": "tw-",
+      "cssVariables": false
+    },
+    "rsc": true,
+    "aliases": {
+        "utils": "@/src/utils",
+        "components": "@/src/components",
+        "ui": "@/src/ui",
+        "hooks": "@/src/hooks",
+        "lib": "@/src/lib"
+    },
+    "style": "default"
+  }
+  

crowdsec-docs/docs/appsec/configuration.md

@@ -58,6 +58,88 @@ inband_rules:
 #...
 ```
 
+## Disabling rules at runtime
+
+Even though we try to provide rules without false positives, sometimes a virtual patching rule can block legitimate requests on a website.
+
+You can disable rules at runtime, either globally (for all requests) or based on specific conditions (source IP, URI, ...).
+
+You can can disable rules by:
+ - Name with `RemoveInBandRuleByName`: Intended for disabling rules provided by crowdsec (the name is the name of the appsec-rule as seen in `cscli appsec-rules list`).
+ - ID with `RemoveInBandRuleByID`: Intended for disabling seclang rules
+ - Tag with `RemoveInBandRuleByTag`: Intended for disabling seclang rules
+
+The same functions exist for out-of-band rules, prefixed with `RemovedOutBandRuleBy...`
+
+To disable a rule, we'll first create a new `appsec-config` to avoid tainting the configuration from the hub (if you are already using a custom configuration, you can update this one instead).
+
+```yaml title="/etc/crowdsec/appsec-configs/my_config.yaml"
+name: custom/my_config
+on_load:
+ - apply:
+    - RemoveInBandRuleByName("crowdsecurity/vpatch-env-access")
+pre_eval:
+ - filter: IsInBand == true && req.URL.Path startsWith "/bar/"
+   apply:
+    - RemoveInBandRuleByName("crowdsecurity/generic-wordpress-uploads-php")
+```
+
+We are using the [hooks](/docs/appsec/hooks.md) provided by the appsec to modify the configuration in 2 places:
+ - `on_load`: Expressions here will be applied when crowdsec loads the configuration, effectively disabling the rule `crowdsecurity/vpatch-env-access` globally.
+ - `pre_eval`: Expressions here will be applied only if the provided filter matches. In this example, we are disabling the rule `crowdsecurity/generic-wordpress-uploads-php` only if the request URI starts with `/blog/` and if we are currently processing in-band rules.
+
+You can also disable native (seclang) rules by providing their ID with the `RemoveInBandRuleByID` helper. See the [hooks](appsec/hooks.md) documentation for a list of available helpers.
+
+Also note that we are not loading any rules in our custom config: the rules are loaded by the `crowdsecurity/appsec-default` config, and we are just modifying the runtime behavior with this config.
+
+Finally, we need to tell crowdsec to load our new config:
+
+```yaml title="/etc/crowdsec/acquis.d/appsec.yaml"
+appsec_configs:
+ - crowdsecurity/appsec-default
+ - custom/my_config
+labels:
+  type: appsec
+listen_addr: 127.0.0.1:7422
+source: appsec
+```
+
+## Allowlisting
+
+### Fully allow a specific IP or range
+
+If you want to ignore all rule matches for a specific IP or range, you can use a [centralized allowlist](local_api/allowlists.md).
+
+Rules will be processed as usual, but the request will not be blocked even if a rule matches.
+
+### Disable specific rules for a specific IP/range
+
+If you want to disable rule(s) for a specific IP (or range), you will need to use the `pre_eval` hook (refer to the section above for more details):
+
+```yaml title="/etc/crowdsec/appsec-configs/my_config.yaml"
+name: custom/my_config
+pre_eval:
+ - filter: req.RemoteAddr == "1.2.3.4"
+   apply:
+    - RemoveInBandRuleByName("crowdsecurity/generic-wordpress-uploads-php")
+```
+
+### Disable appsec for a specific FQDN
+
+If your reverse-proxy forwards all requests to crowdsec, regardless of the FQDN, you can disable the appsec for specific domain with a custom appsec-config (ie, the request will be always allowed):
+
+```yaml title="/etc/crowdsec/appsec-configs/my_config.yaml"
+name: custom/my_config
+on_match:
+ - filter: req.URL.Host == "foo.com"
+   apply:
+    - CancelEvent()
+    - CancelAlert()
+    - SetRemediation("allow")
+```
+
+With this config, the rules will still be evaluated, but if a rule matches no alert or event will be generated, and the remediation will be set to `allow`(ie, instruct the bouncer to let the request through).
+
 ## Appsec configuration
 
 The AppSec configuration is referenced by the acquisition configuration (`appsec_config`, `appsec_configs` or `appsec_config_path`):

crowdsec-docs/docs/appsec/hooks.md

@@ -211,4 +211,14 @@ Any other values (including `ban` and `captcha`) are transmitted as-is to the re
 
 
 
+### `req` object
 
+The `pre_eval`, `on_match` and `post_eval` hooks have access to a `req` variable that represents the HTTP request that was forwarded to the appsec.
+
+It's a Go [http.Request](https://pkg.go.dev/net/http#Request) object, so you can directly access all the details about the request.
+
+For example:
+ - To get the requested URI: `req.URL.Path`
+ - To get the client IP: `req.RemoteAddr`
+ - To get the HTTP method: `req.Method`
+ - To get the FQDN: `req.URL.Host`

crowdsec-docs/docs/appsec/intro.md

@@ -70,8 +70,7 @@ You can follow our quick start guides depending on your web server:
 Or consider learning more about the AppSec capabilities:
 
 -   **Rules**: [How to read, write and debug rules](/appsec/rules_syntax.md)
- <!-- TODO -->
--   **Scenarios**: How to create scenarios that leverage the AppSec Component events
--   **Hooks**: [For advanced use let's talk about possible Hooks](/appsec/hooks.md)
+-   **Scenarios**: [How to create scenarios that leverage the AppSec Component events](/appsec/alerts_and_scenarios.md)
+-   **Hooks**: [To customise behavior of the AppSec at runtime](/appsec/hooks.md)
 -   **Troubleshoot**: [How to troubleshoot the behavior of the AppSec Component](/appsec/troubleshooting.md)
 -   **AppSec Protocol**: [if you're maintaining or creating a remedation component and want to add the AppSec capabilities](/appsec/protocol.md)

crowdsec-docs/docs/appsec/protocol.md

@@ -22,14 +22,15 @@ This documentation can be useful in case you want to write your own remediation
 
 To work with the CrowdSec application security component, some HTTP headers are require, in addition to the other HTTP headers and the body of the original request.
 
-| Header Name                 | Description                                                                |
-| --------------------------- | -------------------------------------------------------------------------- |
-| `X-Crowdsec-Appsec-Ip`      | The Real IP address of the original HTTP request                           |
-| `X-Crowdsec-Appsec-Uri`     | The URI of the original HTTP request                                       |
-| `X-Crowdsec-Appsec-Host`    | The Host of the original HTTP request                                      |
-| `X-Crowdsec-Appsec-Verb`    | The Method of the original HTTP request                                    |
-| `X-Crowdsec-Appsec-Api-Key` | The API Key to communicate with the CrowdSec application security component |
-| `X-Crowdsec-Appsec-User-Agent`| The User-Agent of the original HTTP request                              |
+| Header Name                      | Description                                                                           |
+| -------------------------------- | ------------------------------------------------------------------------------------- |
+| `X-Crowdsec-Appsec-Ip`           | The Real IP address of the original HTTP request                                      |
+| `X-Crowdsec-Appsec-Uri`          | The URI of the original HTTP request                                                  |
+| `X-Crowdsec-Appsec-Host`         | The Host of the original HTTP request                                                 |
+| `X-Crowdsec-Appsec-Verb`         | The Method of the original HTTP request                                               |
+| `X-Crowdsec-Appsec-Api-Key`      | The API Key to communicate with the CrowdSec application security component           |
+| `X-Crowdsec-Appsec-User-Agent`   | The User-Agent of the original HTTP request                                           |
+| `X-Crowdsec-Appsec-Http-Version` | The HTTP version used by the original HTTP request (in integer form `10`, `11`, ...) |
 
 :::note
 
@@ -96,11 +97,11 @@ username=admin' OR '1'='1' -- &password=password
 
 According to the result of the processing of the HTTP request, the application security component will respond with a different HTTP code and body.
 
-| HTTP Code | Description                                                                                                                                          | Body                                             |
-| --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ |
-| `200`     | The HTTP request is allowed                                                                                                                          | `{"action" : "allow"}`                           |
-| `403`     | The HTTP request triggered one or more application security component rules                                                                           | `{"action" : "ban", "http_status": 403}` or `{"action" : "captcha", "http_status": 403}` |
-| `500`     | An error occurred in the application security component. The remediation component must support a `APPSEC_FAILURE_ACTION` parameter to handle this case | `null`                                           |
-| `401`     | The remediation component is not authenticated. It must use the same API Key that was generated to pull the local API request                        | `null`                                           |
+| HTTP Code | Description                                                                                                                                             | Body                                                                                     |
+| --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
+| `200`     | The HTTP request is allowed                                                                                                                             | `{"action" : "allow"}`                                                                   |
+| `403`     | The HTTP request triggered one or more application security component rules                                                                             | `{"action" : "ban", "http_status": 403}` or `{"action" : "captcha", "http_status": 403}` |
+| `500`     | An error occurred in the application security component. The remediation component must support a `APPSEC_FAILURE_ACTION` parameter to handle this case | `null`                                                                                   |
+| `401`     | The remediation component is not authenticated. It must use the same API Key that was generated to pull the local API request                           | `null`                                                                                   |
 
 In case of a `403` response, the body will contain the action to take and the HTTP status code to return to the client.

crowdsec-docs/docs/appsec/quickstart/traefik.mdx

@@ -157,6 +157,7 @@ http:
     crowdsec:
       plugin:
         bouncer:
+          enabled: true
           crowdsecAppsecEnabled: true
           crowdsecAppsecHost: crowdsec:7422
           crowdsecAppsecFailureBlock: true
@@ -168,6 +169,7 @@ Instead if you define the configuration using labels on the containers you can a
 
 ```yaml
   labels:
+      - "traefik.http.middlewares.crowdsec-bar.plugin.bouncer.enabled=true"
       - "traefik.http.middlewares.crowdsec-bar.plugin.bouncer.crowdsecAppsecEnabled=true"
       - "traefik.http.middlewares.crowdsec-bar.plugin.bouncer.crowdsecAppsecHost=crowdsec:7422"
       - "traefik.http.middlewares.crowdsec-bar.plugin.bouncer.crowdsecLapiKey=privateKey-foo"

crowdsec-docs/docs/configuration/crowdsec_configuration.md

@@ -114,6 +114,7 @@ common:
   log_max_age: <max_age_of_log_file>
   log_max_files: <number_of_log_files_to_keep>
   compress_logs: (true|false)
+  log_format: "(text|json)"
 config_paths:
   config_dir: "<path_to_crowdsec_config_folder>"
   data_dir: "<path_to_crowdsec_data_folder>"
@@ -145,6 +146,9 @@ db_config:
   host:     "<db_host_ip>"   # for mysql/pgsql
   port:     "<db_host_port>" # for mysql/pgsql
   sslmode:  "<require/disable>" # for pgsql
+  ssl_ca_cert: "<path_to_ca_cert_file>" # for mysql/pgsql
+  ssl_client_cert: "<path_to_client_cert_file>" # for mysql/pgsql
+  ssl_client_key: "<path_to_client_key_file>" # for mysql/pgsql
   use_wal:  "true|false" # for sqlite
   max_open_conns: "<max_number_of_conns_to_db>"
   flush:
@@ -167,6 +171,7 @@ api:
   client:
     insecure_skip_verify: "(true|false)"
     credentials_path: "<path_to_local_api_client_credential_file>"
+    unregister_on_exit: "(true|false)"
   server:
     enable: <true|false> # enable or disable local API
     log_level: "(error|info|debug|trace>")"
@@ -223,6 +228,7 @@ common:
   log_max_age: <max_age_of_log_file>
   log_max_files: <number_of_log_files_to_keep>
   compress_logs: (true|false)
+  log_format: "(text|json)"
 ```
 
 #### `daemonize`
@@ -279,6 +285,11 @@ Maximum number of old log files to retain.  The default is to retain 3 old log f
 
 Whether to compress the log file after rotation or not.
 
+#### `log_format`
+> string
+
+Format of crowdsec log. Can be `text` (default) or `json`
+
 ### `config_paths`
 
 This section contains most paths to various sub configuration items.
@@ -452,6 +463,9 @@ db_config:
   host:     "<db_host_ip>"   # for mysql/postgresql/pgx # must be omitted if using socket file
   port:     "<db_host_port>" # for mysql/postgresql/pgx # must be omitted if using socket file
   sslmode:  "<require/disable>" # for postgresql/pgx
+  ssl_ca_cert: "<path_to_ca_cert_file>" # for mysql/pgsql
+  ssl_client_cert: "<path_to_client_cert_file>" # for mysql/pgsql
+  ssl_client_key: "<path_to_client_key_file>" # for mysql/pgsql
   max_open_conns: "<max_number_of_conns_to_db>"
   decision_bulk_size: "<decision_bulk_size>"
   flush:
@@ -549,13 +563,48 @@ db_config:
 The port to connect to (only if the type of database is `mysql` or `postgresql`). Must be omitted if using socket file.
 
 
+#### `sslmode`
+
 ```yaml
 db_config:
   type: postgresql
 
   sslmode: require
 ```
-Require or disable ssl connection to database (only if the type of database is `postgresql`). See [PostgreSQL SSL modes](https://www.postgresql.org/docs/current/libpq-ssl.html#LIBPQ-SSL-SSLMODE-STATEMENTS) for possible values.
+Require or disable ssl connection to database (only if the type of database is `mysql` or `postgresql` or `pgx`).
+
+See [PostgreSQL SSL modes](https://www.postgresql.org/docs/current/libpq-ssl.html#LIBPQ-SSL-SSLMODE-STATEMENTS) for possible values.
+See [MySQL SSL modes](https://dev.mysql.com/doc/refman/8.0/en/using-encrypted-connections.html) for possible values within the `Client-Side` configuration.
+
+#### `ssl_ca_cert`
+
+```yaml
+db_config:
+  type: mysql|postgresql|pgx
+
+  ssl_ca_cert: /path/to/ca.crt
+```
+Path to the CA certificate file (only if the type of database is `mysql` or `postgresql` or `pgx`)
+
+#### `ssl_client_cert`
+
+```yaml
+db_config:
+  type: mysql|postgresql|pgx
+
+  ssl_client_cert: /path/to/client.crt
+```
+Path to the client certificate file when using mTLS (only if the type of database is `mysql` or `postgresql` or `pgx`)
+
+#### `ssl_client_key`
+
+```yaml
+db_config:
+  type: mysql|postgresql|pgx
+
+  ssl_client_key: /path/to/client.key
+```
+Path to the client key file when using mTLS (only if the type of database is `mysql` or `postgresql` or `pgx`)
 
 #### `max_open_conns`
 
@@ -693,6 +742,7 @@ api:
   client:
     insecure_skip_verify: "(true|false)"
     credentials_path: "<path_to_local_api_client_credential_file>"
+    unregister_on_exit: "(true|false)"
   server:
     enable: <true|false>
     log_level: "(error|info|debug|trace>"
@@ -782,6 +832,7 @@ The client subsection is used by `crowdsec` and `cscli` to read and write decisi
 client:
   insecure_skip_verify: "(true|false)"
   credentials_path: "<path_to_local_api_client_credential_file>"
+  unregister_on_exit: "(true|false)"
 ```
 
 ##### `insecure_skip_verify`
@@ -794,6 +845,13 @@ Allows the use of https with self-signed certificates.
 
 Path to the credential files (contains API url + login/password).
 
+##### `unregister_on_exit`
+>bool
+
+If set to `true`, the log processor will remove delete itself from LAPI when stopping.
+
+Intended for use in dynamic environment such as Kubernetes.
+
 #### `server`
 
 The `server` subsection is the local API configuration.
@@ -864,6 +922,13 @@ This option will disable the registration of remote agents using `cscli lapi reg
 ##### `capi_whitelists_path`
 > string
 
+:::warning
+
+This option is deprecated.
+You should use [centralized allowlists](local_api/allowlists.md) instead.
+
+:::
+
 The path to whitelists file for community and 3rd party blocklists.
 Those IPs/CIDR whitelists apply on all the IPs received from community blocklist or 3rd party lists subscriptions.
 

crowdsec-docs/docs/cscli/cscli.md

@@ -28,6 +28,7 @@ It is meant to allow you to manage bans, parsers/scenarios/etc, api and generall
 ### SEE ALSO
 
 * [cscli alerts](/cscli/cscli_alerts.md)	 - Manage alerts
+* [cscli allowlists](/cscli/cscli_allowlists.md)	 - Manage centralized allowlists
 * [cscli appsec-configs](/cscli/cscli_appsec-configs.md)	 - Manage hub appsec-configs
 * [cscli appsec-rules](/cscli/cscli_appsec-rules.md)	 - Manage hub appsec-rules
 * [cscli bouncers](/cscli/cscli_bouncers.md)	 - Manage bouncers [requires local API]

crowdsec-docs/docs/cscli/cscli_allowlists.md

@@ -0,0 +1,37 @@
+---
+id: cscli_allowlists
+title: cscli allowlists
+---
+## cscli allowlists
+
+Manage centralized allowlists
+
+### Options
+
+```
+  -h, --help   help for allowlists
+```
+
+### Options inherited from parent commands
+
+```
+      --color string    Output color: yes, no, auto (default "auto")
+  -c, --config string   path to crowdsec config file (default "/etc/crowdsec/config.yaml")
+      --debug           Set logging to debug
+      --error           Set logging to error
+      --info            Set logging to info
+  -o, --output string   Output format: human, json, raw
+      --trace           Set logging to trace
+      --warning         Set logging to warning
+```
+
+### SEE ALSO
+
+* [cscli](/cscli/cscli.md)	 - cscli allows you to manage crowdsec
+* [cscli allowlists add](/cscli/cscli_allowlists_add.md)	 - Add content to an allowlist
+* [cscli allowlists create](/cscli/cscli_allowlists_create.md)	 - Create a new allowlist
+* [cscli allowlists delete](/cscli/cscli_allowlists_delete.md)	 - Delete an allowlist
+* [cscli allowlists inspect](/cscli/cscli_allowlists_inspect.md)	 - Inspect an allowlist
+* [cscli allowlists list](/cscli/cscli_allowlists_list.md)	 - List all allowlists
+* [cscli allowlists remove](/cscli/cscli_allowlists_remove.md)	 - Remove content from an allowlist
+