You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This **Remediation Component** (aka Bouncer) deploys a **Cloudflare Worker** in front of a **Cloudflare Zone/Website**, which checks if incoming request's IP address/Country/AS is present in a KV store and takes necessary remedial actions.
27
+
It also periodically updates the KV store with IPs coming from your **CrowdSec Security Engine** OR a **Blocklist as a Service Integration**for Remediation components.
28
+
29
+
Useful links:
30
+
***Register** this **remediation component** into your **Security engine**: [Here](/u/bouncers/intro).
31
+
* To setup a **Blocklist as a Service** endpoint for a **remediation component**: [Here](/u/integrations/remediationcomponent).
32
+
33
+
The following documentation dives into the installation, configuration, and troubleshooting of the **CrowdSec Cloudflare Worker** Remediation Component.
34
+
35
+
26
36
27
37
:::warning
28
38
This Remediation Component heavily relies on Cloudflare Workers and KV store. It works best on a paid Workers subscription.
@@ -33,7 +43,6 @@ More explanation in the chapter [Test with Cloudflare free plan](#appendix-test-
33
43
After configuring and starting the Remediation Component, please see the [setting up worker fail mode](#setting-up-the-worker-route-fail-mode) section.
34
44
:::
35
45
36
-
This Remediation Component (aka Bouncer) deploys a Cloudflare Worker in front of a Cloudflare Zone/Website, which checks if incoming request's IP address/Country/AS is present in a KV store and takes necessary remedial actions. It also periodically updates the KV store with CrowdSec LAPI's decisions.
1. Create a Cloudflare Worker and a Worker KV per configured account.
138
149
2. Create a Worker Route(s) per configured zone. Any request matching the route would be handled by the worker.
139
150
3. For every matching incoming request, the worker checks whether it's IP, Country and AS have a decision against. It checks for this in it's KV store. If found it performs the corresponding remediation.
140
-
4. The Remediation Component also periodically updates the KV store with the latest decisions from CrowdSec.
151
+
4. The Remediation Component also periodically updates the KV store with the latest decisions from CrowdSec's **Security Engine** OR **Blocklist Integration**.
141
152
142
153
If your Cloudflare token has permission on D1, the remediation component will also automatically create a database to track:
@@ -4,59 +4,158 @@ title: About multi-server setup
4
4
sidebar_position: 10
5
5
---
6
6
7
-
Crowdsec's [architecture](/docs/intro#architecture) allows distributed setups, as most components communicate via [HTTP API](/docs/local_api/intro).
7
+
import useBaseUrl from "@docusaurus/useBaseUrl"
8
+
9
+
## Introduction
10
+
11
+
Crowdsec's [architecture](/docs/next/intro#architecture) allows distributed setups, as most components communicate via [HTTP API](/docs/next/local_api/intro).
8
12
9
13
When doing such, a few considerations must be kept in mind to understand the role of each component:
10
-
- The agent is in charge of [processing the logs](/docs/parsers/intro), matching them against [scenarios](/docs/scenarios/intro), and sending the resulting alerts to the [local API](/docs/local_api/intro)
14
+
- The log processor is in charge of [processing the logs](/docs/next/parsers/intro), matching them against [scenarios](/docs/next/scenarios/intro), and sending the resulting alerts to the [local API](/docs/next/local_api/intro)
11
15
- The local API (LAPI from now on) receives the alerts and converts them into decisions based on your profile
12
-
- The bouncer(s) query the LAPI to receive the decisions to be applied
16
+
- LAPI also takes care of communication with [CAPI](/docs/next/central_api/intro) to pull blocklists and push alerts to the console.
17
+
- The remediation component query the LAPI to receive the decisions to be applied
18
+
19
+
You can mix and match deployment methods and OS in the same setup, for example:
20
+
- LAPI running on a Linux server
21
+
- 1 log processor running on Windows alongside a [Windows Firewall remediation component](/u/bouncers/windows_firewall)
22
+
- 1 log processor running in Docker on Linux alongside a [Firewall remediation component](/u/bouncers/firewall) running on Linux
23
+
- 1 [Nginx remediation component](/u/bouncers/nginx) running on your webserver
This guide will focus on using login/password authentication for the log processors for simplicity.
36
+
37
+
You can also use [TLS Authentication](/docs/next/local_api/tls_auth), which does not require to validate log processors but you will need to create a PKI.
38
+
39
+
:::
40
+
41
+
### LAPI
42
+
43
+
Follow the [getting started guide](/docs/next/getting_started/install_crowdsec) to install Crowdsec.
44
+
45
+
You will need to edit the `/etc/crowdsec/config.yaml` file to make LAPI listen on all interfaces:
46
+
```yaml
47
+
api:
48
+
server:
49
+
listen_uri: 0.0.0.0:8080
50
+
```
51
+
52
+
Optionally, if you only want to run a LAPI instance on this machine, you can disable the log processor in the same file by removing the `crowdsec_service` section.
53
+
54
+
You can also enable automatic registration of new machines to simplify adding log processors in the future by adding the following to the configuration file:
Because a log processor can push arbitrary alerts to LAPI (and hence can easily lock you out), make sure to restrict as much as possible the allowed IPs and keep the token safe.
71
+
72
+
:::
73
+
74
+
Finally, restart crowdsec to apply the changes.
13
75
76
+
Note that LAPI only receives the alerts and turn them into decisions, this means:
77
+
- You do not have to install any parser or scenario on it, they must be installed on the log processors directly.
78
+
- If you want to have custom decisions (custom duration for example), you need to modify the file `/etc/crowdsec/profiles.yaml` on the LAPI, not on the log processors.
14
79
15
-
A typical multi server setup should thus have:
80
+
### Log processors
16
81
17
-
1.**Agents push alerts to LAPI** :
18
-
- The [local_api_credentials.yaml](/docs/configuration/crowdsec_configuration#client) should point to LAPI's Ip
19
-
- The agent should be registered to the local api
82
+
Again, follow the [getting started guide](/docs/next/getting_started/install_crowdsec) to install Crowdsec.
20
83
21
-
**Using login/password authentication**
22
-
- By running `cscli machines add MyMachine` on the LAPI (and copy the generated credentials to the agent)
23
-
- or by running `cscli lapi register --machine MyMachine --url http://<lapi>` on the agent and accepting the machine from LAPI with `cscli machines validate MyMachine`
84
+
Once the installation is done, you need to edit the `/etc/crowdsec/config.yaml` to disable the LAPI running by default.
85
+
To do so, you can remove the entire `api.server` section from the file.
24
86
25
-
**Using client cert authentication**
26
-
- By using setting the [client verification method](/docs/next/configuration/crowdsec_configuration#client_verification)
27
-
- And setting the appropriate [allowed agents ou](/docs/next/configuration/crowdsec_configuration#agents_allowed_ou)
87
+
You can now use `cscli` to register the log processor in your LAPI:
28
88
29
-
Once done, you can check that the agent can communicate with LAPI :
You can also verify the log processors can properly authenticate with LAPI by running this command on the machine the log processor is installed on:
120
+
```bash
121
+
$ sudo cscli lapi status
122
+
Loaded credentials from /etc/crowdsec/local_api_credentials.yaml
123
+
Trying to authenticate with username XXXXX on http://crowdsec.local:8080/
124
+
You can successfully interact with Local API (LAPI)
125
+
```
126
+
127
+
Repeat this procedure for each log processor you want to add to LAPI.
128
+
129
+
:::warning
130
+
131
+
Log processors do not share any information between them.
132
+
For example, if a load balancer randomly distributes traffic accross multiple web servers, it will take more time to detect bad traffic, as each log processor will only be seeing the logs from its respective server.
133
+
134
+
In this case, we recommend using a centralized logging solution and have a single log processor reading the logs there.
39
135
40
-
:::info
41
-
To avoid any confusion, disabling the LAPI service on the machine running the agent can be done by commenting out the api->server section in the `config.yaml` file
42
136
:::
43
137
138
+
### Remediation Components
139
+
140
+
:::info
141
+
142
+
Since crowdsec v1.6.4, multiple remediations components running on different machines can use the same API key.
143
+
144
+
:::
44
145
146
+
On installation, remediations components will try to automatically create an API key if they are installed on the same machine as LAPI, which likely won't be the case for a multi-server installation.
45
147
46
-
2.**Bouncers speaking to LAPI**
47
-
-:warning: Most of the bouncers installers are going to assume that LAPI is running on the same machine
48
-
- You need to modify the bouncer's configuration (in `/etc/crowdsec/bouncers/`) to be sure they speak to the LAPI:
49
-
- Create an API key from LAPI with `cscli bouncers add MyBouncer`
50
-
- Or again, rely on [client certificate authentication](/docs/next/configuration/crowdsec_configuration#bouncers_allowed_ou) for the bouncers that support it
51
-
- Edit the bouncer's configuration file to be sure it points to the LAPI uri and uses the newly generated API key
148
+
In this case, you will need to manually create an API key for you remediation component by running this command on your LAPI instance:
52
149
150
+
```bash
151
+
$ sudo cscli bouncers add MyBouncer
152
+
API key for 'MyBouncer':
53
153
154
+
ulOPOSWxLcD8LaNmOMKOkYaG7AQYY+qZ2ho7pPyCAIU
54
155
55
-
## Things to keep in mind
156
+
Please keep this key since you will not be able to retrieve it!
157
+
```
56
158
57
-
- Parsers and Scenarios must be present on the agents. It's not useful to deploy them on LAPI
58
-
- Decisions are made by LAPI. This is where you want to setup eventual custom profiles, and this is where you bouncers should point
59
-
- You can use the [console](https://app.crowdsec.net), it supports multiserver setups!
60
-
- If you have an important setup, switching LAPI backend from SQLite to MySQL/PgSQL is strongly advised
159
+
Next, update the remediation component configuration file with the API key that you created and the URL to LAPI.
61
160
62
-
[This existing article](https://www.crowdsec.net/blog/multi-server-setup/)might as well be useful!
161
+
Remediation components will generally store their configuration in `/etc/crowdsec/bouncers/`, and the configuration directives naming for the API key and URL might vary from one remediation component to the other, please refer to the specific documentation of the component you have installed.
0 commit comments