You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: crowdsec-docs/unversioned/user_guides/multiserver_setup.md
+154-8Lines changed: 154 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,29 +4,175 @@ title: About multi-server setup
4
4
sidebar_position: 10
5
5
---
6
6
7
-
Crowdsec's [architecture](/docs/intro#architecture) allows distributed setups, as most components communicate via [HTTP API](/docs/local_api/intro).
7
+
import useBaseUrl from "@docusaurus/useBaseUrl"
8
+
9
+
## Introduction
10
+
11
+
Crowdsec's [architecture](/docs/next/intro#architecture) allows distributed setups, as most components communicate via [HTTP API](/docs/next/local_api/intro).
8
12
9
13
When doing such, a few considerations must be kept in mind to understand the role of each component:
10
-
- The agent is in charge of [processing the logs](/docs/parsers/intro), matching them against [scenarios](/docs/scenarios/intro), and sending the resulting alerts to the [local API](/docs/local_api/intro)
14
+
- The log processor is in charge of [processing the logs](/docs/next/parsers/intro), matching them against [scenarios](/docs/next/scenarios/intro), and sending the resulting alerts to the [local API](/docs/next/local_api/intro)
11
15
- The local API (LAPI from now on) receives the alerts and converts them into decisions based on your profile
12
-
- The bouncer(s) query the LAPI to receive the decisions to be applied
16
+
- LAPI also takes care of communication with [CAPI](/docs/next/central_api/intro) to pull blocklists and push alerts to the console.
17
+
- The remediation component query the LAPI to receive the decisions to be applied
18
+
19
+
You can mix and match deployment methods and OS in the same setup, for example:
20
+
- LAPI running on a Linux server
21
+
- 1 log processor running on Windows alongside a [Windows Firewall remediation component](/u/bouncers/windows_firewall)
22
+
- 1 log processor running in Docker on Linux alongside a [Firewall remediation component](/u/bouncers/firewall) running on Linux
23
+
- 1 [Nginx remediation component](/u/bouncers/nginx) running on your webserver
This guide will focus on using login/password authentication for the log processors for simplicity.
36
+
37
+
You can also use [TLS Authentication](/docs/next/local_api/tls_auth), which does not require to validate log processors but you will need to create a PKI.
38
+
39
+
:::
40
+
41
+
### LAPI
42
+
43
+
Follow the [getting started guide](/docs/next/getting_started/install_crowdsec) to install Crowdsec.
44
+
45
+
You will need to edit the `/etc/crowdsec/config.yaml` file to make LAPI listen on all interfaces:
46
+
```yaml
47
+
api:
48
+
server:
49
+
listen_uri: 0.0.0.0:8080
50
+
```
51
+
52
+
Optionally, if you only want to run a LAPI instance on this machine, you can disable the log processor in the same file by removing the `crowdsec_service` section.
53
+
54
+
You can also enable automatic registration of new machines to simplify adding log processors in the future by creating by adding the following to the configuration file:
Because a log processor can push arbitrary alerts to LAPI (and hence can easily lock you out), make sure to restrict as much as possible the allowed IPs and keep the token safe.
71
+
72
+
:::
73
+
74
+
Finally, restart crowdsec to apply the changes.
75
+
76
+
### Log processors
77
+
78
+
Again, follow the [getting started guide](/docs/next/getting_started/install_crowdsec) to install Crowdsec.
79
+
80
+
Once the installation is done, you need to edit the `/etc/crowdsec/config.yaml` to disable the LAPI running by default.
81
+
To do so, you can remove the entire `api.server` section from the file.
82
+
83
+
You can now use `cscli` to register the log processor in your LAPI:
You can also verify the log processors can properly authenticate with LAPI by running this command on the machine the log processor is installed on:
116
+
```bash
117
+
$ sudo cscli lapi status
118
+
Loaded credentials from /etc/crowdsec/local_api_credentials.yaml
119
+
Trying to authenticate with username XXXXX on http://crowdsec.local:8080/
120
+
You can successfully interact with Local API (LAPI)
121
+
```
122
+
123
+
Repeat this procedure for each log processor you want to add to LAPI.
124
+
125
+
:::warning
126
+
127
+
Log processors do not share any information between them.
128
+
For example, if a load balancer randomly distributes traffic accross multiple web servers, it will take more time to detect bad traffic, as each log processor will only be seeing the logs from its respective server.
129
+
130
+
In this case, we recommend using a centralized logging solution and have a single log processor reading the logs there.
131
+
132
+
:::
133
+
134
+
### Remediation Components
135
+
136
+
::: info
137
+
138
+
Since crowdsec v1.6.4, multiple remediations components running on different machines can use the same API key.
139
+
140
+
:::
141
+
142
+
On installation, remediations components will try to automatically create an API key if they are installed on the same machine as LAPI, which likely won't be the case for a multi-server installation.
143
+
144
+
In this case, you will need to manually create an API key for you remediation component:
145
+
146
+
```bash
147
+
$ sudo cscli bouncers add MyBouncer
148
+
API key for 'MyBouncer':
149
+
150
+
ulOPOSWxLcD8LaNmOMKOkYaG7AQYY+qZ2ho7pPyCAIU
151
+
152
+
Please keep this key since you will not be able to retrieve it!
153
+
```
154
+
155
+
Next, update the remediation component configuration file with the API key that you created and the URL to LAPI.
156
+
157
+
Remediation components will generally store their configuration in `/etc/crowdsec/bouncers/`, and the configuration directives naming for the API key and URL might vary from one remediation component to the other, please refer to the specific documentation of the component you have installed.
13
158
14
159
15
160
A typical multi server setup should thus have:
16
161
17
-
1.**Agents push alerts to LAPI** :
162
+
1. **Log processors push alerts to LAPI** :
18
163
- The [local_api_credentials.yaml](/docs/configuration/crowdsec_configuration#client) should point to LAPI's Ip
19
-
- The agent should be registered to the local api
164
+
- The log processor should be registered to the local api
20
165
21
166
**Using login/password authentication**
22
-
- By running `cscli machines add MyMachine` on the LAPI (and copy the generated credentials to the agent)
23
-
- or by running `cscli lapi register --machine MyMachine --url http://<lapi>` on the agent and accepting the machine from LAPI with `cscli machines validate MyMachine`
167
+
- By running `cscli machines add MyMachine` on the LAPI (and copy the generated credentials to the log processor configuration)
168
+
- By running `cscli lapi register --machine MyMachine --url <lapi_url> --token <token>` on the log processor for automatic validation of the machine if configured
169
+
- or by running `cscli lapi register --machine MyMachine --url http://<lapi>` on the log processor and accepting the machine from LAPI with `cscli machines validate MyMachine`
24
170
25
171
**Using client cert authentication**
26
172
- By using setting the [client verification method](/docs/next/configuration/crowdsec_configuration#client_verification)
27
173
- And setting the appropriate [allowed agents ou](/docs/next/configuration/crowdsec_configuration#agents_allowed_ou)
28
174
29
-
Once done, you can check that the agent can communicate with LAPI :
175
+
Once done, you can check that the log processor can communicate with LAPI :
0 commit comments