You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: crowdsec-docs/docs/getting_started/getting_started.md
+10-10Lines changed: 10 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,7 +3,7 @@ id: getting_started
3
3
title: Getting Started
4
4
---
5
5
6
-
import AcademyPromo from '@site/src/components/AcademyPromo';
6
+
import AcademyPromo from '@site/src/components/academy-promo';
7
7
8
8
Welcome to CrowdSec!
9
9
@@ -15,8 +15,7 @@ To embark on your CrowdSec journey, the optimal starting point is to set up a co
15
15
16
16
The CrowdSec console serves as a web-based interface enabling you to conveniently monitor all your CrowdSec instances from a centralized hub. To get started, simply [sign up here](https://app.crowdsec.net/signup).
17
17
18
-
19
-
## Deploy
18
+
## Deploy
20
19
21
20
### Walkthrough
22
21
@@ -43,8 +42,9 @@ In our updated documentation, we now refer to CrowdSec as the "Security Engine"
43
42
#### Prerequisites
44
43
45
44
The Security Engine by default uses the following ports:
46
-
- 8080/tcp for the API
47
-
- 6060/tcp for the Prometheus metrics / Debugging
45
+
46
+
- 8080/tcp for the API
47
+
- 6060/tcp for the Prometheus metrics / Debugging
48
48
49
49
If these ports are not available on your system, you can change them in the configuration file post installation. See [Configuration](/configuration/crowdsec_configuration.md) for more information.
50
50
@@ -55,15 +55,15 @@ Please note that the API is mandatory for your security engine, do not remove it
55
55
For the most straightforward installation of the Security Engine, utilize the official repository, guaranteeing you'll constantly have the latest version.
56
56
57
57
Please see the relevant documentation for your OS:
Should you opt for a source-based installation, you can follow the steps outlined [here](/getting_started/install_source.mdx).
65
66
66
-
67
67
### Remediation Component
68
68
69
69
After installing the Security Engine, you can proceed to install a Remediation Component, which is responsible for executing actions based on the decisions made by the Security Engine.
@@ -76,4 +76,4 @@ If you are uncertain about which one to install, please refer to our [Remediatio
76
76
77
77
Since you created your account on the CrowdSec console, you can now [enroll your Security Engine to your account](https://app.crowdsec.net/security-engines?enroll-engine=true).
78
78
79
-
To do so, you can find steps outlined [here](/u/getting_started/post_installation/console/#engines-page).
79
+
To do so, you can find steps outlined [here](/u/getting_started/post_installation/console/#engines-page).
@@ -81,23 +78,20 @@ From the root of the hub repository :
81
78
82
79
## Configure our test
83
80
84
-
85
81
Let's add our parser to the test configuration (`.tests/myservice-logs/config.yaml`). He specify that we need syslog-logs parser (because myservice logs are shipped via syslog), and then our custom parser.
__note: as our custom parser isn't yet part of the hub, we specify its path relative to the root of the hub directory__
100
-
94
+
**note: as our custom parser isn't yet part of the hub, we specify its path relative to the root of the hub directory**
101
95
102
96
## Parser creation : skeleton
103
97
@@ -110,29 +104,28 @@ onsuccess: next_stage
110
104
name: crowdsecurity/myservice-logs
111
105
description: "Parse myservice logs"
112
106
grok:
113
-
#our grok pattern : capture .*
114
-
pattern: ^%{DATA:some_data}$
115
-
#the field to which we apply the grok pattern : the log message itself
116
-
apply_on: message
107
+
#our grok pattern : capture .*
108
+
pattern: ^%{DATA:some_data}$
109
+
#the field to which we apply the grok pattern : the log message itself
110
+
apply_on: message
117
111
statics:
118
-
- parsed: is_my_service
119
-
value: yes
112
+
- parsed: is_my_service
113
+
value: yes
120
114
```
121
115
122
-
- a [filter](/log_processor/parsers/format.md#filter) : if the expression is `true`, the event will enter the parser, otherwise, it won't
123
-
- a [onsuccess](/log_processor/parsers/format.md#onsuccess) : defines what happens when the event was successfully parsed : shall we continue ? shall we move to next stage ? etc.
124
-
- a `name` & a `description`
125
-
- some [statics](/log_processor/parsers/format.md#statics) that will modify the event
126
-
- a `debug` flag that allows to enable local debugging information
127
-
- a `grok` pattern to capture some data in logs
128
-
116
+
- a [filter](/log_processor/parsers/format.md#filter) : if the expression is `true`, the event will enter the parser, otherwise, it won't
117
+
- a [onsuccess](/log_processor/parsers/format.md#onsuccess) : defines what happens when the event was successfully parsed : shall we continue ? shall we move to next stage ? etc.
118
+
- a `name` & a `description`
119
+
- some [statics](/log_processor/parsers/format.md#statics) that will modify the event
120
+
- a `debug` flag that allows to enable local debugging information
121
+
- a `grok` pattern to capture some data in logs
129
122
130
123
We can then "test" our parser like this :
131
124
132
125
```bash
133
126
▶ cscli hubtest run myservice-logs
134
-
INFO[01-10-2021 12:41:21 PM] Running test 'myservice-logs'
135
-
WARN[01-10-2021 12:41:24 PM] Assert file '/home/dev/github/hub/.tests/myservice-logs/parser.assert' is empty, generating assertion:
127
+
INFO[01-10-2021 12:41:21 PM] Running test 'myservice-logs'
128
+
WARN[01-10-2021 12:41:24 PM] Assert file '/home/dev/github/hub/.tests/myservice-logs/parser.assert' is empty, generating assertion:
#for clarity, we create our pattern syntax beforehand
199
191
pattern_syntax:
200
-
MYSERVICE_BADPASSWORD: bad password for user '%{USERNAME:user}' from '%{IP:source_ip}' #[1]
201
-
MYSERVICE_BADUSER: unknown user '%{USERNAME:user}' from '%{IP:source_ip}' #[1]
192
+
MYSERVICE_BADPASSWORD: bad password for user '%{USERNAME:user}' from '%{IP:source_ip}' #[1]
193
+
MYSERVICE_BADUSER: unknown user '%{USERNAME:user}' from '%{IP:source_ip}' #[1]
202
194
nodes:
203
-
#and we use them to parse our two type of logs
204
-
- grok:
205
-
name: "MYSERVICE_BADPASSWORD" #[2]
206
-
apply_on: message
207
-
statics:
208
-
- meta: log_type #[3]
209
-
value: myservice_failed_auth
210
-
- meta: log_subtype
211
-
value: myservice_bad_password
212
-
- grok:
213
-
name: "MYSERVICE_BADUSER" #[2]
214
-
apply_on: message
215
-
statics:
216
-
- meta: log_type #[3]
217
-
value: myservice_failed_auth
218
-
- meta: log_subtype
219
-
value: myservice_bad_user
195
+
#and we use them to parse our two type of logs
196
+
- grok:
197
+
name: "MYSERVICE_BADPASSWORD" #[2]
198
+
apply_on: message
199
+
statics:
200
+
- meta: log_type #[3]
201
+
value: myservice_failed_auth
202
+
- meta: log_subtype
203
+
value: myservice_bad_password
204
+
- grok:
205
+
name: "MYSERVICE_BADUSER" #[2]
206
+
apply_on: message
207
+
statics:
208
+
- meta: log_type #[3]
209
+
value: myservice_failed_auth
210
+
- meta: log_subtype
211
+
value: myservice_bad_user
220
212
statics:
221
213
- meta: service #[3]
222
214
value: myservice
@@ -227,19 +219,18 @@ statics:
227
219
```
228
220
229
221
Various changes have been made here :
230
-
- We created to patterns to capture the two relevant type of log lines, Using an [online grok debugger](https://grokdebug.herokuapp.com/) or an [online regex debugger](https://www.debuggex.com/) [2]
231
-
)
232
-
- We keep track of the username and the source_ip (Please note that setting the source_ip in `evt.Meta.source_ip` and `evt.Parsed.source_ip` is important [1])
233
-
- We setup various [statics](/log_processor/parsers/format.md#statics) information to classify the log type [3]
234
-
235
222
223
+
- We created to patterns to capture the two relevant type of log lines, Using an [online grok debugger](https://grokdebug.herokuapp.com/) or an [online regex debugger](https://www.debuggex.com/) [2]
224
+
)
225
+
- We keep track of the username and the source_ip (Please note that setting the source_ip in `evt.Meta.source_ip` and `evt.Parsed.source_ip` is important [1])
226
+
- We setup various [statics](/log_processor/parsers/format.md#statics) information to classify the log type [3]
236
227
237
228
Let's run out tests again :
238
229
239
230
```bash {13-20}
240
-
▶ cscli hubtest run myservice-logs
241
-
INFO[01-10-2021 12:49:56 PM] Running test 'myservice-logs'
242
-
WARN[01-10-2021 12:49:59 PM] Assert file '/home/dev/github/hub/.tests/myservice-logs/parser.assert' is empty, generating assertion:
231
+
▶ cscli hubtest run myservice-logs
232
+
INFO[01-10-2021 12:49:56 PM] Running test 'myservice-logs'
233
+
WARN[01-10-2021 12:49:59 PM] Assert file '/home/dev/github/hub/.tests/myservice-logs/parser.assert' is empty, generating assertion:
@@ -291,8 +282,7 @@ line: Dec 8 06:28:43 mymachine myservice[2806]: accepted connection for user 't
291
282
└ 🔴 crowdsecurity/myservice-logs
292
283
```
293
284
294
-
__note: we can see that our log line `accepted connection for user 'toto' from '192.168.1.1'` wasn't parsed by `crowdsecurity/myservice-logs` as we have no pattern for it__
295
-
285
+
**note: we can see that our log line `accepted connection for user 'toto' from '192.168.1.1'` wasn't parsed by `crowdsecurity/myservice-logs` as we have no pattern for it**
296
286
297
287
## Closing word
298
288
@@ -309,4 +299,4 @@ See as well [this blog article](https://crowdsec.net/blog/how-to-write-crowdsec-
@@ -126,16 +126,16 @@ We filter on `evt.Meta.log_type == 'myservice_failed_auth'` because in the parse
126
126
127
127
We have the following fields:
128
128
129
-
- a [type](/log_processor/scenarios/format.md#type): the type of bucket to use (trigger or leaky).
130
-
- a [name](/log_processor/scenarios/format.md#name)
131
-
- a [description](/log_processor/scenarios/format.md#description)
132
-
- a [filter](/log_processor/scenarios/format.md#type): the filter to apply on events to be filled in this bucket.
133
-
- a [leakspeed](/log_processor/scenarios/format.md#leakspeed)
134
-
- a [capacity](/log_processor/scenarios/format.md#capacity): the number of events in the bucket before it overflows.
135
-
- a [groupby](/log_processor/scenarios/format.md#groupby): a field from the event to partition the bucket. It is often the `source_ip` of the event.
136
-
- a [blackhole](/log_processor/scenarios/format.md#blackhole): the number of minute to not retrigger this scenario for the same `groupby` field.
137
-
- a [reprocess](/log_processor/scenarios/format.md#reprocess): ingest the alert in crowdsec for further processing.
138
-
- some [labels](/log_processor/scenarios/format.md#labels): Some labels are mandatory and the scenario will not be validated by the Hub if they are missing. Don't forget to set `remediation: true` if you want the IP to be blocked by bouncers.
129
+
- a [type](/log_processor/scenarios/format.md#type): the type of bucket to use (trigger or leaky).
130
+
- a [name](/log_processor/scenarios/format.md#name)
131
+
- a [description](/log_processor/scenarios/format.md#description)
132
+
- a [filter](/log_processor/scenarios/format.md#type): the filter to apply on events to be filled in this bucket.
133
+
- a [leakspeed](/log_processor/scenarios/format.md#leakspeed)
134
+
- a [capacity](/log_processor/scenarios/format.md#capacity): the number of events in the bucket before it overflows.
135
+
- a [groupby](/log_processor/scenarios/format.md#groupby): a field from the event to partition the bucket. It is often the `source_ip` of the event.
136
+
- a [blackhole](/log_processor/scenarios/format.md#blackhole): the number of minute to not retrigger this scenario for the same `groupby` field.
137
+
- a [reprocess](/log_processor/scenarios/format.md#reprocess): ingest the alert in crowdsec for further processing.
138
+
- some [labels](/log_processor/scenarios/format.md#labels): Some labels are mandatory and the scenario will not be validated by the Hub if they are missing. Don't forget to set `remediation: true` if you want the IP to be blocked by bouncers.
139
139
140
140
We can then "test" our scenario like this :
141
141
@@ -171,8 +171,8 @@ Please fill your assert file(s) for test 'myservice-bf', exiting
171
171
172
172
What happened here ?
173
173
174
-
- The scenario has been triggered and is generating some assertion (for functional test)
175
-
- In production environment, an alert would have been send to the CrowdSec Local API.
174
+
- The scenario has been triggered and is generating some assertion (for functional test)
175
+
- In production environment, an alert would have been send to the CrowdSec Local API.
176
176
177
177
We can again understand more of what is going on thanks to `cscli hubtest explain` :
0 commit comments