Skip to content

Commit 54a9ab8

Browse files
committed
feat ✨update to typescript
1 parent 07fa82f commit 54a9ab8

File tree

75 files changed

+1236
-1234
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

75 files changed

+1236
-1234
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
.vscode
22
node_modules
3+
.history

crowdsec-docs/.npmrc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
v20.18.1

crowdsec-docs/babel.config.js

Lines changed: 0 additions & 3 deletions
This file was deleted.

crowdsec-docs/docs/appsec/quickstart/traefik.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ id: traefik
33
title: Traefik
44
---
55

6-
import FormattedTabs from '@site/src/components/FormattedTabs';
6+
import FormattedTabs from '@site/src/components/formatted-tabs';
77

88
# CrowdSec WAF QuickStart for Traefik
99

crowdsec-docs/docs/getting_started/getting_started.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ id: getting_started
33
title: Getting Started
44
---
55

6-
import AcademyPromo from '@site/src/components/AcademyPromo';
6+
import AcademyPromo from '@site/src/components/academy-promo';
77

88
Welcome to CrowdSec!
99

@@ -15,8 +15,7 @@ To embark on your CrowdSec journey, the optimal starting point is to set up a co
1515

1616
The CrowdSec console serves as a web-based interface enabling you to conveniently monitor all your CrowdSec instances from a centralized hub. To get started, simply [sign up here](https://app.crowdsec.net/signup).
1717

18-
19-
## Deploy
18+
## Deploy
2019

2120
### Walkthrough
2221

@@ -43,8 +42,9 @@ In our updated documentation, we now refer to CrowdSec as the "Security Engine"
4342
#### Prerequisites
4443

4544
The Security Engine by default uses the following ports:
46-
- 8080/tcp for the API
47-
- 6060/tcp for the Prometheus metrics / Debugging
45+
46+
- 8080/tcp for the API
47+
- 6060/tcp for the Prometheus metrics / Debugging
4848

4949
If these ports are not available on your system, you can change them in the configuration file post installation. See [Configuration](/configuration/crowdsec_configuration.md) for more information.
5050

@@ -55,15 +55,15 @@ Please note that the API is mandatory for your security engine, do not remove it
5555
For the most straightforward installation of the Security Engine, utilize the official repository, guaranteeing you'll constantly have the latest version.
5656

5757
Please see the relevant documentation for your OS:
58-
- [Linux](/getting_started/install.mdx)
59-
- [FreeBSD](/getting_started/install_freebsd.md)
60-
- [Windows](/getting_started/getting_started_on_windows.md)
58+
59+
- [Linux](/getting_started/install.mdx)
60+
- [FreeBSD](/getting_started/install_freebsd.md)
61+
- [Windows](/getting_started/getting_started_on_windows.md)
6162

6263
#### Installing from source
6364

6465
Should you opt for a source-based installation, you can follow the steps outlined [here](/getting_started/install_source.mdx).
6566

66-
6767
### Remediation Component
6868

6969
After installing the Security Engine, you can proceed to install a Remediation Component, which is responsible for executing actions based on the decisions made by the Security Engine.
@@ -76,4 +76,4 @@ If you are uncertain about which one to install, please refer to our [Remediatio
7676

7777
Since you created your account on the CrowdSec console, you can now [enroll your Security Engine to your account](https://app.crowdsec.net/security-engines?enroll-engine=true).
7878

79-
To do so, you can find steps outlined [here](/u/getting_started/post_installation/console/#engines-page).
79+
To do so, you can find steps outlined [here](/u/getting_started/post_installation/console/#engines-page).

crowdsec-docs/docs/intro.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ id: intro
44
---
55

66
import useBaseUrl from "@docusaurus/useBaseUrl"
7-
import AcademyPromo from "@site/src/components/AcademyPromo"
7+
import AcademyPromo from "@site/src/components/academy-promo"
88

99
# Security Engine Overview
1010

crowdsec-docs/docs/log_processor/parsers/create.md

Lines changed: 50 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: Creating parsers
44
sidebar_position: 4
55
---
66

7-
import AcademyPromo from '@site/src/components/AcademyPromo';
7+
import AcademyPromo from '@site/src/components/academy-promo';
88

99
## Foreword
1010

@@ -51,10 +51,8 @@ The error message can be useful when one wants to understand why the parser file
5151

5252
![Possible integration](/img/parser_creation/error_message.png)
5353

54-
5554
## Pre-requisites
5655

57-
5856
1. [Create a local test environment](https://doc.crowdsec.net/docs/contributing/contributing_test_env)
5957

6058
2. Clone the hub
@@ -63,7 +61,6 @@ The error message can be useful when one wants to understand why the parser file
6361
git clone https://github.com/crowdsecurity/hub.git
6462
```
6563

66-
6764
## Create our test
6865

6966
From the root of the hub repository :
@@ -81,23 +78,20 @@ From the root of the hub repository :
8178

8279
## Configure our test
8380

84-
8581
Let's add our parser to the test configuration (`.tests/myservice-logs/config.yaml`). He specify that we need syslog-logs parser (because myservice logs are shipped via syslog), and then our custom parser.
8682

8783
```yaml
8884
parsers:
89-
- crowdsecurity/syslog-logs
90-
- ./parsers/s01-parse/crowdsecurity/myservice-logs.yaml
85+
- crowdsecurity/syslog-logs
86+
- ./parsers/s01-parse/crowdsecurity/myservice-logs.yaml
9187
scenarios:
9288
postoverflows:
9389
log_file: myservice-logs.log
9490
log_type: syslog
9591
ignore_parsers: false
96-
9792
```
9893
99-
__note: as our custom parser isn't yet part of the hub, we specify its path relative to the root of the hub directory__
100-
94+
**note: as our custom parser isn't yet part of the hub, we specify its path relative to the root of the hub directory**
10195
10296
## Parser creation : skeleton
10397
@@ -110,29 +104,28 @@ onsuccess: next_stage
110104
name: crowdsecurity/myservice-logs
111105
description: "Parse myservice logs"
112106
grok:
113-
#our grok pattern : capture .*
114-
pattern: ^%{DATA:some_data}$
115-
#the field to which we apply the grok pattern : the log message itself
116-
apply_on: message
107+
#our grok pattern : capture .*
108+
pattern: ^%{DATA:some_data}$
109+
#the field to which we apply the grok pattern : the log message itself
110+
apply_on: message
117111
statics:
118-
- parsed: is_my_service
119-
value: yes
112+
- parsed: is_my_service
113+
value: yes
120114
```
121115
122-
- a [filter](/log_processor/parsers/format.md#filter) : if the expression is `true`, the event will enter the parser, otherwise, it won't
123-
- a [onsuccess](/log_processor/parsers/format.md#onsuccess) : defines what happens when the event was successfully parsed : shall we continue ? shall we move to next stage ? etc.
124-
- a `name` & a `description`
125-
- some [statics](/log_processor/parsers/format.md#statics) that will modify the event
126-
- a `debug` flag that allows to enable local debugging information
127-
- a `grok` pattern to capture some data in logs
128-
116+
- a [filter](/log_processor/parsers/format.md#filter) : if the expression is `true`, the event will enter the parser, otherwise, it won't
117+
- a [onsuccess](/log_processor/parsers/format.md#onsuccess) : defines what happens when the event was successfully parsed : shall we continue ? shall we move to next stage ? etc.
118+
- a `name` & a `description`
119+
- some [statics](/log_processor/parsers/format.md#statics) that will modify the event
120+
- a `debug` flag that allows to enable local debugging information
121+
- a `grok` pattern to capture some data in logs
129122

130123
We can then "test" our parser like this :
131124

132125
```bash
133126
▶ cscli hubtest run myservice-logs
134-
INFO[01-10-2021 12:41:21 PM] Running test 'myservice-logs'
135-
WARN[01-10-2021 12:41:24 PM] Assert file '/home/dev/github/hub/.tests/myservice-logs/parser.assert' is empty, generating assertion:
127+
INFO[01-10-2021 12:41:21 PM] Running test 'myservice-logs'
128+
WARN[01-10-2021 12:41:24 PM] Assert file '/home/dev/github/hub/.tests/myservice-logs/parser.assert' is empty, generating assertion:
136129
137130
len(results) == 2
138131
len(results["s00-raw"]["crowdsecurity/syslog-logs"]) == 3
@@ -153,9 +146,10 @@ Please fill your assert file(s) for test 'myservice-logs', exiting
153146
154147
```
155148

156-
What happened here ?
157-
- Our logs have been processed by syslog-logs parser and our custom parser
158-
- As we have no existing assertion(s), `cscli hubtest` kindly generated some for us
149+
What happened here ?
150+
151+
- Our logs have been processed by syslog-logs parser and our custom parser
152+
- As we have no existing assertion(s), `cscli hubtest` kindly generated some for us
159153

160154
This mostly allows us to ensure that our logs have been processed by our parser, even if it's useless in its current state.
161155
Further inspection can be seen with `cscli hubtest explain` :
@@ -184,10 +178,8 @@ line: Dec 8 06:28:43 mymachine myservice[2806]: accepted connection for user 't
184178

185179
We can see that our log lines were successfully parsed by both syslog-logs and myservice-logs parsers.
186180

187-
188181
## Parser creation : actual parser
189182

190-
191183
Let's modify our parser, `./parsers/s01-parse/crowdsecurity/myservice-logs.yaml` :
192184

193185
```yaml
@@ -197,26 +189,26 @@ name: crowdsecurity/myservice-logs
197189
description: "Parse myservice logs"
198190
#for clarity, we create our pattern syntax beforehand
199191
pattern_syntax:
200-
MYSERVICE_BADPASSWORD: bad password for user '%{USERNAME:user}' from '%{IP:source_ip}' #[1]
201-
MYSERVICE_BADUSER: unknown user '%{USERNAME:user}' from '%{IP:source_ip}' #[1]
192+
MYSERVICE_BADPASSWORD: bad password for user '%{USERNAME:user}' from '%{IP:source_ip}' #[1]
193+
MYSERVICE_BADUSER: unknown user '%{USERNAME:user}' from '%{IP:source_ip}' #[1]
202194
nodes:
203-
#and we use them to parse our two type of logs
204-
- grok:
205-
name: "MYSERVICE_BADPASSWORD" #[2]
206-
apply_on: message
207-
statics:
208-
- meta: log_type #[3]
209-
value: myservice_failed_auth
210-
- meta: log_subtype
211-
value: myservice_bad_password
212-
- grok:
213-
name: "MYSERVICE_BADUSER" #[2]
214-
apply_on: message
215-
statics:
216-
- meta: log_type #[3]
217-
value: myservice_failed_auth
218-
- meta: log_subtype
219-
value: myservice_bad_user
195+
#and we use them to parse our two type of logs
196+
- grok:
197+
name: "MYSERVICE_BADPASSWORD" #[2]
198+
apply_on: message
199+
statics:
200+
- meta: log_type #[3]
201+
value: myservice_failed_auth
202+
- meta: log_subtype
203+
value: myservice_bad_password
204+
- grok:
205+
name: "MYSERVICE_BADUSER" #[2]
206+
apply_on: message
207+
statics:
208+
- meta: log_type #[3]
209+
value: myservice_failed_auth
210+
- meta: log_subtype
211+
value: myservice_bad_user
220212
statics:
221213
- meta: service #[3]
222214
value: myservice
@@ -227,19 +219,18 @@ statics:
227219
```
228220

229221
Various changes have been made here :
230-
- We created to patterns to capture the two relevant type of log lines, Using an [online grok debugger](https://grokdebug.herokuapp.com/) or an [online regex debugger](https://www.debuggex.com/) [2]
231-
)
232-
- We keep track of the username and the source_ip (Please note that setting the source_ip in `evt.Meta.source_ip` and `evt.Parsed.source_ip` is important [1])
233-
- We setup various [statics](/log_processor/parsers/format.md#statics) information to classify the log type [3]
234-
235222

223+
- We created to patterns to capture the two relevant type of log lines, Using an [online grok debugger](https://grokdebug.herokuapp.com/) or an [online regex debugger](https://www.debuggex.com/) [2]
224+
)
225+
- We keep track of the username and the source_ip (Please note that setting the source_ip in `evt.Meta.source_ip` and `evt.Parsed.source_ip` is important [1])
226+
- We setup various [statics](/log_processor/parsers/format.md#statics) information to classify the log type [3]
236227

237228
Let's run out tests again :
238229

239230
```bash {13-20}
240-
▶ cscli hubtest run myservice-logs
241-
INFO[01-10-2021 12:49:56 PM] Running test 'myservice-logs'
242-
WARN[01-10-2021 12:49:59 PM] Assert file '/home/dev/github/hub/.tests/myservice-logs/parser.assert' is empty, generating assertion:
231+
▶ cscli hubtest run myservice-logs
232+
INFO[01-10-2021 12:49:56 PM] Running test 'myservice-logs'
233+
WARN[01-10-2021 12:49:59 PM] Assert file '/home/dev/github/hub/.tests/myservice-logs/parser.assert' is empty, generating assertion:
243234
244235
len(results) == 2
245236
len(results["s00-raw"]["crowdsecurity/syslog-logs"]) == 3
@@ -291,8 +282,7 @@ line: Dec 8 06:28:43 mymachine myservice[2806]: accepted connection for user 't
291282
└ 🔴 crowdsecurity/myservice-logs
292283
```
293284

294-
__note: we can see that our log line `accepted connection for user 'toto' from '192.168.1.1'` wasn't parsed by `crowdsecurity/myservice-logs` as we have no pattern for it__
295-
285+
**note: we can see that our log line `accepted connection for user 'toto' from '192.168.1.1'` wasn't parsed by `crowdsecurity/myservice-logs` as we have no pattern for it**
296286

297287
## Closing word
298288

@@ -309,4 +299,4 @@ See as well [this blog article](https://crowdsec.net/blog/how-to-write-crowdsec-
309299
title="More ways to learn"
310300
course="writing-parsers-and-scenarios"
311301
utm="?utm_source=docs&utm_medium=banner&utm_campaign=parser-page&utm_id=academydocs"
312-
/>
302+
/>

crowdsec-docs/docs/log_processor/scenarios/create.md

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: Creating scenarios
44
sidebar_position: 4
55
---
66

7-
import AcademyPromo from '@site/src/components/AcademyPromo';
7+
import AcademyPromo from '@site/src/components/academy-promo';
88

99
:::caution
1010

@@ -77,13 +77,13 @@ Let's add our parser and scenario to the test configuration (`.tests/myservice-b
7777

7878
```yaml
7979
parsers:
80-
- crowdsecurity/syslog-logs
81-
- crowdsecurity/dateparse-enrich
82-
- ./parsers/s01-parse/crowdsecurity/myservice-logs.yaml
80+
- crowdsecurity/syslog-logs
81+
- crowdsecurity/dateparse-enrich
82+
- ./parsers/s01-parse/crowdsecurity/myservice-logs.yaml
8383
scenarios:
84-
- ./scenarios/crowdsecurity/myservice-bf.yaml
84+
- ./scenarios/crowdsecurity/myservice-bf.yaml
8585
postoverflows:
86-
- ""
86+
- ""
8787
log_file: myservice-bf.log
8888
log_type: syslog
8989
ignore_parsers: true
@@ -126,16 +126,16 @@ We filter on `evt.Meta.log_type == 'myservice_failed_auth'` because in the parse
126126

127127
We have the following fields:
128128

129-
- a [type](/log_processor/scenarios/format.md#type): the type of bucket to use (trigger or leaky).
130-
- a [name](/log_processor/scenarios/format.md#name)
131-
- a [description](/log_processor/scenarios/format.md#description)
132-
- a [filter](/log_processor/scenarios/format.md#type): the filter to apply on events to be filled in this bucket.
133-
- a [leakspeed](/log_processor/scenarios/format.md#leakspeed)
134-
- a [capacity](/log_processor/scenarios/format.md#capacity): the number of events in the bucket before it overflows.
135-
- a [groupby](/log_processor/scenarios/format.md#groupby): a field from the event to partition the bucket. It is often the `source_ip` of the event.
136-
- a [blackhole](/log_processor/scenarios/format.md#blackhole): the number of minute to not retrigger this scenario for the same `groupby` field.
137-
- a [reprocess](/log_processor/scenarios/format.md#reprocess): ingest the alert in crowdsec for further processing.
138-
- some [labels](/log_processor/scenarios/format.md#labels): Some labels are mandatory and the scenario will not be validated by the Hub if they are missing. Don't forget to set `remediation: true` if you want the IP to be blocked by bouncers.
129+
- a [type](/log_processor/scenarios/format.md#type): the type of bucket to use (trigger or leaky).
130+
- a [name](/log_processor/scenarios/format.md#name)
131+
- a [description](/log_processor/scenarios/format.md#description)
132+
- a [filter](/log_processor/scenarios/format.md#type): the filter to apply on events to be filled in this bucket.
133+
- a [leakspeed](/log_processor/scenarios/format.md#leakspeed)
134+
- a [capacity](/log_processor/scenarios/format.md#capacity): the number of events in the bucket before it overflows.
135+
- a [groupby](/log_processor/scenarios/format.md#groupby): a field from the event to partition the bucket. It is often the `source_ip` of the event.
136+
- a [blackhole](/log_processor/scenarios/format.md#blackhole): the number of minute to not retrigger this scenario for the same `groupby` field.
137+
- a [reprocess](/log_processor/scenarios/format.md#reprocess): ingest the alert in crowdsec for further processing.
138+
- some [labels](/log_processor/scenarios/format.md#labels): Some labels are mandatory and the scenario will not be validated by the Hub if they are missing. Don't forget to set `remediation: true` if you want the IP to be blocked by bouncers.
139139

140140
We can then "test" our scenario like this :
141141

@@ -171,8 +171,8 @@ Please fill your assert file(s) for test 'myservice-bf', exiting
171171

172172
What happened here ?
173173

174-
- The scenario has been triggered and is generating some assertion (for functional test)
175-
- In production environment, an alert would have been send to the CrowdSec Local API.
174+
- The scenario has been triggered and is generating some assertion (for functional test)
175+
- In production environment, an alert would have been send to the CrowdSec Local API.
176176

177177
We can again understand more of what is going on thanks to `cscli hubtest explain` :
178178

0 commit comments

Comments
 (0)