enhance: log processors updates

Author: LaurenceJJones

crowdsec-docs/docs/getting_started/intro.md

@@ -18,6 +18,10 @@ See [Version Matrix](/getting_started/versions_matrix.md) for a list of supporte
 
 ## Why is my Security Engine classed as a "log processor" within the console?
 
-The `Security Engine` comes compiled with a number of optional features that can be enabled or disabled at runtime. One of these features is called the "LAPI" (Local API). If this feature is disabled at runtime, the Security Engine will be classed as a "log processor" within the console as it will only be able to process logs and forward the alerts to the local API you define in the configuration.
+The `Security Engine` comes compiled with a number of core components that can be enabled or disabled at runtime.
+
+One of these features is called the "LAPI" (Local API). If this feature is disabled at runtime, the Security Engine will be classed as a "log processor" within the console as it will only be able to process logs and forward the alerts to the local API you define in the configuration.
+
+Read more about the [Log Processor](log_processor/intro.mdx) and the [Local API](local_api/intro.md).
 
 Most commonly this is the case when you are running in a distributed setup, where you have a central server that is running the LAPI and a number of remote servers that are running the "Log processors".

crowdsec-docs/docs/intro.mdx

@@ -49,9 +49,9 @@ In addition to the core "detect and react" mechanism, CrowdSec is committed to s
 
 Under the hood, the Security Engine has various components:
 
--   The Log Processor is in charge of detection: it analyzes logs from [various data sources](data_sources/intro) or [HTTP requests](appsec/intro) from web servers.
+-   The [Log Processor](log_processor/intro.mdx) is in charge of detection: it analyzes logs from [various data sources](data_sources/intro) or [HTTP requests](appsec/intro) from web servers.
 -   The [Appsec](appsec/intro) feature is part of the Log Processor and filters HTTP Requests from the compatible web servers.
--   The [Local API](/local_api/intro.md) acts as a middle man:
+-   The [Local API](local_api/intro.md) acts as a middle man:
     -   Between the [Log Processors](/docs/data_sources/intro) and the [Remediation Components](/u/bouncers/intro) which are in charge of enforcing decisions.
     -   And with the [Central API](/central_api/intro.md) to share alerts and receive blocklists.
 -   The [Remediation Components](/u/bouncers/intro) - also known as bouncers - block malicious IPs at your chosen level—whether via IpTables, firewalls, web servers, or reverse proxies. [See the full list on our CrowdSec Hub.](https://app.crowdsec.net/hub/remediation-components)

crowdsec-docs/docs/log_processor/alert_context/intro.md

@@ -0,0 +1,47 @@
+---
+id: intro
+title: Alert Context
+---
+
+## Introduction
+
+As the [Log Processor](log_processor/intro.mdx) processes logs, it will detect patterns of interest known as [Scenarios](log_processor/scenarios/introduction.mdx). When a scenario is detected, an alert is generated and sent to the [Local API](local_api/intro.md) (LAPI) for evaluation.
+
+When the alert is generated you can define additional Alert Context that can be sent along with the alert to give you context about the alert. This can be useful when you host multiple applications on the same server and you want to know which application generated the alert.
+
+### Format
+
+The format of Alert Context are key value pairs that are sent along with the alert. When you install some [Collections](log_processor/collections/intro.md) you will see that they come with Alert Context pre-configured.
+
+For example if you install the `crowdsecurity/nginx` collection you will see that the `http_base` context is added:
+
+```yaml
+#this context file is intended to provide minimal and useful information about HTTP scenarios.
+context:
+  target_uri:
+  - evt.Meta.http_path
+  user_agent:
+  - evt.Meta.http_user_agent
+  method:
+  - evt.Meta.http_verb
+  status:
+    - evt.Meta.http_status
+```
+
+Contexts are stored within the `contexts` directory within the root of the `config` directory, you can see the directory based on your OS [here](/u/troubleshooting/security_engine#where-is-configuration-stored).
+
+:::info
+As an example the default directory for linux is `/etc/crowdsec/` so the `contexts` directory would be `/etc/crowdsec/contexts/`
+:::
+
+Here a quick breakdown of the context file:
+
+- `context` : This is the root key of the context file.
+- `target_uri` : This is the key that will be used as the "name" of the context.
+- `evt.Meta.http_path` : This is the expression that will be evaluated to get the value of the context. In this case it will be the `http_path` field from the event.
+
+The next key value pair would be `user_agent` and so on.
+
+## Next Steps?
+
+We have written a full guide on Alert Context that you can find [here](/u/user_guides/alert_context). This guide will show you how to create your own Alert Context and how to use it within your scenarios.

crowdsec-docs/docs/log_processor/intro.mdx

@@ -25,9 +25,11 @@ The Log Processor is an internal core component of the Security Engine in charge
 
 Once a pattern of interest is detected, the Log Processor will push alerts to the Local API (LAPI) for alert/decisions to be stored within the database.
 
+All subcategories below are related to the Log Processor and its functionalities. If you are utilizing a multi server architecture, you will only need to configure the functionality that you want to use on the Log Processor.
+
 ## Data Sources
 
-Data Sources are individual modules that can be loaded at runtime by the Log Processor to read logs from various sources. To define a Data Source, you will need to create an acquisition configuration file.
+Data Sources are individual modules that can be loaded at runtime by the Log Processor to read logs from various sources. To use a Data Source, you will need to create an acquisition configuration file.
 
 ### Acquistions
 
@@ -64,40 +66,24 @@ The parsing pipeline is broken down into multiple stages:
 - `s01-parse` : This is the second stage responsible for extracting relevant information from the normalized logs based on the application type to be used by `s02-enrich` and the [Scenarios](log_processor/scenarios/introduction.mdx).
 - `s02-enrich` : This is the third stage responsible for enriching the extracted information with additional context such as GEOIP, ASN etc.
 
-:::info
-We will give a breif overview of each stage, however, for most users this documentation is not required to get started with CrowdSec but can be used to understand the inner workings of the Log Processor.
-:::
-
-#### `s00-raw`
-
-This stage is responsible for normalizing logs from various [Data Sources](log_processor/data_sources/introduction.md) into a predictable format for `s01-parse` and `s02-enrich` to work on.
-
-For example if you have a `syslog` Data Source and a `container` Data Source writing the same application log lines you wouldnt want `s01-parse` to handle this logic twice, since `s00-raw` can normalize the logs into a predictable format. 
-
-For most instances we have already created these `s00-raw` parsers for you are available to view on the [Hub](https://hub.crowdsec.net/).
-
-#### `s01-parse`
-
-The stage is responsible for extracting relevant information from the normalized logs based on the application type.
-
-The application type is defined in different ways based on the Data Source. Please refer to the [Data Sources](log_processor/data_sources/introduction.md) documentation for more information.
+You can see more information on Parsers in the [documentation](log_processor/parsers/introduction.mdx).
 
-We list all available applications we support on the [Hub](https://hub.crowdsec.net/) and within the readme of the collection our users provide an example Acquisition configuration.
+### Scenarios
 
-#### `s02-enrich`
+Scenarios are the patterns of interest that the Log Processor is monitoring for. When a pattern of interest is detected, the Log Processor will push alerts to the Local API (LAPI) for alert/decisions to be stored within the database.
 
-The aim of this stage is to enrich the extracted information with additional context such as GEOIP, ASN etc.
+The patterns can be as simple as tracking the number of failed login attempts or as complex as tracking logging in from multiple countries within a short period of time which can be a indicator of a compromised account or VPN usage.
 
-However, the stage can also be used to perform whitelist checks, however, we have dedicated documentation for this [here](log_processor/whitelist/introduction.md).
+The community provides a number of scenarios on the [Hub](https://hub.crowdsec.net/) that you can install and use. If you would like to create your own, see the [Scenarios](log_processor/scenarios/introduction.mdx) documentation.
 
-Currently we have a few enrichers available on the [Hub](https://hub.crowdsec.net/), that are installed by default so you dont need to worry about this stage unless you want to create your own.
+### whitelists
 
-For more information on Parsers, see the [Parsers](log_processor/parsers/introduction.mdx) documentation.
+Whitelists are used to exclude certain events from being processed by the Log Processor. For example, you may want to exclude certain IP addresses from being processed by the Log Processor.
 
-### Scenarios
+You can see more information on Whitelists in the [documentation](log_processor/whitelist/introduction.md).
 
-Scenarios are the patterns of interest that the Log Processor is monitoring for. When a pattern of interest is detected, the Log Processor will push alerts to the Local API (LAPI) for alert/decisions to be stored within the database.
+### Alert Context
 
-The patterns can be as simple as tracking the number of failed login attempts or as complex as tracking logging in from multiple countries within a short period of time which can be a indicator of a compromised account or VPN usage.
+Alert Context is additional context that can sent with an alert to the LAPI. This context can be shown locally via `cscli` or within the [CrowdSec Console](https://app.crowdsec.net/signup) if you opt in to share context when you enroll your instance.
 
-The community provides a number of scenarios on the [Hub](https://hub.crowdsec.net/) that you can install and use. If you would like to create your own, see the [Scenarios](log_processor/scenarios/introduction.mdx) documentation.
+You can read more about Alert Context in the [documentation](log_processor/alert_context/intro.md).

crowdsec-docs/docs/log_processor/parsers/introduction.mdx

@@ -34,13 +34,35 @@ Parsers are organized into stages to allow pipelines and branching in parsing. A
     </div>
 </div>
 
-Each parser can add, change or even delete data from the event. The current approach is: 
- - `s00-raw`: takes care of the overall log structure (ie. extract log lines from JSON blob, [parse syslog protocol](https://hub.crowdsec.net/author/crowdsecurity/configurations/syslog-logs) info)
- - `s01-parse`: parses the *actual* log line ([ssh](https://hub.crowdsec.net/author/crowdsecurity/configurations/sshd-logs), [nginx](https://hub.crowdsec.net/author/crowdsecurity/configurations/nginx-logs) etc.)
- - `s02-enrich`: does some post processing, such as [geoip-enrich](https://hub.crowdsec.net/author/crowdsecurity/configurations/geoip-enrich) or post-parsing of [http events to provide more context](https://hub.crowdsec.net/author/crowdsecurity/configurations/http-logs)
+The parsing pipeline is broken down into multiple stages:
 
+- `s00-raw` : This is the first stage which aims to normalize the logs from various [Data Sources](log_processor/data_sources/introduction.md) into a predictable format for `s01-parse` and `s02-enrich` to work on.
+- `s01-parse` : This is the second stage responsible for extracting relevant information from the normalized logs based on the application type to be used by `s02-enrich` and the [Scenarios](log_processor/scenarios/introduction.mdx).
+- `s02-enrich` : This is the third stage responsible for enriching the extracted information with additional context such as GEOIP, ASN etc.
 
-Once an event has successfully exited the parsing pipeline, it is ready to be matched against scenarios. As you might expect, each parser relies on the information that is parsed during previous stages.
+### `s00-raw`
+
+This stage is responsible for normalizing logs from various [Data Sources](log_processor/data_sources/introduction.md) into a predictable format for `s01-parse` and `s02-enrich` to work on.
+
+For example if you have a `syslog` Data Source and a `container` Data Source writing the same application log lines you wouldnt want `s01-parse` to handle this logic twice, since `s00-raw` can normalize the logs into a predictable format. 
+
+For most instances we have already created these `s00-raw` parsers for you are available to view on the [Hub](https://hub.crowdsec.net/).
+
+### `s01-parse`
+
+The stage is responsible for extracting relevant information from the normalized logs based on the application type.
+
+The application type is defined in different ways based on the Data Source. Please refer to the [Data Sources](log_processor/data_sources/introduction.md) documentation for more information.
+
+We list all available applications we support on the [Hub](https://hub.crowdsec.net/) and within the readme of the collection our users provide an example Acquisition configuration.
+
+### `s02-enrich`
+
+The aim of this stage is to enrich the extracted information with additional context such as GEOIP, ASN etc.
+
+However, the stage can also be used to perform whitelist checks, however, we have dedicated documentation for this [here](log_processor/whitelist/introduction.md).
+
+Currently we have a few enrichers available on the [Hub](https://hub.crowdsec.net/), that are installed by default so you dont need to worry about this stage unless you want to create your own.
 
 ## Postoverflows
 

crowdsec-docs/sidebars.js

@@ -1,24 +1,3 @@
-/**
- * Creating a sidebar enables you to:
- - create an ordered group of docs
- - render a sidebar for each doc of that group
- - provide next/previous navigation
-
- The sidebars can be generated from the filesystem, or explicitly defined here.
-
- Create as many sidebars as you want.
-
-
-
- - regrouper les choses oriente utilisateur (genre gestion des decisions etc.) dans la partie user guide
- - avoir un item 'profil' de premier level (ou on parle des notifs/plugins)
- - regrouper central et local api (? hes:bof)
- - regrouper : dashboard et observability
- - dupliquer des liens vers le format de config des log_processor/parsers/log_processor/scenarios/... entre la partie "log_processor/parsers" et la partie "configuration files format"
-
-
- */
-
 module.exports = {
     // By default, Docusaurus generates a sidebar from the docs folder structure
     //tutorialSidebar: [{type: 'autogenerated', dirName: '.'}],
@@ -205,6 +184,15 @@ module.exports = {
                         },
                     ],
                 },
+                {
+                    type: "category",
+                    label: "Alert Context",
+                    link: {
+                        type: "doc",
+                        id: "log_processor/alert_context/intro",
+                    },
+                    items: [],
+                },
             ],
         },
         {