🐛 fix start up config error for appsec and review doc for appsec tls (#300)

* 🐛 fix start up config error for appsec

* :doc: add documentation on appsec variables and missing conf parameter

* 🍱 fix lint

* 🍱 fix lint

* 🍱 fix lint

* 🍱 fix after lot of tests

* update exemple tls with new variables tested

* fix exemple appsec with release and not localplugin

---------

Co-authored-by: mhx <mathieu@hanotaux.fr>

Author: maxlerebourg

Makefile

@@ -99,7 +99,7 @@ clean_all_docker:
 	docker compose -f examples/redis-cache/docker-compose.yml down --remove-orphans
 	docker compose -f examples/trusted-ips/docker-compose.yml down --remove-orphans
 	docker compose -f examples/tls-auth/docker-compose.yml down --remove-orphans
-	docker compose -f examples/appsec-enabled/docker-compose.appsec-enabled.yml down --remove-orphans
+	docker compose -f examples/appsec-enabled/docker-compose.yml down --remove-orphans
 	docker compose -f examples/captcha/docker-compose.yml down --remove-orphans
 	docker compose -f examples/custom-captcha/docker-compose.yml down --remove-orphans
 	docker compose -f examples/custom-ban-page/docker-compose.yml down --remove-orphans

README.md

@@ -310,17 +310,16 @@ make run
 ### Note
 
 > [!IMPORTANT]  
-> Some of the behaviours and configuration parameters are shared globally across *all* crowdsec middlewares even if you declare different middlewares with different settings. 
+> Some of the behaviours and configuration parameters are shared globally across _all_ crowdsec middlewares even if you declare different middlewares with different settings.
 >
 > **Cache is shared by all services**: This means if an IP is banned, all services which are protected by an instance of the plugin will deny requests from that IP
 >
 > If you define different caches for different middlewares, only the first one to be instantiated will be bound to the crowdsec stream.
 >
-> Overall, this middleware is designed in such a way that **only one instance of the plugin is *possible*.** You can have multiple crowdsec middlewares in the same cluster, the key parameters must be aligned (MetricsUpdateIntervalSeconds, CrowdsecMode, CrowdsecAppsecEnabled, etc.)
+> Overall, this middleware is designed in such a way that **only one instance of the plugin is _possible_.** You can have multiple crowdsec middlewares in the same cluster, the key parameters must be aligned (MetricsUpdateIntervalSeconds, CrowdsecMode, CrowdsecAppsecEnabled, etc.)
 
 > [!WARNING]  
-> **Appsec maximum body limit is defaulted to 10MB**
-> *Be careful when you upgrade to >1.4.x*
+> **Appsec maximum body limit is defaulted to 10MB** > _Be careful when you upgrade to >1.4.x_
 
 ### Variables
 
@@ -351,7 +350,18 @@ make run
 - CrowdsecAppsecHost
   - string
   - default: "crowdsec:7422"
-  - Crowdsec Appsec Server available on which host and port. The scheme will be handled by the CrowdsecLapiScheme var.
+  - Crowdsec Appsec Server available on which host and port.
+- CrowdsecAppsecTlsInsecureVerify
+  - bool
+  - default: false
+  - Disable verification of certificate presented by Appsec
+- CrowdsecAppsecTlsCertificateAuthority
+  - string
+  - default: ""
+  - PEM-encoded Certificate Authority of Appsec
+- CrowdsecAppsecScheme
+  - string
+  - default: value of `CrowdsecLapiScheme`, expected values are: `http`, `https`
 - CrowdsecAppsecPath
   - string
   - default: "/"
@@ -368,6 +378,10 @@ make run
   - int64
   - default: 10485760 (= 10MB)
   - Transmit only the first number of bytes to Crowdsec Appsec Server.
+- CrowdsecAppsecKey
+  - string
+  - default: value of `CrowdsecLapiKey`
+  - Crowdsec AppSec key for the bouncer.
 - CrowdsecLapiScheme
   - string
   - default: `http`, expected values are: `http`, `https`
@@ -614,7 +628,7 @@ http:
 
 #### Fill variable with value of file
 
-`CrowdsecLapiTlsCertificateBouncerKey`, `CrowdsecLapiTlsCertificateBouncer`, `CrowdsecLapiTlsCertificateAuthority`, `CrowdsecCapiMachineId`, `CrowdsecCapiPassword`, `CrowdsecLapiKey`, `CaptchaSiteKey`, `CaptchaSecretKey` and `RedisCachePassword` can be provided with the content as raw or through a file path that Traefik can read.  
+`CrowdsecLapiTlsCertificateBouncerKey`, `CrowdsecLapiTlsCertificateBouncer`, `CrowdsecLapiTlsCertificateAuthority`, `CrowdsecAppsecTlsCertificateAuthority`, `CrowdsecCapiMachineId`, `CrowdsecCapiPassword`, `CrowdsecLapiKey`, `CrowdsecAppsecKey`, `CaptchaSiteKey`, `CaptchaSecretKey` and `RedisCachePassword` can be provided with the content as raw or through a file path that Traefik can read.  
 The file variable will be used as preference if both content and file are provided for the same variable.
 
 Format is:
@@ -677,6 +691,13 @@ Set the `crowdsecLapiScheme` to https.
 Crowdsec must be listening in HTTPS for this to work.
 Please see the [tls-auth example](https://github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/blob/main/examples/tls-auth/README.md) or the official documentation: [docs.crowdsec.net/docs/local_api/tls_auth/](https://docs.crowdsec.net/docs/local_api/tls_auth/)
 
+#### Use HTTPS to communicate with the Appsec
+
+To communicate with the Appsec in HTTPS you need to either accept any certificates by setting the `crowdsecAppsecTLSInsecureVerify` to true or add the CA used by the server certificate of Crowdsec using `crowdsecAppsecTLSCertificateAuthority` or `crowdsecAppsecTLSCertificateAuthorityFile`.
+Set the `crowdsecAppsecScheme` to https.
+
+Currently AppSec does not support mTLS authentication for the AppSec Component.
+
 #### Manually add an IP to the blocklist (for testing purposes)
 
 ```bash

bouncer.go

@@ -134,17 +134,23 @@ func New(_ context.Context, next http.Handler, config *configuration.Config, nam
 
 	serverChecker, _ := ip.NewChecker(log, config.ForwardedHeadersTrustedIPs)
 	clientChecker, _ := ip.NewChecker(log, config.ClientTrustedIPs)
-	tlsAppsecConfig, err := configuration.GetTLSConfigCrowdsec(config, log, true)
-	if err != nil {
-		log.Error("New:getTLSConfigCrowdsec fail to get tlsAppsecConfig " + err.Error())
-		return nil, err
-	}
-	apiAppsecKey, errAppsecKey := configuration.GetVariable(config, "CrowdsecAppsecKey")
-	if errAppsecKey != nil && len(tlsAppsecConfig.Certificates) == 0 {
-		log.Error("New:crowdsecLapiKey fail to get CrowdsecAppsecKey and no client certificate setup " + errAppsecKey.Error())
-		return nil, errAppsecKey
+
+	var tlsAppsecConfig *tls.Config
+	if config.CrowdsecAppsecEnabled {
+		tlsAppsecConfig, err = configuration.GetTLSConfigCrowdsec(config, log, true)
+		if config.CrowdsecAppsecScheme == "" {
+			config.CrowdsecAppsecScheme = config.CrowdsecLapiScheme
+		}
+		if err != nil {
+			log.Error("New:getTLSConfigCrowdsec fail to get tlsAppsecConfig " + err.Error())
+			return nil, err
+		}
+		apiAppsecKey, errAppsecKey := configuration.GetVariable(config, "CrowdsecAppsecKey")
+		if errAppsecKey != nil && len(tlsAppsecConfig.Certificates) == 0 {
+			log.Info("New:crowdsecLapiKey fail to get CrowdsecAppsecKey and no client certificate setup " + errAppsecKey.Error())
+		}
+		config.CrowdsecAppsecKey = apiAppsecKey
 	}
-	config.CrowdsecAppsecKey = apiAppsecKey
 
 	var tlsConfig *tls.Config
 	crowdsecStreamRoute := ""
@@ -155,7 +161,6 @@ func New(_ context.Context, next http.Handler, config *configuration.Config, nam
 		config.CrowdsecLapiScheme = configuration.HTTPS
 		config.CrowdsecLapiHost = crowdsecCapiHost
 		config.CrowdsecLapiPath = "/"
-		config.CrowdsecAppsecEnabled = config.CrowdsecAppsecEnabled && config.CrowdsecAppsecScheme != ""
 		config.UpdateIntervalSeconds = 7200 // 2 hours
 		crowdsecStreamRoute = crowdsecCapiStreamRoute
 		crowdsecHeader = crowdsecCapiHeader
@@ -173,6 +178,9 @@ func New(_ context.Context, next http.Handler, config *configuration.Config, nam
 			return nil, errKey
 		}
 		config.CrowdsecLapiKey = apiKey
+		if config.CrowdsecAppsecKey == "" {
+			config.CrowdsecAppsecKey = apiKey
+		}
 	}
 
 	var banTemplate *htmltemplate.Template
@@ -374,6 +382,9 @@ func (bouncer *Bouncer) ServeHTTP(rw http.ResponseWriter, req *http.Request) {
 		}
 	} else {
 		value, err := handleNoStreamCache(bouncer, remoteIP)
+		if err != nil {
+			bouncer.log.Debug("handleNoStreamCache:crowdsecQuery " + err.Error())
+		}
 		if value == cache.NoBannedValue {
 			bouncer.handleNextServeHTTP(rw, req, remoteIP)
 		} else {

docker-compose.local.yml

@@ -1,6 +1,6 @@
 services:
   traefik:
-    image: "traefik:v3.0.0"
+    image: "traefik:v3.5.0"
     container_name: "traefik"
     restart: unless-stopped
     command:
@@ -16,8 +16,8 @@ services:
     volumes:
       - /var/run/docker.sock:/var/run/docker.sock:ro
       - logs-local:/var/log/traefik
-      - './ban.html:/ban.html:ro'
-      - './captcha.html:/captcha.html:ro'
+      - "./ban.html:/ban.html:ro"
+      - "./captcha.html:/captcha.html:ro"
       - ./:/plugins-local/src/github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
     ports:
       - 8000:80
@@ -52,6 +52,7 @@ services:
       - "traefik.http.middlewares.crowdsec.plugin.bouncer.crowdsecappsecenabled=true"
       - "traefik.http.middlewares.crowdsec.plugin.bouncer.crowdsecmode=stream"
       - "traefik.http.middlewares.crowdsec.plugin.bouncer.crowdseclapikey=40796d93c2958f9e58345514e67740e5="
+      - "traefik.http.middlewares.crowdsec.plugin.bouncer.ForwardedHeadersTrustedIPs=172.21.0.1/8"
 
   bar2:
     image: traefik/whoami

…nabled/docker-compose.appsec-enabled.yml …amples/appsec-enabled/docker-compose.ymlexamples/appsec-enabled/docker-compose.appsec-enabled.yml renamed to examples/appsec-enabled/docker-compose.yml examples/appsec-enabled/docker-compose.appsec-enabled.yml renamed to examples/appsec-enabled/docker-compose.yml

@@ -1,6 +1,6 @@
 services:
   traefik:
-    image: "traefik:v3.0.0"
+    image: "traefik:v3.5.0"
     container_name: "traefik"
     restart: unless-stopped
     command:
@@ -13,7 +13,7 @@ services:
       - "--entrypoints.web.address=:80"
 
       - "--experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
-      - "--experimental.plugins.bouncer.version=v1.3.0"
+      - "--experimental.plugins.bouncer.version=v1.5.0"
       # - "--experimental.localplugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
     volumes:
       - /var/run/docker.sock:/var/run/docker.sock:ro

examples/tls-auth/README.md

@@ -1,7 +1,9 @@
 # Example
+
 ## Using https communication and tls authentication with Crowdsec
 
 ##### Summary
+
 This example demonstrates the use of https between the Traefik plugin and the Crowdsec LAPI.
 
 It is possible to communicate with the LAPI in https and still authenticate with API key.
@@ -17,7 +19,9 @@ In that case the setting **crowdsecLapiTLSInsecureVerify** must be set to true.
 It is recommended to validate the certificate presented by Crowdsec LAPI using the Certificate Authority which created it.
 
 You can provide the Certificate Authority using:
-* A file path readable by Traefik
+
+- A file path readable by Traefik
+
 ```yaml
 http:
   middlewares:
@@ -26,25 +30,29 @@ http:
         bouncer:
           crowdsecLapiTlsCertificateAuthorityFile: /etc/traefik/certs/crowdsecCA.pem
 ```
-* The PEM encoded certificate as a text variable
+
+- The PEM encoded certificate as a text variable
 
 In the static file configuration of Traefik
+
 ```yaml
 http:
   middlewares:
     crowdsec:
       plugin:
         bouncer:
           crowdsecLapiTlsCertificateAuthority: |-
-              -----BEGIN CERTIFICATE-----
-              MIIEBzCCAu+gAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwgZQxCzAJBgNVBAYTAlVT
-              MRAwDgYDVQQHDAdTZWF0dGxlMRMwEQYDVQQIDApXYXNoaW5ndG9uMSIwIAYDVQQK
-              ...
-              C6qNieSwcvWL7C03ri0DefTQMY54r5wP33QU5hJ71JoaZI3YTeT0Nf+NRL4hM++w
-              Q0veeNzBQXg1f/JxfeA39IDIX1kiCf71tGlT
-              -----END CERTIFICATE-----
+            -----BEGIN CERTIFICATE-----
+            MIIEBzCCAu+gAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwgZQxCzAJBgNVBAYTAlVT
+            MRAwDgYDVQQHDAdTZWF0dGxlMRMwEQYDVQQIDApXYXNoaW5ndG9uMSIwIAYDVQQK
+            ...
+            C6qNieSwcvWL7C03ri0DefTQMY54r5wP33QU5hJ71JoaZI3YTeT0Nf+NRL4hM++w
+            Q0veeNzBQXg1f/JxfeA39IDIX1kiCf71tGlT
+            -----END CERTIFICATE-----
 ```
+
 In a dynamic configuration of a provider (ex docker) as a Label
+
 ```yaml
 services:
   whoami-foo:
@@ -71,26 +79,34 @@ The service `whoami-foo` will authenticate with an **API key** over HTTPS after
 The service `whoami-bar` will authenticate with a **client certificate** signed by the CA.
 
 Access to a route that communicate via https and authenticate with API-key:
+
 ```
 curl http://localhost:8000/foo
 ```
+
 Access to a route that communicate via https and authenticate with a client certificate:
+
 ```
 curl http://localhost:8000/bar
 ```
+
 Access to the traefik dashboard
+
 ```
 curl http://localhost:8080/dashboard/#/
 ```
 
 To play the demo environnement run:
+
 ```bash
 make run_tlsauth
 ```
 
 Note:
-> Traefik need to be restarted if certificates are regenerated after his launch
+
+> Traefik need to be restarted if certificates are regenerated after his launch, crowdsec also
 
 ## Separate LAPI and Appsec HTTP/S config
+
 To separate TLS config for LAPI and Appsec, you can use all the TLS LAPI variable beginning with `CrowdsecLapi...` into `CrowdsecAppsec...`.
 Don't forget to set `CrowdsecAppsecScheme: HTTP` or `HTTPS` to trigger the separate setup.

examples/tls-auth/config/acquis.yaml

@@ -2,3 +2,12 @@ filenames:
  - /var/log/traefik/access.log
 labels:
   type: traefik
+---
+listen_addr: 0.0.0.0:7422
+appsec_config: crowdsecurity/virtual-patching
+name: myAppSecComponent
+source: appsec
+labels:
+  type: appsec
+cert_file: /etc/crowdsec/certs/server.pem
+key_file: /etc/crowdsec/certs/server-key.pem