feat: Update WAF install and configuration steps

content/includes/waf/install-post-checks.md

@@ -60,4 +60,13 @@ Verify that Enforcement functionality is working by checking the following reque
 
 ```shell
 curl "localhost/<script>"
-```
+```
+
+If you notice problems, there are ways to remediate them based on the context:
+
+| Description             | Solution  |
+| ----------------------- | --------  |
+| *NGINX is not running or F5 WAF for NGINX does not behave as expected* | Review warning or error messages within [the log files]({{< ref "/waf/logging/logs-overview.md" >}}) |
+| *unknown directive app_protect_xxx error message* | Ensure F5 WAF for NGINX is [loaded as a module](#update-configuration-files) in the main context of NGINX configuration. |
+| *Too many open files error message* | Increase the maximum amount of open files with the [worker_rlimit_nofile](https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_nofile) directive. |
+| *setrlimit ... failed (Permission denied) error message* | Increase the limit by by running the following command as root: `setsebool -P httpd_setrlimit 1` |

content/includes/waf/table-policy-features.md

@@ -7,6 +7,7 @@
 | ----------------------------------- | ----------- |
 | [Allowed methods]({{< ref "/waf/policies/allowed-methods.md" >}}) | Checks allowed HTTP methods. By default, all the standard HTTP methods are allowed. |
 | [Attack signatures]({{< ref "/waf/policies/attack-signatures.md" >}}) | The default policy covers the OWASP top 10 attack patterns. Specific signature sets can be added or disabled. |
+| [Bot signatures]({{< ref "/waf/policies/bot-signatures.md" >}}) | Bot signatures and headers can be inspected to authenticate the identity of a client making a request. |
 | [Brute force attack preventions]({{< ref "/waf/policies/brute-force-attacks.md" >}}) | Configure parameters to secure areas of a web application from brute force attacks. |
 | [Cookie enforcement]({{< ref "/waf/policies/cookie-enforcement.md" >}}) | By default all cookies are allowed and not enforced for integrity. The user can add specific cookies, wildcards or explicit, that will be enforced for integrity. It is also possible to set the cookie attributes: HttpOnly, Secure and SameSite for cookies found in the response. |
 | [Data guard]({{< ref "/waf/policies/data-guard.md" >}}) | Detects and masks Credit Card Number (CCN) and/or U.S. Social Security Number (SSN) and/or custom patterns in HTTP responses. Disabled by default. |

content/waf/changelog/2024.md

@@ -199,7 +199,7 @@ _March 19, 2024_
 
 {{< table >}}
 
-| Distribution name        | NGINX Open Source (5.1)                                           |  NGINX Plus (5.1)                                              | NGINX Plus (4.9)                                  |
+| Distribution name        | NGINX Open Source (5.1)                                           |  NGINX Plus (5.1)                                              | NGINX Plus (4.8.1)                                  |
 | ------------------------ | ----------------------------------------------------------------- | -------------------------------------------------------------- |----------------------------------------------------|
 | Alpine 3.17              | _app-protect-module-oss-1.25.4+4.815.0-r1.apk_                    | _app-protect-module-plus-31+4.815.0-r1.apk_                    | _app-protect-31.4.815.0-r1.apk_                    |
 | Amazon Linux 2023        | _app-protect-module-oss-1.25.4+4.815.0-1.amzn2023.ngx.x86_64.rpm_ | _app-protect-module-plus-31+4.815.0-1.amzn2023.ngx.x86_64.rpm_ | _app-protect-31+4.815.0-1.amzn2023.ngx.x86_64.rpm_ |

content/waf/configure/compiler.md

@@ -228,6 +228,27 @@ docker run \
  -dump -bundle $(pwd)/compiled_policy.tgz
 ```
 
+## Debugging bundle errors
+
+When [configuring policies]({{< ref "/waf/policies/configuration.md">}}), you may run into problems.
+
+There are ways to remediate them based on the context:
+
+{{< table >}}
+| Description             | Solution  |
+| ----------------------- | --------  |
+| _Expected declarative policy_ | Ensure the JSON of the policy is well-formed | 
+| _Policy Bundles version is older than the local version_ | You must recompile all your bundles from scratch when installing security updates. |
+| _Policy Bundles version is newer than the local version_ | You must recompile all your bundles from scratch when installing security updates. |
+| _Found mixed content of compiled and raw configuration_ | Only pre-compiled bundles can be used in NGINX configuration: compile JSON to bundles first. |
+| _Compiler is required, but not installed: Missing /opt/app_protect/bin/config_set_compiler_ | Only pre-compiled bundles can be used in NGINX configuration: compile JSON to bundles first. |
+| _Policy Bundles have differing global states_ | Recompile all of your bundles from scratch with your custom compiler. Bundles must be compiled with the same compiler: you cannot mix default and custom bundles. |
+| _Policy Bundles have differing cookie seeds_ | Recompile all of your bundles from scratch with your custom compiler. Bundles must be compiled with the same compiler: you cannot mix default and custom bundles. |
+| _Duplicate policy name found_ | Don't compile multiple policies with the same name, or one policy to multiple bundles. Each policy can be compiled once but a bundle can be re-used. |
+| _Duplicate logging profile name found | Don't compile the same logging profile to multiple bundles. Each profile can be compiled once but a bundle can be re-used. |
+| _Timeout waiting for enforcer_ | Likely an internal issue: [contact Support]({{< ref "/waf/support.md" >}}) |
+{{< /table >}}
+
 ## Global settings
 
 The global settings allows configuration of the following items:

content/waf/configure/selinux.md

@@ -12,14 +12,20 @@ nd-content-type: how-to
 nd-product: NAP-WAF
 ---
 
-The default settings for Security-Enhanced Linux (SELinux) on modern Red Hat Enterprise Linux (RHEL) and related distros can be very strict, erring on the side of security rather than convenience.
+The default settings for Security-Enhanced Linux (SELinux) on modern Red Hat Enterprise Linux (RHEL) and related distros can be very strict, prioritizing security over user convenience.
 
 To ensure F5 WAF for NGINX operates smoothly without compromising security, consider setting up a custom SELinux policy or AppArmor profile. 
 
 For troubleshooting, you may use permissive (SELinux) or complain (AppArmor) mode to avoid these restrictions, but this is inadvisable for prolonged use.
 
 Although F5 WAF for NGINX provides an optional package with prebuilt a SELinux policy (`app-protect-selinux`), your specific configuration might be blocked unless you adjust the policy or modify file labels.
 
+{{< call-out "note" >}}
+
+You may want to read the [Using NGINX and NGINX Plus with SELinux](https://www.f5.com/company/blog/nginx/using-nginx-plus-with-selinux) blog post for more information.
+
+{{< /call-out >}}
+
 ## Modifying file labels
 
 If you plan to store your security policy files in an alternative folder such as _/etc/security_policies_, you should change the default SELinux file context:
@@ -43,4 +49,16 @@ Review the syslog ports by entering the following command:
 semanage port -l | grep syslog
 ```
 
-For more information related to syslog, see the [Security logs]({{< ref "/waf/logging/security-logs.md" >}}) topic.
+For more information related to syslog, see the [Security logs]({{< ref "/waf/logging/security-logs.md" >}}) topic.
+
+## Review audits
+
+F5 WAF for NGINX files and processes are labelled with the `nap-compiler_t` and `nap-engine_t` contexts. NGINX Plus is labelled with `httpd_t`.
+
+If SELinux denies access to something, you can search audit denials using one of the above contexts.
+
+```shell
+ausearch --start recent -m avc --raw -se nap-engine_t
+```
+
+The _--start recent_ argument searches the previous 10 minutes.

content/waf/install/update-signatures.md

@@ -19,8 +19,8 @@ For other deployment methods, you should read [Build and use the compiler tool](
 Signatures are divided into three groups:
 
 - [Attack signatures]({{< ref "/waf/policies/attack-signatures.md" >}})
+- [Bot signatures]({{< ref "/waf/policies/bot-signatures.md" >}})
 - [Threat campaigns]({{< ref "/waf/policies/threat-campaigns.md" >}})
-- Bot signatures
 
 F5 WAF for NGINX signature updates are released at a higher frequency than F5 WAF for NGINX itself, and are subsequently available in their own packages.
 
@@ -35,7 +35,7 @@ Installing these packages also installed their dependencies, which includes the
 They will be named something in the following list:
 
 - `app-protect-attack-signatures`
-- `app-protect-threat-campaigns`
 - `app-protect-bot-signatures`
+- `app-protect-threat-campaigns`
 
 You can update these packages independently of the core F5 WAF for NGINX packages, ensuring you always have the latest signatures.

content/waf/install/virtual-environment.md

@@ -40,6 +40,12 @@ Navigate to your chosen operating system, which are alphabetically ordered.
 
 ### Alpine Linux
 
+Add the F5 WAF for NGINX signing key:
+
+```shell
+sudo wget -O /etc/apk/keys/app-protect-security-updates.rsa.pub https://cs.nginx.com/static/keys/app-protect-security-updates.rsa.pub
+```
+
 Add the F5 WAF for NGINX repository:
 
 ```shell
@@ -76,6 +82,13 @@ sudo dnf install app-protect
 
 ### Debian
 
+Add the F5 WAF for NGINX signing key:
+
+```shell
+wget -qO - https://cs.nginx.com/static/keys/app-protect-security-updates.key | gpg --dearmor | \
+sudo tee /usr/share/keyrings/app-protect-security-updates.gpg > /dev/null
+```
+
 Add the F5 WAF for NGINX repositories:
 
 ```shell
@@ -115,6 +128,12 @@ Add F5 WAF for NGINX dependencies:
 sudo wget -P /etc/yum.repos.d https://cs.nginx.com/static/files/dependencies.repo
 ```
 
+Enable F5 WAF for NGINX dependencies:
+
+```shell
+sudo dnf config-manager --set-enabled crb
+```
+
 Enable the _ol8_codeready_builder_ repository:
 
 ```shell
@@ -155,6 +174,13 @@ sudo dnf install app-protect
 
 ### Ubuntu
 
+Add the F5 WAF for NGINX signing key:
+
+```shell
+wget -qO - https://cs.nginx.com/static/keys/app-protect-security-updates.key | \
+gpg --dearmor | sudo tee /usr/share/keyrings/app-protect-security-updates.gpg > /dev/null
+```
+
 Add the F5 WAF for NGINX repositories:
 
 ```shell

content/waf/policies/bot-signatures.md

@@ -0,0 +1,197 @@
+---
+# We use sentence case and present imperative tone
+title: "Bot signatures"
+# Weights are assigned in increments of 100: determines sorting order
+weight: 550
+# Creates a table of contents and sidebar, useful for large documents
+toc: true
+# Types have a 1:1 relationship with Hugo archetypes, so you shouldn't need to change this
+nd-content-type: reference
+# Intended for internal catalogue and search, case sensitive:
+# Agent, N4Azure, NIC, NIM, NGF, NAP-DOS, NAP-WAF, NGINX One, NGINX+, Solutions, Unit
+nd-product: NAP-WAF
+---
+
+Bot signatures are a feature that protects applications by detecting signatures and clients that falsely claim to be browsers or search engines.
+
+This feature is enabled by default with the `bot-defense` parameter, and includes both bot signatures and header anomalies, which can be disabled separately.
+
+## Bot signatures
+
+Bot signature detection works by inspecting the the User-Agent header and URI of a request.
+
+Each detected bot signature belongs to a bot class: search engine signatures such as `googlebot` are under the trusted_bots class, but F5 WAF for NGINX performs additional checks to authenticate a trusted bot.
+
+If these checks fail, it means the detected bot signature impersonated a search engine, and it will be given the class `malicous_bot`, anomaly `Search engine verification failed`.
+
+The request will be blocked, regardless of the class's mitigation actions configuration.
+
+This is a list of trusted bots, all of which are search engines.
+
+| Name               | Description |
+| ------------------ | ----------- |
+| Ask                | [Ask.com engine](https://www.ask.com) |
+| Baidu              | [Baidu search engine](https://www.baidu.com/) |
+| Baidu Image Spider | [Baidu search engine for images](https://image.baidu.com/) |
+| Bing               | [Microsoft Bing search engine](https://www.bing.com/) |
+| BingPreview        | [Microsoft Bing page snapshot generation engine](https://www.bing.com/) |
+| Daum               | [Daum search engine](https://www.daum.net/) |
+| DuckDuckGo Bot     | [DuckDuckGo search engine](https://duckduckgo.com/) |
+| fastbot            | [fastbot search engine](https://www.fastbot.de/) |
+| Google             | [Google search engine](https://www.google.com/) |
+| MojeekBot          | [Mojeek search engine](https://www.mojeek.com/) |
+| Yahoo! Slurp       | [Yahoo search engine](https://www.yahoo.com/) |
+| Yandex             | [Yandex search engine](https://yandex.com/) |
+| YioopBot           | Yioop search engine |
+
+An action can be configured for each bot class, or configured for each bot signature individually:
+
+* `ignore`    - Bot signature is ignored (disabled)
+* `detect`    - Only report without raising the violation - `VIOL_BOT_CLIENT`. The request is considered `legal` unless another violation is triggered.
+* `alarm`     - Report, raise the violation, but pass the request. The request is marked as `illegal`.
+* `block`     - Report, raise the violation, and block the request
+
+This example enables bot signatures using the default bot configuration:
+
+```json
+{
+    "policy": {
+        "name": "bot_defense_policy",
+        "template": {
+            "name": "POLICY_TEMPLATE_NGINX_BASE"
+        },
+        "applicationLanguage": "utf-8",
+        "enforcementMode": "blocking",
+        "bot-defense": {
+            "settings": {
+                "isEnabled": true
+            }
+        }
+    }
+}
+```
+
+The default actions for classes are: `detect` for `trusted-bot`, `alarm` for `untrusted-bot`, and `block` for `malicious-bot`. 
+
+The next example enables bot defense, configuring a violation for `trusted-bot`, and block for `untrusted-bot`.
+
+```json
+{
+    "policy": {
+        "name": "bot_defense_policy",
+        "template": {
+            "name": "POLICY_TEMPLATE_NGINX_BASE"
+        },
+        "applicationLanguage": "utf-8",
+        "enforcementMode": "blocking",
+        "bot-defense": {
+            "settings": {
+                "isEnabled": true
+            },
+            "mitigations": {
+                "classes": [
+                    {
+                        "name": "trusted-bot",
+                        "action": "alarm"
+                    },
+                    {
+                        "name": "untrusted-bot",
+                        "action": "block"
+                    },
+                    {
+                        "name": "malicious-bot",
+                        "action": "block"
+                    }
+                ]
+            }
+        }
+    }
+}
+```
+
+The next example overrides the action for a specific signature (python-requests):
+
+```json
+{
+    "policy": {
+        "name": "bot_defense_policy",
+        "template": {
+            "name": "POLICY_TEMPLATE_NGINX_BASE"
+        },
+        "applicationLanguage": "utf-8",
+        "enforcementMode": "blocking",
+        "bot-defense": {
+            "settings": {
+                "isEnabled": true
+            },
+            "mitigations": {
+                "signatures": [
+                    {
+                        "action": "ignore",
+                        "name": "python-requests"
+                    }
+                ]
+            }
+        }
+    }
+}
+```
+
+The bot signature file, `included_bot_signatures`, is located at the following path: `/opt/app-protect/var/update_files/bot_signatures/included_bot_signatures`.
+
+This file is an up-to-date list of all bot signatures, following a format similar to the README-style text file found for [attack signatures]({{< ref "/waf/policies/attack-signatures.md" >}}).
+
+It contains information such as:
+
+- Bot name
+- Bot type
+- Bot classification/category
+
+It is part of the _app-protect-bot-signatures_ package: for more information, see the [Update F5 WAF for NGINX signatures]({{< ref "/waf/install/update-signatures.md" >}}) topic.
+
+## Header anomalies
+
+In addition to detecting bot signatures, F5 WAF for NGINX verifies that a client is the browser it claims to be by inspecting the HTTP headers.
+
+Each request receives a score and anomaly category, and is enforced according to the default anomaly action:
+
+| Range          | Anomaly                                   | Action | Class              |
+|--------------- | ----------------------------------------- | ------ | ------------------ |
+| 0-49           | None                                      | None   | Browser            |
+| 50-99          | Suspicious HTTP Headers Presence or Order | Alarm  | Suspicious Browser |
+| 100 and above  | Invalid HTTP Headers Presence or Order    | Block  | Malicious Bot      |
+| Non Applicable | SEARCH_ENGINE_VERIFICATION_FAILED         | Block  | Malicious Bot      |
+
+The default scores for each anomaly can be changed. 
+
+In this example, the score and action of the default bot configuration has been overrided:
+
+```json
+{
+    "policy": {
+        "name": "bot_anomalies_and_signatures",
+        "template": {
+            "name": "POLICY_TEMPLATE_NGINX_BASE"
+        },
+        "applicationLanguage": "utf-8",
+        "enforcementMode": "blocking",
+        "bot-defense": {
+            "mitigations": {
+                "anomalies": [
+                    {
+                        "name": "Suspicious HTTP Headers",
+                        "action": "alarm",
+                        "scoreThreshold": 50
+                    },
+                    {
+                        "name": "Invalid HTTP Headers",
+                        "action": "block",
+                        "scoreThreshold": 99
+                    }
+                ]
+            }
+        }
+    }
+}
+
+```