Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -2,36 +2,35 @@
pcx_content_type: troubleshooting
source: https://support.cloudflare.com/hc/en-us/articles/360057896851-Required-Firewall-Rule-changes-to-enable-URL-normalization
title: Required firewall rule changes to enable URL normalization

---

import { Render } from "~/components"
import { Render } from "~/components";

<Render file="deprecation-notice" />

On 2021-04-08, Cloudflare announced [URL normalization](/rules/normalization/), a feature that protects zones by normalizing HTTP request URI paths.

Malicious users can craft specific URIs that could be interpreted differently by firewall systems and origin systems. When you enable **Normalize incoming URLs**, all rules filtering on the URI path will receive the URL in a canonical form, which provides an extra layer of protection against these malicious users. 
Malicious users can craft specific URIs that could be interpreted differently by firewall systems and origin systems. When you enable **Normalize incoming URLs**, all rules filtering on the URI path will receive the URL in a canonical form, which provides an extra layer of protection against these malicious users.

Cloudflare gradually enabled URL normalization for all Cloudflare zones except for those that could be impacted by this change. We determined the impacted zones by analyzing all firewall rules, looking for patterns in HTTP fields that would no longer match when using URL normalization techniques.
Cloudflare gradually enabled URL normalization for all Cloudflare zones except for those that could be impacted by this change. We determined the impacted zones by analyzing all firewall rules, looking for patterns in HTTP fields that would no longer match when using URL normalization techniques.

These fields are the following:

* `http.request.uri.path`
* `http.request.full_uri`
* `http.request.uri`
- `http.request.uri.path`
- `http.request.full_uri`
- `http.request.uri`

Cloudflare did not enable URL normalization automatically for zones that would be impacted by these changes to prevent any change in behavior of your existing firewall rules.

## Why URL normalization is important

Cloudflare strongly recommends that you enable **Normalize incoming URLs** in **Rules** > **Settings** to strengthen your zone's security posture. Not doing so leaves your zone at greater risk of a successful attack. Malicious parties could craft the URL in a way that the rules are not accounting for.
Cloudflare strongly recommends that you enable **Normalize incoming URLs** in **Rules** > **URL Normalization** to strengthen your zone's security posture. Not doing so leaves your zone at greater risk of a successful attack. Malicious parties could craft the URL in a way that the rules are not accounting for.

For example, a firewall rule with an expression such as `http.request.uri.path contains "/login"` could be bypassed if the malicious actor has encoded the `l` character as `%6C`. In this scenario, and with URL normalization disabled, traffic would not be matched by the firewall rule.

Refer to [How URL normalization works](/rules/normalization/how-it-works/) for more information and additional examples.

***
---

## Recommended procedure

Expand All @@ -46,23 +45,23 @@ These steps will ensure a stronger security posture on your zone(s).

Before enabling URL normalization, you should review the affected firewall rules on your zone(s) and take one of the following approaches:

* Edit these firewall rules to remove the parts which will no longer trigger once normalized — for example, any rules that look for `//` or `../` in URL paths. Administrators previously created these rules to perform a limited URL normalization, and these rules can now be safely disabled and then deleted.
- Edit these firewall rules to remove the parts which will no longer trigger once normalized — for example, any rules that look for `//` or `../` in URL paths. Administrators previously created these rules to perform a limited URL normalization, and these rules can now be safely disabled and then deleted.

* If you wish to identify visitors with non-normalized URI paths with these firewall rules, you should update them to use the original (or raw) non-normalized fields. These fields are the following:
- If you wish to identify visitors with non-normalized URI paths with these firewall rules, you should update them to use the original (or raw) non-normalized fields. These fields are the following:

* `raw.http.request.uri.path`
* `raw.http.request.full_uri`
* `raw.http.request.uri`
- `raw.http.request.uri.path`
- `raw.http.request.full_uri`
- `raw.http.request.uri`

### 2. Enable URL normalization

Once you have updated the affected firewall rules, enable URL normalization in **Rules** > **Settings**.
Once you have updated the affected firewall rules, enable URL normalization in **Rules** > **URL Normalization**.

A Cloudflare user must have the [Firewall role](/fundamentals/setup/manage-members/roles/) or one of the Administrator roles to access URL normalization settings in the dashboard.

***
---

## Related resources

* [URL normalization](/rules/normalization/)
* [Transform Rules](/rules/transform/)
- [URL normalization](/rules/normalization/)
- [Transform Rules](/rules/transform/)
43 changes: 19 additions & 24 deletions src/content/docs/rules/normalization/examples.mdx
Original file line number Diff line number Diff line change
@@ -1,36 +1,31 @@
---
title: Examples
title: URL normalization examples
pcx_content_type: configuration
sidebar:
order: 6
head:
- tag: title
content: URL normalization examples
label: Examples
description: Examples of the impact of different URL normalization settings in
the URLs of incoming requests.

tableOfContents: false
---

The following table shows how different [URL normalization settings](/rules/normalization/settings/) affect request URLs before they pass to other Cloudflare features and to the origin server:



| Incoming URL | Normalization type | Normalize incoming URLs | Normalize URLs to origin | URL at Cloudflare's network | URL passed to origin server |
| ---------------------------- | ------------------ | ----------------------- | ------------------------ | ------------------------------ | ------------------------------ |
| `www.example.com/hello` | (any) | *Off* | *Off* | `www.example.com/hello` | `www.example.com/hello` |
| `www.example.com/hello` | (any) | *On* | *Off* | `www.example.com/hello` | `www.example.com/hello` |
| `www.example.com/hello` | (any) | *On* | *On* | `www.example.com/hello` | `www.example.com/hello` |
| `example.com/%68ello` | (any) | *Off* | *Off* | `example.com/%68ello` | `example.com/%68ello` |
| `example.com/%68ello` | (any) | *On* | *Off* | `example.com/hello` | `example.com/%68ello` |
| `example.com/%68ello` | (any) | *On* | *On* | `example.com/hello` | `example.com/hello` |
| `example.com/%68ello//pa\th` | *RFC-3986* | *Off* | *Off* | `example.com/%68ello//pa\th` | `example.com/%68ello//pa\th` |
| `example.com/%68ello//pa\th` | *RFC-3986* | *On* | *Off* | `example.com/hello//pa%5Cth` | `example.com/%68ello//pa\th` |
| `example.com/%68ello//pa\th` | *RFC-3986* | *On* | *On* | `example.com/hello//pa%5Cth` | `example.com/hello//pa%5Cth` |
| `example.com/%68ello//pa\th` | *Cloudflare* | *Off* | *Off* | `example.com/%68ello//pa\th` | `example.com/%68ello//pa\th` |
| `example.com/%68ello//pa\th` | *Cloudflare* | *On* | *Off* | `example.com/hello/pa/th` | `example.com/%68ello//pa\th` |
| `example.com/%68ello//pa\th` | *Cloudflare* | *On* | *On* | `example.com/hello/pa/th` | `example.com/hello/pa/th` |
| `example.com/hello//../path` | *RFC-3986* | *On* | *On* | `example.com/hello/path` | `example.com/hello/path` |
| `example.com/hello//../path` | *Cloudflare* | *On* | *On* | `example.com/path` | `example.com/path` |
| `example.com/hello/\../path` | *RFC-3986* | *On* | *On* | `example.com/hello/%5C../path` | `example.com/hello/%5C../path` |
| `example.com/hello/\../path` | *Cloudflare* | *On* | *On* | `example.com/path` | `example.com/path` |

| `www.example.com/hello` | (any) | _Off_ | _Off_ | `www.example.com/hello` | `www.example.com/hello` |
| `www.example.com/hello` | (any) | _On_ | _Off_ | `www.example.com/hello` | `www.example.com/hello` |
| `www.example.com/hello` | (any) | _On_ | _On_ | `www.example.com/hello` | `www.example.com/hello` |
| `example.com/%68ello` | (any) | _Off_ | _Off_ | `example.com/%68ello` | `example.com/%68ello` |
| `example.com/%68ello` | (any) | _On_ | _Off_ | `example.com/hello` | `example.com/%68ello` |
| `example.com/%68ello` | (any) | _On_ | _On_ | `example.com/hello` | `example.com/hello` |
| `example.com/%68ello//pa\th` | _RFC-3986_ | _Off_ | _Off_ | `example.com/%68ello//pa\th` | `example.com/%68ello//pa\th` |
| `example.com/%68ello//pa\th` | _RFC-3986_ | _On_ | _Off_ | `example.com/hello//pa%5Cth` | `example.com/%68ello//pa\th` |
| `example.com/%68ello//pa\th` | _RFC-3986_ | _On_ | _On_ | `example.com/hello//pa%5Cth` | `example.com/hello//pa%5Cth` |
| `example.com/%68ello//pa\th` | _Cloudflare_ | _Off_ | _Off_ | `example.com/%68ello//pa\th` | `example.com/%68ello//pa\th` |
| `example.com/%68ello//pa\th` | _Cloudflare_ | _On_ | _Off_ | `example.com/hello/pa/th` | `example.com/%68ello//pa\th` |
| `example.com/%68ello//pa\th` | _Cloudflare_ | _On_ | _On_ | `example.com/hello/pa/th` | `example.com/hello/pa/th` |
| `example.com/hello//../path` | _RFC-3986_ | _On_ | _On_ | `example.com/hello/path` | `example.com/hello/path` |
| `example.com/hello//../path` | _Cloudflare_ | _On_ | _On_ | `example.com/path` | `example.com/path` |
| `example.com/hello/\../path` | _RFC-3986_ | _On_ | _On_ | `example.com/hello/%5C../path` | `example.com/hello/%5C../path` |
| `example.com/hello/\../path` | _Cloudflare_ | _On_ | _On_ | `example.com/path` | `example.com/path` |
27 changes: 12 additions & 15 deletions src/content/docs/rules/normalization/how-it-works.mdx
Original file line number Diff line number Diff line change
@@ -1,12 +1,9 @@
---
title: How it works
title: How URL normalization works
pcx_content_type: concept
sidebar:
order: 3
head:
- tag: title
content: How URL normalization works

label: How it works
---

URL normalization modifies separators, encoded elements, and literal bytes in incoming URLs so that they conform to a consistent formatting standard.
Expand All @@ -15,8 +12,8 @@ For example, consider a WAF custom rule that blocks requests whose URLs match `w

The two available types of URL normalization are:

* [RFC 3986 normalization](#rfc-3986-normalization)
* [Cloudflare normalization](#cloudflare-normalization)
- [RFC 3986 normalization](#rfc-3986-normalization)
- [Cloudflare normalization](#cloudflare-normalization)

The location where URL normalization will occur depends on the [configured settings](/rules/normalization/settings/).

Expand All @@ -26,14 +23,14 @@ For examples of the different settings and their impact on request URLs, refer t

The URL normalization performed according to [RFC 3986](https://www.ietf.org/rfc/rfc3986.txt) is as follows:

* The following unreserved characters are [percent decoded](https://tools.ietf.org/html/rfc3986#section-2.1):
* Alphabetical characters: `a`-`z`, `A`-`Z` (decoded from `%41`-`%5A` and `%61`-`%7A`)
* Digit characters: `0`-`9` (decoded from `%30`-`%39`)
* hyphen `-` (`%2D`), period `.` (`%2E`), underscore `_` (`%5F`), and tilde `~` (`%7E`)
* These reserved characters are not encoded or decoded: `: / ? # [ ] @ ! $ & ' ( ) * + , ; =`
* Other characters, for example literal byte values, are percent encoded.
* Percent encoded representations are converted to upper case.
* URL paths are normalized according to the [Remove Dot Segments](https://tools.ietf.org/html/rfc3986#section-5.2.4) protocol.
- The following unreserved characters are [percent decoded](https://tools.ietf.org/html/rfc3986#section-2.1):
- Alphabetical characters: `a`-`z`, `A`-`Z` (decoded from `%41`-`%5A` and `%61`-`%7A`)
- Digit characters: `0`-`9` (decoded from `%30`-`%39`)
- hyphen `-` (`%2D`), period `.` (`%2E`), underscore `_` (`%5F`), and tilde `~` (`%7E`)
- These reserved characters are not encoded or decoded: `: / ? # [ ] @ ! $ & ' ( ) * + , ; =`
- Other characters, for example literal byte values, are percent encoded.
- Percent encoded representations are converted to upper case.
- URL paths are normalized according to the [Remove Dot Segments](https://tools.ietf.org/html/rfc3986#section-5.2.4) protocol.

## Cloudflare normalization

Expand Down
12 changes: 7 additions & 5 deletions src/content/docs/rules/normalization/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,25 +3,27 @@ pcx_content_type: concept
title: URL normalization
sidebar:
order: 11

---

import { GlossaryTooltip, Render } from "~/components"
import { GlossaryTooltip, Render } from "~/components";

Cloudflare provides a URL normalization feature to modify the URLs of incoming requests so that they conform to a consistent formatting standard.

When you enable URL normalization, all incoming URLs are normalized before they pass to subsequent global network features that accept a URL input, such as WAF custom rules, Workers, and Access. Rule expressions that filter traffic based on URLs will therefore trigger correctly, regardless of the format of the incoming URL. When URL normalization is disabled, Cloudflare forwards the URL to origin in its original form.

:::caution

When traffic is proxied via Cloudflare, essential request URL normalization is always applied regardless whether URL normalization is enabled for a specific zone. For example, you cannot disable the conversion of two or more adjacent slashes into a single slash in a request URL by turning off URL normalization.
When traffic is proxied via Cloudflare, essential request URL normalization is always applied regardless whether URL normalization is enabled for a specific zone. For example, you cannot disable the conversion of two or more adjacent slashes into a single slash in a request URL by turning off URL normalization.
:::

URL normalization does not perform any <GlossaryTooltip term="redirect">redirects</GlossaryTooltip>, and therefore it will not change the address displayed in the visitor's browser. The normalization operation, when enabled, occurs on the global network and affects Cloudflare features executed later and (optionally) the URL received at the origin server.

<Render file="rules-requirements" params={{ one: "URL normalization requires" }} />
<Render
file="rules-requirements"
params={{ one: "URL normalization requires" }}
/>

***
---

## Availability

Expand Down
13 changes: 5 additions & 8 deletions src/content/docs/rules/normalization/manage.mdx
Original file line number Diff line number Diff line change
@@ -1,19 +1,16 @@
---
title: Configure in the dashboard
title: Configure URL normalization in the dashboard
pcx_content_type: how-to
sidebar:
order: 4
head:
- tag: title
content: Configure URL normalization in the dashboard
label: Configure in the dashboard
description: How to configure URL normalization in the Cloudflare dashboard.

---

1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/), and select your account and website.

2. Go to **Rules** > **Settings**.
2. Go to **Rules** > **URL Normalization**.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.


![Available URL normalization settings in the Cloudflare dashboard](~/assets/images/rules/normalization/url-normalization-settings.png)
3. Configure the [available URL normalization settings](/rules/normalization/settings/).

3. Under **Configure Normalization**, configure the [available URL normalization settings](/rules/normalization/settings/).
![Available URL normalization settings in the Cloudflare dashboard](~/assets/images/rules/normalization/url-normalization-settings.png)
37 changes: 15 additions & 22 deletions src/content/docs/rules/normalization/settings.mdx
Original file line number Diff line number Diff line change
@@ -1,42 +1,35 @@
---
title: Settings
title: URL normalization settings
pcx_content_type: reference
sidebar:
order: 5
head:
- tag: title
content: URL normalization settings

label: Settings
---

The Cloudflare dashboard provides the following settings to manage URL normalization:

- **Normalization type** (default: _RFC-3986_)

- Selects the type of normalization to perform:

* **Normalization type** (default: *RFC-3986*)

* Selects the type of normalization to perform:

* *RFC-3986* – Applies URL normalization strictly according to [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986).

* *Cloudflare* – In addition to what is defined in RFC 3986, applies [extra URL normalization techniques](/rules/normalization/how-it-works/#cloudflare-normalization).

* **Normalize incoming URLs** (default: *On*)
- _RFC-3986_ – Applies URL normalization strictly according to [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986).

* Configures the URLs of all incoming traffic to Cloudflare:
- _Cloudflare_ – In addition to what is defined in RFC 3986, applies [extra URL normalization techniques](/rules/normalization/how-it-works/#cloudflare-normalization).

* When enabled, all incoming URLs are normalized before they pass to subsequent Cloudflare features that can receive a URL as input, such as Page Rules, WAF custom rules, Workers, and Access.
* When disabled, incoming URLs are not normalized before passing to subsequent Cloudflare features.
- **Normalize incoming URLs** (default: _On_)

* **Normalize URLs to origin** (default: *Off*)
- Configures the URLs of all incoming traffic to Cloudflare:

* Configures URLs sent to the origin:
- When enabled, all incoming URLs are normalized before they pass to subsequent Cloudflare features that can receive a URL as input, such as Page Rules, WAF custom rules, Workers, and Access.
- When disabled, incoming URLs are not normalized before passing to subsequent Cloudflare features.

* When enabled, requests sent to the origin are normalized.
* When disabled, requests sent to the origin are not modified.
- **Normalize URLs to origin** (default: _Off_)

* You can only view and enable this option when **Normalize incoming URLs** is enabled.
- Configures URLs sent to the origin:

- When enabled, requests sent to the origin are normalized.
- When disabled, requests sent to the origin are not modified.

- You can only view and enable this option when **Normalize incoming URLs** is enabled.

For examples of how these settings affect URL normalization, refer to the [URL normalization examples](/rules/normalization/examples/).
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
---
pcx_content_type: how-to
title: Create via API
title: Create Bulk Redirects via API
sidebar:
order: 7
head:
- tag: title
content: Create Bulk Redirects via API
label: Create via API
description: Learn how to create Bulk Redirects using the Cloudflare API.
---

Expand Down
Loading
Loading