Skip to content

Commit 936719f

Browse files
patriciasantaanapatriciasantaana
committed
schema learning
1 parent 84a0694 commit 936719f

File tree

7 files changed

+192
-169
lines changed

7 files changed

+192
-169
lines changed

‎src/content/docs/api-shield/management-and-monitoring/api-routing/index.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ pcx_content_type: how-to
33
type: overview
44
title: API Routing
55
sidebar:
6-
order: 3
6+
order: 4
77

88
---
99

‎src/content/docs/api-shield/management-and-monitoring/developer-portal.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ pcx_content_type: how-to
33
type: overview
44
title: Build developer portals
55
sidebar:
6-
order: 4
6+
order: 5
77

88
---
99

‎src/content/docs/api-shield/management-and-monitoring/endpoint-labels.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ pcx_content_type: how-to
33
type: overview
44
title: Endpoint labeling service
55
sidebar:
6-
order: 1
6+
order: 2
77
label: Labeling service
88
---
99

‎src/content/docs/api-shield/management-and-monitoring/endpoint-management/index.mdx‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
---
2+
pcx_content_type: concept
3+
type: overview
4+
title: Endpoint Management
5+
sidebar:
6+
order: 1
7+
8+
---
9+
10+
import { GlossaryTooltip, Plan } from "~/components"
11+
12+
<Plan type="all" />
13+
14+
Monitor the health of your <GlossaryTooltip term="API endpoint">API endpoints</GlossaryTooltip> by saving, updating, and monitoring performance metrics using API Shield’s Endpoint Management.
15+
16+
**Add endpoints** allows customers to save endpoints directly from [API Discovery](/api-shield/security/api-discovery/) or manually by method, path, and host.
17+
18+
This will add the specified endpoints to your list of managed endpoints. You can view your list of saved endpoints in the **Endpoint Management** page.
19+
20+
Cloudflare will start collecting [performance data](/api-shield/management-and-monitoring/#endpoint-analysis) on your endpoint when you save an endpoint.
21+
22+
:::note
23+
24+
When an endpoint is using [Cloudflare Workers](/workers/), the metrics data will not be populated.
25+
:::
26+
27+
## Access
28+
29+
1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/login) and select your account and domain.
30+
2. Select **Security** > **API Shield**.
31+
3. Add your endpoints [manually](#add-endpoints-manually), from [Schema validation](#add-endpoints-from-schema-validation), or from [API Discovery](#add-endpoints-from-api-discovery).
32+
33+
## Add endpoints from API Discovery
34+
35+
There are two ways to add API endpoints from Discovery.
36+
37+
### Add from the Endpoint Management Tab
38+
39+
1. From Endpoint Management, select **Add endpoints** > **Select from Discovery** tab.
40+
2. Select the discovered endpoints you would like to add.
41+
3. Select **Add endpoints**.
42+
43+
### Add from the Discovery Tab
44+
45+
1. From Endpoint Management, select the **Discovery** tab.
46+
2. Select the discovered endpoints you would like to add.
47+
3. Select **Save selected endpoints**.
48+
49+
## Add endpoints from Schema validation
50+
51+
1. Add a schema by [configuring Schema validation](/api-shield/security/schema-validation/).
52+
2. On **Review schema endpoints**, save new endpoints to endpoint management by checking the box.
53+
3. Select **Save as draft** or **Save and Deploy**. Endpoints will be saved regardless of whether the Schema is saved as a draft or published.
54+
55+
API Shield will look for duplicate endpoints that have the same host, method, and path. Duplicate endpoints will not be saved to endpoint management.
56+
57+
:::note
58+
59+
If you deselect **Save new endpoints to endpoint management**, the endpoints will not be added.
60+
:::
61+
62+
## Add endpoints manually
63+
64+
1. From Endpoint Management, select **Add endpoints** > **Manually add**.
65+
2. Choose the method from the dropdown menu and add the path and hostname for the endpoint.
66+
3. Select **Add endpoints**.
67+
68+
:::note
69+
70+
By selecting multiple checkboxes, you can add several endpoints from Discovery at once instead of individually.
71+
:::
72+
73+
When adding an endpoint manually, you can specify variable fields in the path or host by enclosing them in braces, `/api/user/{var1}/details` or `{hostVar1}.example.com`.
74+
75+
Cloudflare supports hostname variables in the following formats:
76+
77+
```txt
78+
79+
{hostVar1}.example.com
80+
81+
foo.{hostVar1}.example.com
82+
83+
{hostVar2}.{hostVar1}.example.com
84+
```
85+
86+
Hostname variables must comprise the entire domain field and must not be used with other text in the field.
87+
88+
The following format is not supported:
89+
90+
```txt
91+
92+
foo-{hostVar1}.example.com
93+
```
94+
95+
For more information on how Cloudflare uses variables in API Shield, refer to the examples from [API Discovery](/api-shield/security/api-discovery/).
96+
97+
## Delete endpoints manually
98+
99+
You can delete endpoints one at a time or in bulk.
100+
101+
1. From Endpoint Management, select the checkboxes for the endpoints that you want to delete.
102+
2. Select **Delete endpoints**.
103+
104+
## Endpoint Analysis
105+
106+
For each saved endpoint, customers can view:
107+
108+
- **Request count**: The total number of requests to the endpoint over time.
109+
- **Rate limiting recommendation**: per 10 minutes. This is guided by the request count.
110+
- **Latency**: The average origin response time in milliseconds (ms). This metric shows how long it takes from the moment a visitor makes a request to the moment the visitor gets a response back from the origin.
111+
- **Error rate** vs. overall traffic: grouped by 4xx, 5xx, and their sum.
112+
- **Response size**: The average size of the response (in bytes) returned to the request.
113+
- **Labels**: The current [labels](/api-shield/management-and-monitoring/endpoint-labels/) assigned to the endpoint.
114+
- **Authentication status**: The breakdown of which [session identifiers](/api-shield/get-started/#session-identifiers) were seen on successful requests to this endpoint.
115+
- **Sequences**: The number of [Sequence Analytics](/api-shield/security/sequence-analytics/) sequences the endpoint was found in.
116+
117+
:::note
118+
119+
Customers viewing analytics have the ability to toggle detailed metrics view between the last 24 hours and 7 days.
120+
:::
121+
122+
## Using the Cloudflare API
123+
124+
You can interact with Endpoint Management through the Cloudflare API. Refer to [Endpoint Management’s API documentation](/api/resources/api_gateway/subresources/discovery/subresources/operations/methods/list/) for more information.
125+
126+
## Sensitive Data Detection
127+
128+
Sensitive data comprises various personally identifiable information and financial data. Cloudflare created this ruleset to address common data loss threats, and the WAF can search for this data in HTTP response bodies from your origin.
129+
130+
API Shield will alert users to the presence of sensitive data in the response body of API endpoints listed in Endpoint Management if the zone is also subscribed to the [Sensitive Data Detection managed ruleset](/waf/managed-rules/reference/sensitive-data-detection/).
131+
132+
Sensitive Data Detection is available to Enterprise customers on our Advanced application security plan.
133+
134+
Once Sensitive Data Detection is enabled for your zone, API Shield queries firewall events from the WAF for the last seven days and places a notification icon on the Endpoint Management table row if there are any matched sensitive responses for your endpoint.
135+
136+
API Shield displays the types of sensitive data found if you expand the Endpoint Management table row to view further details. Select **Explore Events** to view the matched events in Security Events.
137+
138+
After Sensitive Data Detection is enabled for your zone, you can [browse the Sensitive Data Detection ruleset](https://dash.cloudflare.com/?to=/:account/:zone/security/data/ruleset/e22d83c647c64a3eae91b71b499d988e/rules). The link will not work if Sensitive Data Detection is not enabled.

‎src/content/docs/api-shield/management-and-monitoring/endpoint-management/schema-learning.mdx‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
pcx_content_type: concept
3+
type: overview
4+
title: Schema learning
5+
sidebar:
6+
order: 2
7+
---
8+
9+
Cloudflare learns schema parameters via traffic inspection. For all endpoints saved to Endpoint Management, you can export the learned schema in OpenAPI `v3.0.0` format by hostname.
10+
11+
To protect your API with a learned schema, refer to [Schema validation](/api-shield/security/schema-validation/#add-validation-by-applying-a-learned-schema-to-an-entire-hostname).
12+
13+
## Export a schema
14+
15+
1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/) and select your account and domain.
16+
2. Select **Security** > **API Shield**.
17+
3. Navigate to **Endpoint Management**.
18+
4. Select **Export schema** and choose a hostname to export.
19+
5. Select whether to include [learned parameters](/api-shield/management-and-monitoring/#learned-schemas-will-always-include) and [rate limit recommendations](/api-shield/security/volumetric-abuse-detection/)
20+
6. Select **Export schema** and choose a location to save the file.
21+
22+
:::note
23+
The schema is saved as a JSON file in OpenAPI `v3.0.0` format.
24+
:::
25+
26+
Learned schemas will always include:
27+
28+
- The listed hostname in the servers section
29+
- All endpoints by host, method, and path
30+
31+
For endpoints that receive sufficient traffic, learned schemas will also include:
32+
33+
- Detected path variables and formats
34+
- Detected query parameters and formats
35+
- Detected `POST`, `PUT`, and `PATCH` body variable names and formats for `application/json` content types
36+
37+
Learned schemas can optionally include:
38+
39+
- API Shield's rate limit threshold recommendations
40+
41+
## Limitations
42+
43+
Endpoints must be added for at least 24 hours before Schema Learning begins. Schema Learning is a continuous process that inspects the last 72 hours of traffic to an endpoint.
44+
45+
Schema Learning only learns from requests with `2xx` response codes.
46+
47+
Schema Learning works best with high volumes of traffic. You may see less confident learned schemas for endpoints with less than 10,000 requests in the last 72 hours.

‎src/content/docs/api-shield/management-and-monitoring/index.mdx‎

Lines changed: 3 additions & 165 deletions
Original file line numberDiff line numberDiff line change
@@ -1,171 +1,9 @@
11
---
2-
pcx_content_type: concept
3-
type: overview
2+
pcx_content_type: reference
43
title: Management and Monitoring
54
sidebar:
65
order: 6
7-
label: Endpoint Management
6+
group:
7+
hideIndex: true
88

99
---
10-
11-
import { GlossaryTooltip, Plan } from "~/components"
12-
13-
<Plan type="all" />
14-
15-
Monitor the health of your <GlossaryTooltip term="API endpoint">API endpoints</GlossaryTooltip> by saving, updating, and monitoring performance metrics using API Shield’s Endpoint Management.
16-
17-
**Add endpoints** allows customers to save endpoints directly from [API Discovery](/api-shield/security/api-discovery/) or manually by method, path, and host.
18-
19-
This will add the specified endpoints to your list of managed endpoints. You can view your list of saved endpoints in the **Endpoint Management** page.
20-
21-
Cloudflare will start collecting [performance data](/api-shield/management-and-monitoring/#endpoint-analysis) on your endpoint when you save an endpoint.
22-
23-
:::note
24-
25-
When an endpoint is using [Cloudflare Workers](/workers/), the metrics data will not be populated.
26-
:::
27-
28-
## Access
29-
30-
1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/login) and select your account and domain.
31-
2. Select **Security** > **API Shield**.
32-
3. Add your endpoints [manually](#add-endpoints-manually), from [Schema validation](#add-endpoints-from-schema-validation), or from [API Discovery](#add-endpoints-from-api-discovery).
33-
34-
## Add endpoints from API Discovery
35-
36-
There are two ways to add API endpoints from Discovery.
37-
38-
### Add from the Endpoint Management Tab
39-
40-
1. From Endpoint Management, select **Add endpoints** > **Select from Discovery** tab.
41-
2. Select the discovered endpoints you would like to add.
42-
3. Select **Add endpoints**.
43-
44-
### Add from the Discovery Tab
45-
46-
1. From Endpoint Management, select the **Discovery** tab.
47-
2. Select the discovered endpoints you would like to add.
48-
3. Select **Save selected endpoints**.
49-
50-
## Add endpoints from Schema validation
51-
52-
1. Add a schema by [configuring Schema validation](/api-shield/security/schema-validation/).
53-
2. On **Review schema endpoints**, save new endpoints to endpoint management by checking the box.
54-
3. Select **Save as draft** or **Save and Deploy**. Endpoints will be saved regardless of whether the Schema is saved as a draft or published.
55-
56-
API Shield will look for duplicate endpoints that have the same host, method, and path. Duplicate endpoints will not be saved to endpoint management.
57-
58-
:::note
59-
60-
If you deselect **Save new endpoints to endpoint management**, the endpoints will not be added.
61-
:::
62-
63-
## Add endpoints manually
64-
65-
1. From Endpoint Management, select **Add endpoints** > **Manually add**.
66-
2. Choose the method from the dropdown menu and add the path and hostname for the endpoint.
67-
3. Select **Add endpoints**.
68-
69-
:::note
70-
71-
By selecting multiple checkboxes, you can add several endpoints from Discovery at once instead of individually.
72-
:::
73-
74-
When adding an endpoint manually, you can specify variable fields in the path or host by enclosing them in braces, `/api/user/{var1}/details` or `{hostVar1}.example.com`.
75-
76-
Cloudflare supports hostname variables in the following formats:
77-
78-
```txt
79-
80-
{hostVar1}.example.com
81-
82-
foo.{hostVar1}.example.com
83-
84-
{hostVar2}.{hostVar1}.example.com
85-
```
86-
87-
Hostname variables must comprise the entire domain field and must not be used with other text in the field.
88-
89-
The following format is not supported:
90-
91-
```txt
92-
93-
foo-{hostVar1}.example.com
94-
```
95-
96-
For more information on how Cloudflare uses variables in API Shield, refer to the examples from [API Discovery](/api-shield/security/api-discovery/).
97-
98-
## Delete endpoints manually
99-
100-
You can delete endpoints one at a time or in bulk.
101-
102-
1. From Endpoint Management, select the checkboxes for the endpoints that you want to delete.
103-
2. Select **Delete endpoints**.
104-
105-
## Endpoint schema learning
106-
107-
Cloudflare learns schema parameters via traffic inspection. For all endpoints saved to Endpoint Management, you can export OpenAPI schemas in `v3.0.0` format by hostname. You can also include learned schema parameters.
108-
109-
To protect your API with a learned schema, refer to [Schema validation](/api-shield/security/schema-validation/#add-validation-by-applying-a-learned-schema-to-an-entire-hostname).
110-
111-
### Export a schema
112-
113-
1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/) and select your account and domain.
114-
2. Select **Security** > **API Shield**.
115-
3. Navigate to **Endpoint Management**.
116-
4. Select **Export schema** and choose a hostname to export.
117-
5. Select whether to include [learned parameters](/api-shield/management-and-monitoring/#learned-schemas-will-always-include) and [rate limit recommendations](/api-shield/security/volumetric-abuse-detection/)
118-
6. Select **Export schema** and choose a location to save the file.
119-
120-
:::note
121-
122-
The schema is saved as a JSON file in OpenAPI `v3.0.0` format.
123-
:::
124-
125-
#### Learned schemas will always include:
126-
127-
- The listed hostname in the servers section
128-
- All endpoints by host, method, and path
129-
- Detected path variables
130-
- JSON `POST` body formats
131-
132-
#### Learned schemas can optionally include:
133-
134-
- Detected query parameters and its format
135-
- API Shield’s rate limit threshold recommendations
136-
137-
## Endpoint Analysis
138-
139-
For each saved endpoint, customers can view:
140-
141-
- **Request count**: The total number of requests to the endpoint over time.
142-
- **Rate limiting recommendation**: per 10 minutes. This is guided by the request count.
143-
- **Latency**: The average origin response time in milliseconds (ms). This metric shows how long it takes from the moment a visitor makes a request to the moment the visitor gets a response back from the origin.
144-
- **Error rate** vs. overall traffic: grouped by 4xx, 5xx, and their sum.
145-
- **Response size**: The average size of the response (in bytes) returned to the request.
146-
- **Labels**: The current [labels](/api-shield/management-and-monitoring/endpoint-labels/) assigned to the endpoint.
147-
- **Authentication status**: The breakdown of which [session identifiers](/api-shield/get-started/#session-identifiers) were seen on successful requests to this endpoint.
148-
- **Sequences**: The number of [Sequence Analytics](/api-shield/security/sequence-analytics/) sequences the endpoint was found in.
149-
150-
:::note
151-
152-
Customers viewing analytics have the ability to toggle detailed metrics view between the last 24 hours and 7 days.
153-
:::
154-
155-
## Using the Cloudflare API
156-
157-
You can interact with Endpoint Management through the Cloudflare API. Refer to [Endpoint Management’s API documentation](/api/resources/api_gateway/subresources/discovery/subresources/operations/methods/list/) for more information.
158-
159-
## Sensitive Data Detection
160-
161-
Sensitive data comprises various personally identifiable information and financial data. Cloudflare created this ruleset to address common data loss threats, and the WAF can search for this data in HTTP response bodies from your origin.
162-
163-
API Shield will alert users to the presence of sensitive data in the response body of API endpoints listed in Endpoint Management if the zone is also subscribed to the [Sensitive Data Detection managed ruleset](/waf/managed-rules/reference/sensitive-data-detection/).
164-
165-
Sensitive Data Detection is available to Enterprise customers on our Advanced application security plan.
166-
167-
Once Sensitive Data Detection is enabled for your zone, API Shield queries firewall events from the WAF for the last seven days and places a notification icon on the Endpoint Management table row if there are any matched sensitive responses for your endpoint.
168-
169-
API Shield displays the types of sensitive data found if you expand the Endpoint Management table row to view further details. Select **Explore Events** to view the matched events in Security Events.
170-
171-
After Sensitive Data Detection is enabled for your zone, you can [browse the Sensitive Data Detection ruleset](https://dash.cloudflare.com/?to=/:account/:zone/security/data/ruleset/e22d83c647c64a3eae91b71b499d988e/rules). The link will not work if Sensitive Data Detection is not enabled.

‎src/content/docs/api-shield/management-and-monitoring/session-identifiers.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ pcx_content_type: how-to
33
type: overview
44
title: Session identifiers
55
sidebar:
6-
order: 2
6+
order: 3
77

88
---
99

0 commit comments

Comments
 (0)