Skip to content

Commit da23c75

Browse files
committed
Reorganized sources
1 parent 5bdc9f8 commit da23c75

File tree

8 files changed

+38
-28
lines changed

8 files changed

+38
-28
lines changed

src/content/docs/pipelines/build-with-pipelines/output-settings.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
---
2-
title: Customize output settings
2+
title: Configure output settings
33
pcx_content_type: how-to
44
sidebar:
55
order: 3
66
head:
77
- tag: title
8-
content: Customize output settings
8+
content: Configure output settings
99
---
1010

1111
import { Render, PackageManagers } from "~/components";

src/content/docs/pipelines/build-with-pipelines/http.mdx renamed to src/content/docs/pipelines/build-with-pipelines/sources/http.mdx

Lines changed: 3 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
11
---
2-
title: Configure HTTP Endpoint
2+
title: Configure HTTP endpoint
33
pcx_content_type: concept
44
sidebar:
55
order: 1
66
head:
77
- tag: title
8-
content: Configure HTTP Endpoint
8+
content: Configure HTTP endpoint
99
---
1010

1111
import { Render, PackageManagers } from "~/components";
1212

13-
Pipelines support data ingestion over HTTP. When you create a new pipeline, you will receive a globally scalable ingestion endpoint. To ingest data, make HTTP POST requests to the endpoint.
13+
Pipelines support data ingestion over HTTP. When you create a new pipeline using the default settings you will receive a globally scalable ingestion endpoint. To ingest data, make HTTP POST requests to the endpoint.
1414

1515
```sh
1616
$ npx wrangler@latest pipelines create my-clickstream-pipeline --r2-bucket my-bucket
@@ -56,15 +56,6 @@ curl -X POST https://<PIPELINE-ID>.pipelines.cloudflare.com \
5656
{"success":true,"result":{"committed":3}}
5757
```
5858
59-
## Turning HTTP ingestion off
60-
By default, ingestion via HTTP is turned on. You can turn it off by excluding it from the list of sources by using `--sources` when creating or updating a pipeline.
61-
62-
```sh
63-
$ npx wrangler pipelines create [PIPELINE-NAME] --r2-bucket [R2-BUCKET-NAME] --sources worker
64-
```
65-
66-
Ingestion URLs are tied to your pipeline ID. Turning HTTP off, and then turning it back on, will not change the URL.
67-
6859
## Authentication
6960
You can secure your HTTP ingestion endpoint using Cloudflare API tokens. By default, authentication is turned off. To configure authentication, use the `--require-http-auth` flag while creating or updating a pipeline.
7061
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
title: Sources
3+
pcx_content_type: concept
4+
sidebar:
5+
order: 1
6+
group:
7+
hideIndex: false
8+
---
9+
10+
Pipelines supports the following sources:
11+
* [HTTP Clients](/pipelines/build-with-pipelines/sources/http), with optional authentication and CORS settings
12+
* [Cloudflare Workers](/workers/), using the [Pipelines Workers API](/pipelines/build-with-pipelines/sources/workers-apis)
13+
14+
Multiple sources can be active on a single pipeline simultaneously. For example, you can create a pipeline which accepts data from Workers and via HTTP. There is no limit to the number of source clients. Multiple workers can be configured to send data to the same pipeline.
15+
16+
## Configuring allowed sources
17+
By default, ingestion via HTTP and from Workers is turned on. You can configure the allowed sources by using the `--source` flag while creating or updating a pipeline.
18+
19+
For example, to create a pipeline which only accepts data via a Worker, you can run this command:
20+
```sh
21+
$ npx wrangler pipelines create [PIPELINE-NAME] --r2-bucket [R2-BUCKET-NAME] --source worker
22+
```
23+
24+
:::note
25+
HTTP ingestion URLs are tied to your pipeline ID. Turning HTTP off, and then turning it back on, will not change the URL.
26+
:::

src/content/docs/pipelines/build-with-pipelines/workers-apis.mdx renamed to src/content/docs/pipelines/build-with-pipelines/sources/workers-apis.mdx

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -54,13 +54,6 @@ export default {
5454
## Accepted data formats
5555
Pipelines accept arrays of valid JSON objects. You can send multiple objects in a single request, provided the total data volume is within the [documented limits](/pipelines/platform/limits). Sending data in a different format will result in an error.
5656

57-
## Turning ingestion via a Worker off
58-
By default, ingestion via a Worker is turned on. You can turn it off by excluding it from the list of sources, by using `--sources` when creating or updating a pipeline.
59-
60-
```sh
61-
$ npx wrangler pipelines create [PIPELINE-NAME] --r2-bucket [R2-BUCKET-NAME] --sources http
62-
```
63-
6457
## Workers API
6558
### `Pipeline`
6659
A binding which allows a Worker to send messages to a Pipeline.

src/content/docs/pipelines/concepts/how-pipelines-work.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,8 @@ Cloudflare Pipelines let you ingest data from a source and deliver to a sink. It
1313

1414
### Sources
1515
Pipelines supports the following sources:
16-
* [HTTP Clients](/pipelines/build-with-pipelines/http), with optional authentication and CORS settings
17-
* [Cloudflare Workers](/workers/), using the [Pipelines Workers API](/pipelines/build-with-pipelines/workers-apis)
16+
* [HTTP Clients](/pipelines/build-with-pipelines/sources/http), with optional authentication and CORS settings
17+
* [Cloudflare Workers](/workers/), using the [Pipelines Workers API](/pipelines/build-with-pipelines/sources/workers-apis)
1818

1919
Multiple sources can be active on a single pipeline simultaneously. For example, you can create a pipeline which accepts data from Workers and via HTTP. Multiple workers can be configured to send data to the same pipeline. There is no limit to the number of source clients.
2020

src/content/docs/pipelines/getting-started.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,7 @@ Open the [R2 dashboard](https://dash.cloudflare.com/?to=/:account/r2/overview),
133133
134134
## Next steps
135135
136-
* Learn about how to [setup authentication, or CORS settings](/pipelines/build-with-pipelines/http), on your HTTP endpoint.
137-
* Send data to your Pipeline from a Cloudflare Worker using the [Workers API documentation](/pipelines/build-with-pipelines/workers-apis).
136+
* Learn about how to [setup authentication, or CORS settings](/pipelines/build-with-pipelines/sources/http), on your HTTP endpoint.
137+
* Send data to your Pipeline from a Cloudflare Worker using the [Workers API documentation](/pipelines/build-with-pipelines/sources/workers-apis).
138138
139139
If you have any feature requests or notice any bugs, share your feedback directly with the Cloudflare team by joining the [Cloudflare Developers community on Discord](https://discord.cloudflare.com).

src/content/docs/pipelines/index.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -77,11 +77,11 @@ While in beta, you will not be billed for Pipelines usage. You will be billed on
7777
***
7878
## Features
7979
80-
<Feature header="HTTP as a source" href="/pipelines/build-with-pipelines/http/">
80+
<Feature header="HTTP as a source" href="/pipelines/build-with-pipelines/sources/http">
8181
Each pipeline generates a globally scalable HTTP endpoint, which supports authentication and CORS settings.
8282
</Feature>
8383
84-
<Feature header="Workers API" href="/pipelines/build-with-pipelines/workers-apis/">
84+
<Feature header="Workers API" href="/pipelines/build-with-pipelines/sources/workers-apis/">
8585
Send data to a pipeline directly from a Cloudflare Worker.
8686
</Feature>
8787

src/content/docs/pipelines/tutorials/send-data-from-client/index.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -380,7 +380,7 @@ By default, the HTTP ingestion endpoint for your pipeline does not allow cross-o
380380
npx wrangler pipelines update clickstream-pipeline-client --cors-origins http://localhost:8787
381381
```
382382

383-
Now when you run the development server locally, and open the website in a browser, clickstream data will be successfully sent to the pipeline. You can learn more about the CORS settings in the [Specifying CORS settings](/pipelines/build-with-pipelines/http/#specifying-cors-settings) documentation.
383+
Now when you run the development server locally, and open the website in a browser, clickstream data will be successfully sent to the pipeline. You can learn more about the CORS settings in the [Specifying CORS settings](/pipelines/build-with-pipelines/sources/http/#specifying-cors-settings) documentation.
384384

385385
## 8. Deploy the application
386386

0 commit comments

Comments
 (0)