From 9561bef947a0c5fcb9fd868719ba5b9555518cbc Mon Sep 17 00:00:00 2001 From: Danny Roosevelt Date: Wed, 12 Feb 2025 15:57:01 -0800 Subject: [PATCH 1/7] Adding refs to proxy macros --- docs-v2/pages/connect/api-proxy.mdx | 47 +++++++++++++++++-- docs-v2/pages/connect/api.mdx | 3 +- .../connect/managed-auth/oauth-clients.mdx | 7 +-- 3 files changed, 49 insertions(+), 8 deletions(-) diff --git a/docs-v2/pages/connect/api-proxy.mdx b/docs-v2/pages/connect/api-proxy.mdx index c0e8e24779e0b..f6205a79c230b 100644 --- a/docs-v2/pages/connect/api-proxy.mdx +++ b/docs-v2/pages/connect/api-proxy.mdx @@ -33,7 +33,45 @@ You can send requests to the Connect proxy using the [Pipedream SDK](/connect/sd Refer to the full Connect API [here](/connect/api). -### Using the Pipedream SDK (preferred) +### Authenticating on behalf of your user + +Most API integrations that use OAuth to authenticate requests require that you pass a user's access token in the `Authorization` header with the `Bearer` prefix. For these apps, the Connect proxy will automatically handle that for you — you don't need to pass any reference to their OAuth access token in this case. + +For apps that require a different authentication method, you should include the necessary headers with the value surrounded by `{{ }}` in your request to the proxy, and Pipedream will automatically replace the macro with the real values and forward to the downstream API. For example: + +```javascript +/* +OpenAI requires that you pass the API key +in the `Authorization` header with the `Bearer` prefix: +*/ + +headers: { + authorization: "Bearer {{api_key}}", +} + +// Pipedream will replace the `{{api_key}}` macro with the actual API key +``` + +```javascript +/* +Zoho apps require that you pass the OAuth access token +in the `Authorization` header with a custom `Zoho-oauthtoken` prefix: +*/ + +headers: { + authorization: "Zoho-oauthtoken {{oauth_access_token}}", +} + +// Pipedream will replace the `{{oauth_access_token}}` macro with the actual token +``` + + +Refer to the relevant API's developer documentation for the correct way to authenticate requests. + + +### Making a request + +#### Using the Pipedream SDK You can use the [Pipedream SDK](https://www.npmjs.com/package/@pipedream/sdk) to send a fetch-style request: @@ -61,8 +99,9 @@ const resp = await pd.makeProxyRequest( url: "https://slack.com/api/chat.postMessage", // Include any query params you need; no need to Base64 encode the URL if using the SDK options: { method: "POST", - headers: { - hello: "world!" // These get sent to the downstream API + headers: { // These get sent to the downstream API + authorization: "Bearer {{oauth_access_token}}", // Include the special {{ oauth_access_token }} macro + hello: "world!" }, body: { text: "hello, world", @@ -76,7 +115,7 @@ const resp = await pd.makeProxyRequest( console.log(resp); ``` -### Using the REST API +#### Using the REST API You can also send a request to the Connect REST API with the below config: diff --git a/docs-v2/pages/connect/api.mdx b/docs-v2/pages/connect/api.mdx index b29f526de8d09..abeda08894333 100644 --- a/docs-v2/pages/connect/api.mdx +++ b/docs-v2/pages/connect/api.mdx @@ -147,7 +147,8 @@ External User IDs are limited to 250 characters. | API Endpoint | Rate Limit | |----------------------------|------------------------------------------------------| | `POST /tokens` | 100 requests per minute per `external_user_id` | -| `GET */accounts/*`| The sum of requests across all `*/accounts/*` endpoints must not exceed 100 requests per minute. This includes requests to `/accounts`, `/apps/:app_id/accounts`, `/accounts/:account_id`, and more — any request for account metadata and credentials is counted towards this total. | +| `GET /accounts`| 100 requests per minute per project

The sum of requests across all `*/accounts/*` endpoints must not exceed 100 requests per minute. This includes requests to,
• `/accounts`
• `/apps/:app_id/accounts`
• `/accounts/:account_id` | +| `/proxy` | 100 requests per minute per project | If you need higher rate limits, please [reach out](https://pipedream.com/support). diff --git a/docs-v2/pages/connect/managed-auth/oauth-clients.mdx b/docs-v2/pages/connect/managed-auth/oauth-clients.mdx index 657dc1144823d..317cd68b43ab7 100644 --- a/docs-v2/pages/connect/managed-auth/oauth-clients.mdx +++ b/docs-v2/pages/connect/managed-auth/oauth-clients.mdx @@ -21,9 +21,10 @@ For any OAuth app that supports it, **you can always use your own client.** Your
| Operation | Details | Environment | |--------|---------|------------------------------| -| [Retrieve credentials via API](/connect/api#accounts) | Fetch the credentials for your end user from Pipedream's API to use in your application. | ✅ `development`
❌ `production` | -| [Invoke workflows](/connect/workflows) | Trigger any Pipedream workflow and use the connected account of your end user | ✅ `development`
❌ `production` | -| [Run components via API](/connect/components) | Run any action and deploy any trigger directly from your application | ✅ `development`
✅ `production` | +| Retrieve user credentrials | [Fetch the credentials](/connect/api#accounts) for your end user from Pipedream's API to use in your app | ✅ `development`
❌ `production` | +| Invoke workflows | [Trigger any Pipedream workflow](/connect/workflows) and use the connected account of your end users | ✅ `development`
❌ `production` | +| Embed prebuilt tools | [Run any action and deploy any trigger](/connect/components) directly from your AI agent or app | ✅ `development`
✅ `production` | +| Proxy API requests | [Write custom code to interface with any integrated API](/connect/api-proxy) while avoiding dealing with user auth | ✅ `development`
✅ `production` |
## Using a custom OAuth client From beb9644eafac236c4e7aeba3943e69bc7b354307 Mon Sep 17 00:00:00 2001 From: Danny Roosevelt Date: Wed, 12 Feb 2025 15:57:32 -0800 Subject: [PATCH 2/7] Update pnpm-lock.yaml --- pnpm-lock.yaml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 4fcb44f41e7d5..e2e4005c445d9 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -12910,7 +12910,7 @@ importers: version: 3.1.7 ts-jest: specifier: ^29.2.5 - version: 29.2.5(@babel/core@7.26.0)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@7.26.0))(esbuild@0.24.2)(jest@29.7.0(@types/node@20.17.6)(babel-plugin-macros@3.1.0))(typescript@5.7.2) + version: 29.2.5(@babel/core@8.0.0-alpha.13)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@8.0.0-alpha.13))(esbuild@0.24.2)(jest@29.7.0(@types/node@20.17.6)(babel-plugin-macros@3.1.0))(typescript@5.7.2) tsup: specifier: ^8.3.6 version: 8.3.6(@microsoft/api-extractor@7.47.12(@types/node@20.17.6))(jiti@1.21.6)(postcss@8.4.49)(typescript@5.7.2)(yaml@2.6.1) @@ -45057,7 +45057,7 @@ snapshots: ts-interface-checker@0.1.13: {} - ts-jest@29.2.5(@babel/core@7.26.0)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@7.26.0))(esbuild@0.24.2)(jest@29.7.0(@types/node@20.17.6)(babel-plugin-macros@3.1.0))(typescript@5.7.2): + ts-jest@29.2.5(@babel/core@8.0.0-alpha.13)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@8.0.0-alpha.13))(esbuild@0.24.2)(jest@29.7.0(@types/node@20.17.6)(babel-plugin-macros@3.1.0))(typescript@5.7.2): dependencies: bs-logger: 0.2.6 ejs: 3.1.10 @@ -45071,10 +45071,10 @@ snapshots: typescript: 5.7.2 yargs-parser: 21.1.1 optionalDependencies: - '@babel/core': 7.26.0 + '@babel/core': 8.0.0-alpha.13 '@jest/transform': 29.7.0 '@jest/types': 29.6.3 - babel-jest: 29.7.0(@babel/core@7.26.0) + babel-jest: 29.7.0(@babel/core@8.0.0-alpha.13) esbuild: 0.24.2 ts-jest@29.2.5(@babel/core@8.0.0-alpha.13)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@8.0.0-alpha.13))(jest@29.7.0(@types/node@20.17.6)(babel-plugin-macros@3.1.0))(typescript@5.6.3): From 413d90290ed429f57164dea8db17f92626e684e1 Mon Sep 17 00:00:00 2001 From: Danny Roosevelt Date: Wed, 12 Feb 2025 15:58:04 -0800 Subject: [PATCH 3/7] Update pnpm-lock.yaml --- pnpm-lock.yaml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index e2e4005c445d9..4fcb44f41e7d5 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -12910,7 +12910,7 @@ importers: version: 3.1.7 ts-jest: specifier: ^29.2.5 - version: 29.2.5(@babel/core@8.0.0-alpha.13)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@8.0.0-alpha.13))(esbuild@0.24.2)(jest@29.7.0(@types/node@20.17.6)(babel-plugin-macros@3.1.0))(typescript@5.7.2) + version: 29.2.5(@babel/core@7.26.0)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@7.26.0))(esbuild@0.24.2)(jest@29.7.0(@types/node@20.17.6)(babel-plugin-macros@3.1.0))(typescript@5.7.2) tsup: specifier: ^8.3.6 version: 8.3.6(@microsoft/api-extractor@7.47.12(@types/node@20.17.6))(jiti@1.21.6)(postcss@8.4.49)(typescript@5.7.2)(yaml@2.6.1) @@ -45057,7 +45057,7 @@ snapshots: ts-interface-checker@0.1.13: {} - ts-jest@29.2.5(@babel/core@8.0.0-alpha.13)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@8.0.0-alpha.13))(esbuild@0.24.2)(jest@29.7.0(@types/node@20.17.6)(babel-plugin-macros@3.1.0))(typescript@5.7.2): + ts-jest@29.2.5(@babel/core@7.26.0)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@7.26.0))(esbuild@0.24.2)(jest@29.7.0(@types/node@20.17.6)(babel-plugin-macros@3.1.0))(typescript@5.7.2): dependencies: bs-logger: 0.2.6 ejs: 3.1.10 @@ -45071,10 +45071,10 @@ snapshots: typescript: 5.7.2 yargs-parser: 21.1.1 optionalDependencies: - '@babel/core': 8.0.0-alpha.13 + '@babel/core': 7.26.0 '@jest/transform': 29.7.0 '@jest/types': 29.6.3 - babel-jest: 29.7.0(@babel/core@8.0.0-alpha.13) + babel-jest: 29.7.0(@babel/core@7.26.0) esbuild: 0.24.2 ts-jest@29.2.5(@babel/core@8.0.0-alpha.13)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@8.0.0-alpha.13))(jest@29.7.0(@types/node@20.17.6)(babel-plugin-macros@3.1.0))(typescript@5.6.3): From e5c82231e517c43516436426c5111b8024748bfd Mon Sep 17 00:00:00 2001 From: Danny Roosevelt Date: Wed, 12 Feb 2025 16:42:39 -0800 Subject: [PATCH 4/7] Update api-proxy.mdx --- docs-v2/pages/connect/api-proxy.mdx | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/docs-v2/pages/connect/api-proxy.mdx b/docs-v2/pages/connect/api-proxy.mdx index f6205a79c230b..c5eeb2deffb09 100644 --- a/docs-v2/pages/connect/api-proxy.mdx +++ b/docs-v2/pages/connect/api-proxy.mdx @@ -100,7 +100,6 @@ const resp = await pd.makeProxyRequest( options: { method: "POST", headers: { // These get sent to the downstream API - authorization: "Bearer {{oauth_access_token}}", // Include the special {{ oauth_access_token }} macro hello: "world!" }, body: { @@ -136,9 +135,15 @@ You can also send a request to the Connect REST API with the below config: - If using the REST API, include the `Authorization` header with your Pipedream OAuth access token (`Bearer {access_token}`) - Headers that contain the prefix `x-pd-proxy` will get forwarded to the downstream API +- If the downstream API requires [custom authorization headers](#authenticating-on-behalf-of-a-user), make sure to prepend with `x-pd-proxy` and include the macro `{{ }}` that Pipedream will replace with the actual value. For example, + +```javascript +"x-pd-proxy-apiKey: {{api_key}}" +``` ```bash -# First, obtain an OAuth access token +# First, obtain an OAuth access token to authenticate to the Pipedream API + curl -X POST https://api.pipedream.com/v1/oauth/token \ -H "Content-Type: application/json" \ -d '{ From 7446a07709e5b09a8402266abd6b0052c1587a20 Mon Sep 17 00:00:00 2001 From: Danny Roosevelt Date: Wed, 12 Feb 2025 16:44:46 -0800 Subject: [PATCH 5/7] Update api-proxy.mdx --- docs-v2/pages/connect/api-proxy.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs-v2/pages/connect/api-proxy.mdx b/docs-v2/pages/connect/api-proxy.mdx index c5eeb2deffb09..a260e7b58e89c 100644 --- a/docs-v2/pages/connect/api-proxy.mdx +++ b/docs-v2/pages/connect/api-proxy.mdx @@ -99,8 +99,8 @@ const resp = await pd.makeProxyRequest( url: "https://slack.com/api/chat.postMessage", // Include any query params you need; no need to Base64 encode the URL if using the SDK options: { method: "POST", - headers: { // These get sent to the downstream API - hello: "world!" + headers: { + hello: "world!" // Include any headers you need to send to the downstream API }, body: { text: "hello, world", From 3cb572e016f12f1c854bae5736951a4066c22467 Mon Sep 17 00:00:00 2001 From: Danny Roosevelt Date: Wed, 12 Feb 2025 20:29:37 -0800 Subject: [PATCH 6/7] Update api-proxy.mdx --- docs-v2/pages/connect/api-proxy.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-v2/pages/connect/api-proxy.mdx b/docs-v2/pages/connect/api-proxy.mdx index a260e7b58e89c..8986e151a1f78 100644 --- a/docs-v2/pages/connect/api-proxy.mdx +++ b/docs-v2/pages/connect/api-proxy.mdx @@ -135,7 +135,7 @@ You can also send a request to the Connect REST API with the below config: - If using the REST API, include the `Authorization` header with your Pipedream OAuth access token (`Bearer {access_token}`) - Headers that contain the prefix `x-pd-proxy` will get forwarded to the downstream API -- If the downstream API requires [custom authorization headers](#authenticating-on-behalf-of-a-user), make sure to prepend with `x-pd-proxy` and include the macro `{{ }}` that Pipedream will replace with the actual value. For example, +- If the downstream API requires [custom authorization headers](#authenticating-on-behalf-of-your-user), make sure to prepend with `x-pd-proxy` and include the macro `{{ }}` that Pipedream will replace with the actual value. For example, ```javascript "x-pd-proxy-apiKey: {{api_key}}" From 407709ad821b9798ddfc1ebf6639b8667f5fa55a Mon Sep 17 00:00:00 2001 From: Danny Roosevelt Date: Thu, 13 Feb 2025 12:49:02 -0800 Subject: [PATCH 7/7] Update api-proxy.mdx --- docs-v2/pages/connect/api-proxy.mdx | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs-v2/pages/connect/api-proxy.mdx b/docs-v2/pages/connect/api-proxy.mdx index 8986e151a1f78..6443cae331702 100644 --- a/docs-v2/pages/connect/api-proxy.mdx +++ b/docs-v2/pages/connect/api-proxy.mdx @@ -163,4 +163,8 @@ curl -X POST "https://api.pipedream.com/v1/connect/{your_project_id}/proxy/{url_ }' # Parse and return the data you need -``` \ No newline at end of file +``` + +## Rate limits + +The Connect proxy limits API requests to **100 per minute per project**. [Let us know](https://pipedream.com/support) if you need higher limits.