File Stash docs #17034
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| export default { | ||
| "index": "Overview", | ||
| "files": "Working with Files", | ||
| } as const |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,193 @@ | ||
| import { Tabs } from "nextra/components"; | ||
| import Callout from '@/components/Callout'; | ||
|
|
||
| # Working with Files | ||
|
|
||
| Pipedream provides a file storage system that allows you to store and retrieve files from tool executions via Connect. When a trigger or action downloads files to the `/tmp` directory in Pipedream's execution environment, you can sync these files with a Pipedream File Store, making them accessible outside of Pipedream. | ||
|
|
||
| ## File Stash | ||
|
|
||
| When you execute an action via Connect that downloads files to the `/tmp` directory, those files normally only exist within Pipedream's execution environment. With File Stash syncing, you can now make these files available via presigned URLs that can be accessed from anywhere. | ||
|
|
||
| ### How it works | ||
|
|
||
| 1. Files created in `/tmp` during execution are synced with a Pipedream File Store when you pass `stashId` in the action execution payload | ||
| 2. Each file is assigned a presigned URL that remains valid for 30 minutes | ||
| 3. These URLs allow anyone with the link to download the file directly | ||
|
|
||
| ## When to use stashId | ||
|
|
||
| You should pass the `stashId` parameter when: | ||
|
|
||
| 1. You're working with actions that download or generate files in `/tmp`, such as actions with "file", "upload", or "download" in their name | ||
| 2. You're working with actions that have a `filePath` or `filename` prop | ||
| 3. You want to make files accessible outside of Pipedream's execution environment | ||
|
|
||
| To generate a new stash ID, you can pass: | ||
| - An empty string (`""`) | ||
| - The string `"NEW"` | ||
| - Boolean `true` | ||
|
|
||
| To reuse an existing stash ID (valid for 24 hours), pass the previously returned `stashId` value. This allows you to reference multiple files from the same stash. | ||
|
|
||
| <Callout type="info"> | ||
| Avoid passing `stashId` unnecessarily as it will slightly increase response time. | ||
| </Callout> | ||
|
|
||
| ## API Reference | ||
|
|
||
| ### Running actions with File Stash | ||
|
|
||
| To enable File Stash when running an action, use the `stash_id` parameter in your request: | ||
|
|
||
| | Parameter | Type | Description | | ||
| | --- | --- | --- | | ||
| | `stash_id` | string | (Optional) The key that identifies the external user-specific File Stash. If set to `""` (or `"NEW"` or `true`), Pipedream will generate a stash ID for you. If omitted, Pipedream will not sync files in `/tmp` with File Stash. | | ||
|
|
||
| <Tabs items={['Node.js', 'HTTP (cURL)']}> | ||
| <Tabs.Tab> | ||
| ```javascript | ||
| const resp = await pd.actionRun({ | ||
| externalUserId: "abc-123", | ||
| actionId: "google_drive-download-file", | ||
| configuredProps: { | ||
| googleDrive: { | ||
| authProvisionId: "apn_gyhLaz3" | ||
| }, | ||
| fileId: { | ||
| "__lv": { | ||
| "label": "important files > mcp-hot.jpg", | ||
| "value": "16nlbFcgtgZkxLLMT2DcnBrEeQXQSriLs" | ||
| } | ||
| }, | ||
| filePath: "/tmp/mcp-hot.jpg" | ||
| }, | ||
| stashId: "" // An empty string will generate a new stash ID | ||
| }); | ||
|
|
||
| // The response contains file URLs in $filestash_uploads | ||
| console.log(resp.exports.$filestash_uploads); | ||
| ``` | ||
| </Tabs.Tab> | ||
| <Tabs.Tab> | ||
| ```bash | ||
| curl -X POST https://api.pipedream.com/v1/connect/{project_id}/actions/run \ | ||
| -H "Content-Type: application/json" \ | ||
| -H "X-PD-Environment: {environment}" \ | ||
| -H "Authorization: Bearer {access_token}" \ | ||
| -d '{ | ||
| "external_user_id": "abc-123", | ||
| "id": "google_drive-download-file", | ||
| "configured_props": { | ||
| "googleDrive": { | ||
| "authProvisionId": "apn_jyhKbx4" | ||
| }, | ||
| "fileId": { | ||
| "__lv": { | ||
| "label": "important files > mcp-hot.jpg", | ||
| "value": "16nlbFcgtgZkxLLMT2DcnBrEeQXQSriLw" | ||
| } | ||
| }, | ||
| "filePath": "/tmp/mcp.png" | ||
| }, | ||
| "stash_id": "" | ||
| }' | ||
| ``` | ||
| </Tabs.Tab> | ||
| </Tabs> | ||
|
|
||
| ### Response | ||
|
|
||
| The response includes a `stashId` and a `$filestash_uploads` export with information about the uploaded files: | ||
|
|
||
| ```json | ||
| { | ||
| "exports": { | ||
| "$summary": "Successfully downloaded the file, \"mcp.png\"", | ||
| "$filestash_uploads": [ | ||
| { | ||
| "localPath": "/tmp/mcp.png", | ||
| "s3Key": "1day/proj_lgsqAAJ/exu_x1iK86/d4ffb5b1081d3aacd2929f23f270268c/u/mcp.png", | ||
| "get_url": "https://pipedream-file-stash-production.s3.us-east-1.amazonaws.com/1day/proj_lgsqAAJ/exu_x1iK86/d4ffb5b1081d3aacd2929f23f270267d/u/mcp.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIA5F5AGIEAUKI2SRTS%2F20250606%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250606T204541Z&X-Amz-Expires=1800&X-Amz-SignedHeaders=host&X-Amz-Security-Token=IQoJb3JpZ2luY3VjEI3%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJIMEYCIQD2OIYvdeYa%2Fo5sVMU7IIsYWF3%2FD%2BNQqEybkjaOsj5hCwIhAOxElK8xyskFD9Ta3PELIyKe1km1%2B2BXt3q%2FkGrmhjkhKr8ECGYQAxoMOTA2MDM3MTgyNzIxIgyGEDixxP3rrPl3d4IqnATAHC5Hoed80bn5mNhA07bRrqqBh3FxkWV5rdtiGI9TluRqrZfC%2FIzxnhqMMXcW0ovj8ZmIjBEDFTQRy50COfYeXFriHB5iLVC5dr%2FumYt2y%2FJEIwIaCVlDJTSkmzOOZcyzFwzGNgqKarSD1P63vySsH7LfsocM4GQKfH1KbHYKkX4GIIEAcL9T9JYU7j3zQcOE2uNpF%2BZ1fVQ8Yg0stYhMIUzSy1fLNS1CRHvejU793PSgJoKrZq8zICQFz3yL5ZxWqfrT%2BxGSZKsSH0iEOKVKq7MK0cdxrVJJsgyzl6ixiIsDKhwgmA0PhT6kvZOof0XyozdJjPAN33v2XSx%2F4BD3MrDonk4d%2F8vweQubfrOwangOPG8USZo31PXvdf8AXnx5rqVmFUL3etUsdPO2NzF6K%2B8bXNHfwgROMVG54tVGhxAX80OuflLN9lhPq%2B0%2BKS0cIC%2BpG9RNk4iToz1IFP9OWQaJPgOjOf90cPQgYfOV%2F%2FqIR9133NtKBzksB%2F%2F%2Bu1M6HS8MAfhF%2BAf9vpT%2FjvTlJhcvtiqyCzGz4TqJzxzIlFRv1dSyS08U82C7rVgOKpNWwDDqB1IjqeAZFap6tFP3s5apixPvipUERd8c8%2F9izz4%2Bz%2BD0f3Gn%2BQIRuToKQpPp%2FKfJZ15g4Mu6H4s7s7Nsr4znzdT2SOlWGi%2Bw%2FrIKi47vJfA4MKwTlW9K8e%2FsmhzHkB9LEqU7Km%2Fk36Qw8KaNwgY6nAFw%2BP4e8vTHE2MyMAZ2GiwvdlE4%2BNPtJAX4L%2BrabrgxnAHgqR0xB%2B3rNI5b62zaMrUZCm7T28Fec%2BA2x16PFLw4yUUv8UksV3r0H3J9dO6%2FrORTxYz0UYq8aiARGvg8kcjOGJ72Q5wv%2B48Up8r39coHlyACOQdd6Lg4HsohStWgeDJV0LKXru6RkNmM3FJWcWUqOy8oZxgaWe%2F%2BBAo%3D&X-Amz-Signature=a7fb66bdf5de28bf3848c25eacbc417e3a1493ed55d38a7eb68b22bef873413d" | ||
| } | ||
| ] | ||
| }, | ||
| "os": [], | ||
| "ret": { | ||
| "fileMetadata": { | ||
| "name": "mcp.png", | ||
| "mimeType": "image/png" | ||
| }, | ||
| "filePath": "/tmp/mcp.png" | ||
| }, | ||
| "stashId": "d4ffb5b1081d3aacd2929f23f270237d" | ||
| } | ||
| ``` | ||
dannyroosevelt marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| Each file in the `$filestash_uploads` array includes: | ||
|
|
||
| - `localPath`: The path to the file in the `/tmp` directory | ||
| - `s3Key`: The unique key for the file in the Pipedream File Store | ||
| - `get_url`: A presigned URL that allows downloading the file for 30 minutes | ||
|
|
||
| ## Usage Examples | ||
|
|
||
| ### Reusing a stash ID | ||
|
|
||
| To add more files to an existing stash, provide the same `stash_id` in subsequent action runs: | ||
|
|
||
| <Tabs items={['Node.js', 'HTTP (cURL)']}> | ||
| <Tabs.Tab> | ||
| ```javascript | ||
| // First run generates a stash ID | ||
| const firstRun = await pd.actionRun({ | ||
| externalUserId: "abc-123", | ||
| actionId: "download-file-1", | ||
| configuredProps: { /* ... */ }, | ||
| stashId: "NEW" | ||
| }); | ||
|
|
||
| // Save the stash ID | ||
| const savedStashId = firstRun.stashId; | ||
|
|
||
| // Later, add more files to the same stash | ||
| const secondRun = await pd.actionRun({ | ||
| externalUserId: "abc-123", | ||
| actionId: "download-file-2", | ||
| configuredProps: { /* ... */ }, | ||
| stashId: savedStashId // Reuse the existing stash ID | ||
| }); | ||
| ``` | ||
| </Tabs.Tab> | ||
| <Tabs.Tab> | ||
| ```bash | ||
| # First request with new stash | ||
| curl -X POST https://api.pipedream.com/v1/connect/{project_id}/actions/run \ | ||
| -H "Content-Type: application/json" \ | ||
| -H "Authorization: Bearer {access_token}" \ | ||
| -d '{ | ||
| "external_user_id": "abc-123", | ||
| "id": "download-file-1", | ||
| "configured_props": { /* ... */ }, | ||
| "stash_id": "NEW" | ||
| }' | ||
|
|
||
| # Get the stash_id from the response | ||
|
|
||
| # Second request using the same stash | ||
| curl -X POST https://api.pipedream.com/v1/connect/{project_id}/actions/run \ | ||
| -H "Content-Type: application/json" \ | ||
| -H "Authorization: Bearer {access_token}" \ | ||
| -d '{ | ||
| "external_user_id": "abc-123", | ||
| "id": "download-file-2", | ||
| "configured_props": { /* ... */ }, | ||
| "stash_id": "d4ffb5b1081d3aacd2929f23f270237d" | ||
| }' | ||
| ``` | ||
| </Tabs.Tab> | ||
| </Tabs> | ||
|
|
||
| ## File Storage Duration | ||
|
|
||
| Files in the File Stash are automatically deleted after 24 hours. The presigned URLs remain valid for 30 minutes from the time they are generated. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.