Skip to content

[BUG] Improve Dropbox Download File action performance #16874

@TomFrankly

Description

@TomFrankly

Describe the bug
This isn't an out and out bug, but this seemed like the best category for this issue. The Dropbox → Download to TMP action is not very memory-efficient.

While the Google Drive and Microsoft OneDrive actions for downloading files to /tmp/ use streams, the Dropbox action loads the entire file into memory (potentially twice), which causes Out of Memory errors on very small files. More detail in this Claude chat: https://cloud.typingmind.com/share/d971d56e-fff5-4974-b04b-a1760cc45a72

Expected behavior
The action should be updated to use streams.

I wrote my own, which is a standalone action (it doesn't use the dropbox.app.mjs common file):

import { Dropbox } from "dropbox";
import fs from "fs";
import stream from "stream";
import { promisify } from "util";
import got from "got";

export default {
    name: "Download File to TMP",
    description: "Download a specific file to the temporary directory using streaming to avoid memory issues. Memory-efficient alternative to the official PipedreamDropbox action.",
    key: "download-file-to-tmp",
    version: "0.1.1",
    type: "action",
    props: {
        dropbox: {
            type: "app",
            app: "dropbox",
        },
        path: {
            type: "string",
            label: "File Path",
            description: `The path to the file in Dropbox (e.g., '/folder/file.mp3'). This should be **{{steps.trigger.event.path_lower}}**.`,
        },
        name: {
            type: "string",
            label: "File Name",
            description: "The new name of the file to be saved, including its extension. e.g: `myFile.mp3`. This should be **{{steps.trigger.event.name}}**.",
            optional: true,
        },
    },
    methods: {
        getDropboxClient() {
            return new Dropbox({
                accessToken: this.dropbox.$auth.oauth_access_token,
            });
        },
    },
    async run({ $ }) {
        try {
            const client = this.getDropboxClient();
            
            // Get temporary download link
            const linkResponse = await client.filesGetTemporaryLink({
                path: this.path,
            });

            if (!linkResponse || !linkResponse.result) {
                throw new Error("Failed to get temporary download link from Dropbox");
            }

            const { link, metadata } = linkResponse.result;
            
            // Determine the file extension and name
            const originalName = metadata.name;
            const extension = originalName.split(".").pop();
            
            // Use provided name or original name
            const fileName = this.name || originalName;
            
            // Clean the filename to remove problematic characters
            const cleanFileName = fileName.replace(/[\?$#&\{\}\[\]<>\*!@:\+\\\/]/g, "");
            
            // Define the tmp file path
            const tmpPath = `/tmp/${cleanFileName}`;

            // Stream download the file to avoid memory issues
            const pipeline = promisify(stream.pipeline);
            
            console.log(`Streaming download of ${originalName} to ${tmpPath}...`);
            
            await pipeline(
                got.stream(link),
                fs.createWriteStream(tmpPath)
            );

            console.log(`File successfully downloaded and saved to ${tmpPath}`);

            // Create return object that matches the official Dropbox action format
            const result = {
                tmpPath,
                name: originalName,
                path_lower: metadata.path_lower,
                path_display: metadata.path_display,
                id: metadata.id,
                client_modified: metadata.client_modified,
                server_modified: metadata.server_modified,
                rev: metadata.rev,
                size: metadata.size,
                is_downloadable: metadata.is_downloadable,
                content_hash: metadata.content_hash,
            };

            $.export("$summary", `File successfully saved in "${tmpPath}"`);

            return result;
            
        } catch (error) {
            throw new Error(`Failed to download file: ${error.message}`);
        }
    },
}

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions