Skip to content

Backpressure propagation #375

@sigJoe

Description

@sigJoe

Hi I recently worked on an app using Node 22 busboy 1.6.0 and found that the app sometimes hitting OOM when handling uploads for large files (e.g. 1 GB). Heap dumps showed all the bloat is coming from busboy's FileStream._readableState.buffer, where it is trying to store almost the entire file upload.

The app pipes busboy's filestream (file.pipe(writeStream)), and I saw backpressure caused by a slow writestream was (as expected) pausing the filestream, but then busboy in turn was not pausing the original request. Where the app flow is request -> busboy filestream -> writestream, the request was continuing to push chunks to the filestream.

A quick fix was to add this snippet to my app:

file.on('pause', () => {
    if (req.readable && !req.isPaused()) {
        req.pause();
    }
});
file.on('resume', () => {
    if (req.readable && req.isPaused()) {
        req.resume();
    }
});

It took me a while to figure out where the problem was, so I thought I'd ask is this expected behaviour or not?

If this is expected, then I suggest adding a warning or similar code snippet to examples in the readme.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions