File uploading finishes early or not all chunks receiving

[dalisoft]

Hi @alexhultman

I found strange bug, when uploading file, not all content will be uploaded, e.g, sometimes 70%, sometimes 90%, sometimes 100%.

Code

Example code used from here

const uWS = require("../dist/uws.js");
const fs = require("fs");
const path = require("path");
const port = 9001;

const app = uWS
  ./*SSL*/ App({
    key_file_name: "misc/key.pem",
    cert_file_name: "misc/cert.pem",
    passphrase: "1234"
  })
  .post("/audio.mp3", (res, req) => {
    console.log("Posted to " + req.getUrl());
    const fileStream = fs.createWriteStream(
      path.resolve("./examples/audio.mp3")
    );
    res.onData((chunk, isLast) => {
      /* Buffer this anywhere you want to */
      fileStream.write(Buffer.from(chunk));

      if (isLast) {
        fileStream.end();
        res.end("File Uploaded");
      }
    });

    res.onAborted(() => {
      /* Request was prematurely aborted, stop reading */
      fileStream.destroy();
      console.log("Eh, okay. Thanks for nothing!");
    });
  })
  .listen(port, token => {
    if (token) {
      console.log("Listening to port " + port);
    } else {
      console.log("Failed to listen to port " + port);
    }
  });

Calling screenshot

Env

OS: macOS (10.15.2)
Node: 12.14.0 / 6.13.4
HTTP Client: Postman (7.14.0)
uWebSockets.js: 16.4.0

[ghost]

You're constructing a Buffer from an ArrayBuffer and passing that Buffer to Node.js. The ArrayBuffer is only valid in this callback, but Node.js's write method may take more time to actually write the buffer to disk.

You probably solve this by copying the buffer, not just referencing it.

[dalisoft]

Thanks, i will try

[ghost]

fileStream.write(Buffer.from(chunk));

You're giving a reference to something that gets invalidated once the callback returns, to an async function that may or may not be finished in one go. If you do Buffer.concat([Buffer.from(chunk)]) you get a copy.

[dalisoft]

Buffer.concat([Buffer.from(chunk)]) solved my issue, thanks a lot

[rizrmd]

Turns out you cannot call await before calling res.onData, please make sure to move your await after res.onData is called.

Spent 5 hours straight solving this problem, need to put this here to warn future reader.