Pipe complete function not called

[espruino-discuss3]

Posted at 2015-03-10 by net-tobi

Dear community,

I tried to pipe an http response directly to a file to save RAM. I experienced that sometimes the file contains the right content, sometimes it contains strange characters. Furthermore the "complete" function defined in the pipe options is never getting called.

I formatted the SD card before I tested and I typed E.unmountSD() to be sure that it is not a problem with the SD card filesystem.

Could you please help me? Below you find the function.

Best,

Tobias

var downloadFile = function(path, filePath) {
    var file = E.openFile(filePath, 'w');
    var body = "";
    var options = {
        method: 'GET',
        host: '192.168.0.11',
        port: 3000,
        path: path,
        headers: {
            "Content-Type": "application/json",
            "Content-Length": body.length
        }
    };
    var req = require('http').request(options, function(res) {
      res.pipe(file, { chunkSize : 32, end: true, complete: function() { console.log('Complete'); } });
      res.on('close', function() {
        console.log('Closed.');
      });
    });
    req.write(body);
    req.end();
};

[espruino-discuss3]

Posted at 2015-03-10 by net-tobi

No idea?

---

Posted at 2015-03-11 by @gfwilliams

To be fair, you only gave me 11 hours to answer :)

Looks like this was a bug - pipes only closed when the destination closed, rather than the source. I've fixed it in GitHub now, however it's obviously pretty easy to work around by closing the file in the close hander.

Also, if you check E.getErrorFlags() I bet it'll tell you ["BUFFER_FULL"]. It's because the HTTP client was grabbing data as fast as it could, but the pipe was only dealing with it in 32 byte chunks. I've fixed this now, but if you want to get it working with the current version I think you could change chunkSize to 512.

---

Posted at 2015-03-11 by net-tobi

Perfect, thank you!

One additional question while we are speaking about http:

Is there a way to get the statusCode of the responses? res.statusCode is set to undefined.

---

Posted at 2015-03-11 by net-tobi

I tested again with your recent build from GitHub and it looks much better, but there are 12 bytes missing at the end of the file. It looks like it is getting closed too early.

---

Posted at 2015-03-11 by @gfwilliams

There's no way to get it at the moment, but again - I've just updated it so it'll be in the next version

---

Posted at 2015-03-18 by net-tobi

Hello again,

I still have the problem that 12 bytes are missing at the end of my file, when I use the pipe function like I wrote in my last post. Any ideas what I could do to get it working? I use the code above with a chunkSize of 512.

Best,

Tobias

---

Posted at 2015-03-19 by @gfwilliams

Just to check, does E.getErrorFlags() report anything wrong, or is it just []?

---

Posted at 2015-03-19 by net-tobi

No, nothing is getting reported. It is [].

---

Posted at 2015-03-21 by net-tobi

The other direction is as well not working. I cannot pipe to requests like:

file.pipe(req);

If I do that, the connection is getting established to the server, but no data is sent.