Getting POST data in an HTTP server ...

[espruino-discuss]

Posted at 2016-09-20 by Kolban

My goal is to write a script that sets up an HTTP server and when I post data in from a client, I want to see that data within Espruino. To that end, I set up the following script:

var wifi = require("Wifi");
var esp8266 = require("ESP8266");
var http = require("http");
wifi.startAP("ESP8266", {
  "authMode": "open"
}, function(err) {
  var httpServer = http.createServer(function(request, response) {
    console.log("Received a server request!" + JSON.stringify(request));
    if (request.available() === 0) {
      console.log("No data");
    } else {
      var receivedData = request.read();
      console.log("Data = " + receivedData);
    }
    response.write("Response data!");
    response.end();
  });
  httpServer.listen(80);
});

I then send in a POST request using:

wget http://192.168.4.1/hello  --post-data="hello" --quiet --output-document=-

when the request is received by the ESP8266, the message logged is:

Received a server request!{"type":1,"res":{},"svr":{"type":1,"port":80,"sckt":9},"sckt":10,"headers":{"User-Agent":"Wget/1.16 (linux-gnueabihf)","Accept":"*/*","Host":"192.168.4.1","Connection":"Keep-Alive","Content-Type":"application/x-www-form-urlencoded","Content-Length":"5"},"method":"POST","url":"/hello","hdrs":true,"dRcv":""}
No data

From this note two things. The first is that the amount of available bytes is flagged as 0 while the HTTP header says that 5 bytes (what I would expect) was sent.

If I change the code to include:

request.on("data", function(data) {
   console.log(data);
});

I do in fact see the data. What is puzzling me is the notion that I had expected available() to return 5 for the amount of payload data and response.read() to be able to read that data. Can anyone spot what I might have mis understood?

[espruino-discuss]

Posted at 2016-09-20 by @gfwilliams

Maybe I'm misundertanding, but when it's called the data hasn't arrived yet so available data reports 0? Espruino's event based, so you can't do like on Arduino and just block execution until data arrives. Using the callback as you suggested at the end is the right way to handle it.

Content-Length is sent before everything else, so it's hardly surprising you get that in the header, before you have the 5 bytes.

---

Posted at 2016-09-20 by ClearMemory041063

Have a look at this code for some ideas on Post and Get

http://forum.espruino.com/conversations/292196/#comment13195299

---

Posted at 2016-09-20 by Kolban

@gfwilliams
Thank you, thank you, thank you ... that explains it all ... based on your pointer, I found the following excellent document on how Node.js handles HTTP transactions:

https://nodejs.org/en/docs/guides/anatomy-of-an-http-transaction/

What you say makes perfect sense now and is (in my opinion) the correct implementation. I'm making notes for what might be a programmer's guide to using Espruino and will cover this area. I had mistakenly assumed that Espruino would have received the whole HTTP transaction before invoking the HTTP server side callback ... but now I understand all it would have to do is receive the HTTP headers before invoking the callback and then the callback has a "stream" reader on the rest of the data. That data may or may not have arrived at the time when the callback is made.

... later ...
For completeness, and it is likely most users will know this ... but if we want to process a complete transaction of data, then we can register a callback for the "close" event of the request at which time both available() and read() will be honored to return the total length and all the data. For example:

  var httpServer = http.createServer(function(request, response) {
    console.log("Received a server request!" + JSON.stringify(request));
    request.on("close", function() {
      console.log("The request has completed sending its data!");
      console.log("Available data is: " + request.available());
      var data = request.read();
      console.log("The data is: " + data);
    });
    response.write("Response data!");
    response.end();
  });
  httpServer.listen(80);

---

Posted at 2017-12-04 by @gfwilliams

For anyone else who gets here, there's now a tutorial on HTTP POST and Forms: http://www.espruino.com/Posting+Forms