HTTPS on Pixl

[espruino-discuss]

Posted at 2020-08-03 by sammachin

So I know its not currently possible but I'm wondering what options I have for this.
Currently I've got a board with a few neopixels and an ESP8266 on it that plugs onto a pixl, that then makes requests to an API to get some data which is displayed on the pixl and LEDs.

The server I'm connecting to only offers HTTPS using a self signed cert and its on an internal network,there are only 2 endpoints I need to get JSON from.
Currently I'm using a very simple NodeRED flow to act as a reverse proxy and take in a plain HTTP request from the pixl then forward on over HTTPS, but this isn't ideal long term, I'd like to have everyting self contained on the device.

I don't really want to swap the Pixl for an Espruino WiFi, the screen buttons and BLE are all useful parts of the design.
Could I replace the ESP8266 with an ESP32 but use this to make the HTTPS requests using its own internal client, I guess I'd need some custom code to talk serial to the E32 and then have it make the requests? Anyone done that? I guess running a second copy of espruino on the ESP-32 would be one way to go
Whats the fundimental blocker with getting HTTPS from teh nRF52 is it just not capable? Is it a question of time/money?
Any other ideas?

[gfwilliams]

Posted at 2020-08-03 by @gfwilliams

Hi!

The only issue with HTTPS on Pixl.js is really the memory. By default mbedtls allocates big RAM buffers - they're needed for the spec but in reality you may not need them if sending small buffers (AFAIK). Then there's flash - but you can remove some stuff that's not needed there to make space.

So... You could just try compiling in HTTPS support (edit board.py, add 'TLS' under libraries and ) and maybe remove FILESYSTEM and see if it works - now you can save code to flash there's a lot more RAM than there was previously. If you need a hand let me know.

If you run out of RAM, you could change MBEDTLS_SSL_MAX_CONTENT_LEN to 1024 at https://github.com/espruino/Espruino/blob/e06b3e24de42adb1de6e2cc6046e525880760e7d/libs/crypto/mbedtls/include/mbedtls/ssl.h#L234 and see if you have any success there.

However, it looks like newer ESP8266 firmwares actually support HTTPS (search for 'SSL' in
https://www.espressif.com/sites/default/files/documentation/4a-esp8266_at_instruction_set_en.pdf) so you could do that.

While it's not supported by Espruino firmware at the moment you could just create your own version of the esp8266 driver: https://github.com/espruino/EspruinoDocs/blob/master/devices/ESP8266WiFi_0v25.js#L89

and replace TCP with SSL...

[gfwilliams]

Posted at 2020-08-03 by @gfwilliams

Ok, just had a quick look...

I've just built a PIXLJS firmware without filesystem and with SSH with the reduced buffer size (attached). I'd be interested to see how it goes :)

Attachments:

• espruino_2v06.78_pixljs.zip

[espruino-discuss]

Posted at 2020-08-03 by sammachin

Tried with that build on the pixl and just updating the url to https but it still seems to be making the request over http, this is my code, do I need to do something else?

digitalWrite(D12,1); // enable ESP8266
Serial1.setup(115200, { rx: D2, tx : D3 });
var wifi = require("ESP8266WiFi_0v25").connect(Serial1, function(err) {
  if (err) throw err;
  console.log("Connecting to WiFi");
  wifi.connect(SSID, PASSWORD, function(err) {
    if (err) throw err;
    console.log("WiFi Connected");
    getCharge();
  });
});




function getCharge(){
  var http = require("http");
  http.get("https://192.168.1.127/api/system_status/soe", function(res) {
    var contents = "";
    res.on('data', function(data) { contents += data; });
    res.on('close', function() {
      var d = JSON.parse(contents);
      var charge = Math.round(d.percentage);
      g.clear();
      // Use the small font for a title
      g.setFontBitmap();
      g.drawString("Charge:");
      // Use a large font for the value itself
      g.setFontVector(40);
      g.drawString(`${charge}%`, (g.getWidth()-g.stringWidth(`${charge}%`))/2,10);
      // Update the screen
      g.flip();
    });
  });
}

[gfwilliams]

Posted at 2020-08-05 by @gfwilliams

Looking into this now - looks like I messed up the build :)

[gfwilliams]

Posted at 2020-08-05 by @gfwilliams

Ok, try this! I had to cut a bunch of stuff out to get it to fit (like the vector font :( ) but it does work!

Potentially with a bit more fiddling we could get the vector font in and just remove a few other things you didn't need.

Attachments:

• espruino_2v06.79_pixljs.zip

[espruino-discuss]

Posted at 2020-08-05 by sammachin

Ok tried that with a modified bit of code to remove the vector font and just log to console but I get an error

TLS check
http
TLS yes?
=undefined
Uncaught InternalError: Failed! mbedtls_ssl_handshake returned -0x7780

[gfwilliams]

Posted at 2020-08-05 by @gfwilliams

Hi - it's worth googling mbedtls_ssl_handshake 0x7780 (or whatever error you get).

Can you try https://pur3.co.uk/hello.txt just as a test? Most likely it's your HTTPS server doesn't support any of the cipher suites that the Pixl.js does :(

[espruino-discuss]

Posted at 2020-08-06 by sammachin

I did some more testing last night;

Firstly trying to fetch the pur3 page with the pixl returns an out of memory error

TLS check
http
TLS yes?
=undefined
Uncaught InternalError: Failed! mbedtls_ssl_handshake returned Not enough memory

Then I tried with my EspruinoWiFi, fetches the pur3 page fine but on the internal IP I get the same 0x7780 error, with a bit of googling this error seems to be related to the certificate chain, as my device uses a self signed cert this makes sense.

So there are now 2 problems:
The pixl still can't make an HTTPS request even to a 'good' host
Espruino doesn't like self signed certs.
Is there any way to set a flag to ignore the certificate validation and just accept the cert, like the -k option in curl?

[gfwilliams]

Posted at 2020-08-06 by @gfwilliams

How are you uploading code? Could you try uploading to flash and see if that gives you enough extra RAM?

Potentially we could rebuild with smaller buffers but we're at 4096 right now and 1024 doesn't work.

I'm afraid I don't know enough about mbedtls - you'd have to see if you could dig around. I think there's a possibility that the server having its own cert means that it's forced into using a cipher suite that the current build of mbedtls doesn't support?

[espruino-discuss]

Posted at 2020-08-22 by sammachin

Did a bit more digging on this and some testing with my espriunio WiFi.
The 0x7780 error seems to relate to a problem with the certificate chain, likely due to the server using a self signed cert. I get this error thrown with the EsprinoWiFi as well as the Pixl.
I'm a bit out of my depth but supposedly if you call mbedtls with MBEDTLS_SSL_VERIFY_NONE set then it will ignore the cert chain, I think this line says that should be set in esprunio though? https://github.com/espruino/Espruino/blob/master/libs/network/network.c#L599

Yes I was uploading to flash already, seems like this might be just too much for the nRF52 to deal with on its own, I think I need to look at offloading the SSL to something else like an ESP-32 instead of the ESP-8266 that I'm using anyway
Its rather annoying that Tesla are forcing the powerwall to use SSL with a self signed cert, the whole project works really nicely if I could just overcome that issue.

[gfwilliams]

Posted at 2020-08-24 by @gfwilliams

Yes, I'm pretty sure MBEDTLS_SSL_VERIFY_NONE is already set.

Maybe you could find out what Cipher is being used when you connect via your Raspberry Pi/PC? If it's not something supported by the current Espruino mbledtls build then that'd be a good place to start?

I fact I imagine that even compiling Espruino to run on Linux or a Raspberry Pi would expose the same problems, and might be an easier way of tracking down what's wrong?

When I'd tried the Pixl I'd been doing it with a WIZnet Ethernet shield - I'm sure we could get HTTPS to work with ESP8266 with a bit of fiddling, but it's probably worth making sure that you can get a good connection with the Espruino WiFi first before looking into it too much more.

Or as I'd mentioned in http://forum.espruino.com/conversations/351824/#15444448 if you've got up to date firmware on the ESP8266 you might be able to get it to use HTTPS with a normal Pixl.js firmware just by replacing "TCP" with "SSL" in the WiFi driver.