Adjusting variables when building custom firmware

[espruino-discuss]

Posted at 2020-07-28 by parasquid

I'm trying to get https://www.espruino.com/MPU6050 working on my Pixl.js and I'm running into memory issues when doing the MPU6050_DMP example. Looking at https://www.espruino.com/modules/MPU6050_DMP.js and the minified version https://www.espruino.com/modules/MPU6050_DMP.min.js it seems that I'll need to cut down on the unnecessary libraries included in the firmware.

I've removed many of the libraries and only left BLUETOOTH and PIXLJS and have reduced the size of the DFU zip to just 312k but it looks like I also would need to adjust the variables section to actually make use of the freed space.

How do I calculate the actual amount of variables I can set?

process.memory() gives me this:

>process.memory()
={ free: 1733, usage: 767, total: 2500, history: 8,
  gc: 0, gctime: 4.30297851562, blocksize: 16, "stackEndAddress": 536929600, flash_start: 0,
  "flash_binary_end": 444468, "flash_code_start": 446464, flash_length: 524288 }

so does that mean 524288 - 446464 = 77824 variables I can use for the variables section? (it was originally set to 2500) or should I just do a trial and error to find the thresholds?

[espruino-discuss]

Posted at 2020-07-28 by parasquid

Hm I might be conflating RAM and Flash here. I'm guessing that variables would be related to RAM and the firmware size would be related to Flash.

And I'm guessing if I can somehow get the constants in that library be a lookup table located in storage, I should be able to free up more memory for actual code.

[gfwilliams]

Posted at 2020-07-28 by @gfwilliams

Ahh - yes, so you can remove libraries and then adjust the PIXLJS.py file to increase the available flash storage, but generally we try pretty hard to ensure that libraries don't use RAM so it won't actually free up any more variables.

I've just taken a look at that module and there's a load of room for improvement (for instance it stores the data in RAM even though it's only used on initialisation).

So, I made some changes. I don't have one here to test with so haven't made it live yet, but please could you try:

// replace
require("MPU6050_DMP")
// with
require("https://raw.githubusercontent.com/espruino/EspruinoDocs/master/devices/MPU6050_DMP.js")

Then ensure that module minification is on (the default) and that you're saving to flash.

The minified file is down from 11k to 7k and RAM usage should be down massively. If that works I'll make the modified file live on the website.

[espruino-discuss]

Posted at 2020-07-28 by parasquid

Ah the heatshrink technique is nice! :)

I'm still getting memory issues though.

>reset()
=undefined
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v06 (c) 2019 G.Williams
No errors in https://raw.githubusercontent.com/espruino/EspruinoDocs/master/devices/MPU6050_DMP.js. Minified 34114 bytes to 9872 bytes.
No errors. Minified 15565 bytes to 15435 bytes.
>
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v06 (c) 2019 G.Williams
>Uncaught SyntaxError: Got EOF expected '}'
 at line 1 col 13307
...etMemoryBank(b),this.setMem
                              ^
Execution Interrupted
New interpreter error: LOW_MEMORY,MEMORY
> 

and that's with the minification options Esprima(offline) for both modules and editor.

Enabling mangling gives me:

No errors in https://raw.githubusercontent.com/espruino/EspruinoDocs/master/devices/MPU6050_DMP.js. Minified 34114 bytes to 8640 bytes.
No errors. Minified 14333 bytes to 13937 bytes.
>
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v06 (c) 2019 G.Williams
>Uncaught Error: Function "setClockSource" not found!
 at line 1 col 6
this.setClockSource(b.CLOCK_PLL_XGYRO),this.setFullScaleAcce...
     ^
in function "initialize" called from line 1 col 103
...AD0_HIGH:a,this.initialize()
                              ^
in function "a" called from line 1 col 10
new a(b,c)
         ^
in function "connect" called from line 1 col 13768
...ire('MPU6050').connect(I2C1);const DMP=require('https://raw....
                              ^
New interpreter error: LOW_MEMORY,MEMORY

and turning on pretokenise:

No errors in https://raw.githubusercontent.com/espruino/EspruinoDocs/master/devices/MPU6050_DMP.js. Minified 34114 bytes to 8640 bytes.
Error parsing JavaScript, but uploading anyway.
SyntaxError: Unexpected token (1:126)
Error: Line 1: Unexpected token {
> 

I'll try playing around with the closure minifiers as well

[espruino-discuss]

Posted at 2020-07-28 by parasquid

Here's the code I'm using btw (the same as in the docs, just the I2C options and trigger changed)

I2C1.setup({scl:A5, sda:A4});

const MPU = require("MPU6050").connect(I2C1);
// 2nd parameter is the fifoRate. The DMP output frequency = 200Hz / (1 + fifoRate)
const DMP = require("https://raw.githubusercontent.com/espruino/EspruinoDocs/master/devices/MPU6050_DMP.js").create(MPU, 3);

function pidLoop(args) {
  var data = DMP.getData();
  if(data !== undefined) console.log(DMP.getYawPitchRoll(data));
}

setWatch(pidLoop, D13, { repeat:true, edge:'rising' });

[espruino-discuss]

Posted at 2020-07-28 by parasquid

closure with simple optimizations gives

No errors in https://raw.githubusercontent.com/espruino/EspruinoDocs/master/devices/MPU6050_DMP.js. Minifying 34114 bytes to 8589 bytes
No errors. Minified 14282 bytes to 13871 bytes.
>
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v06 (c) 2019 G.Williams
>Execution Interrupted
New interpreter error: LOW_MEMORY,MEMORY
> 

[espruino-discuss]

Posted at 2020-07-28 by parasquid

setting both editor and module to closure simple optimizations works!

No errors. Minifying 14282 bytes to 12743 bytes
>
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v06 (c) 2019 G.Williams
>Resetting MPU6050...
OTP bank is valid!
Success! DMP code written and verified.
Success! DMP configuration written and verified.
FIFO count: 84 Data: 63,255,238,253,0,46,156,106,255,254,101,6,0,1,165,107,255,254,76,31,255,255,88,89,0,0,67,14,255,248,58,153,255,125,61,218,17,187,240,115,178,106,63,255,245,148,0,36,48,162,255,251,206,109,0,2,164,50,255,254,76,31,255,255,77,44,0,0,67,14,255,242,133,74,255,88,179,152,23,30,78,155,178,106
IntStatus: 7
FIFO count: 210 Data: 63,253,254,164,254,255,215,54,0,8,189,86,0,3,123,84,255,254,76,31,255,255,65,254,0,0,78,59,255,240,234,183,255,61,205,82,25,253,25,113,178,106,63,254,25,27,255,6,103,126,255,254,244,193,0,4,124,176,255,254,87,76,255,255,77,44,0,0,67,14,255,248,200,130,255,47,96,246,27,142,83,194,178,106,63,254,108,71,255,28,202,54,0,4,39,61,0,5,178,250,255,254,76,31,255,255,77,44,0,0,67,14,255,253,191,163,255,41,161,249,28,105,250,127,178,106,63,254,17,157,255,6,135,65,0,31,114,164,0,7,70,45,255,254,76,31,255,255,65,254,0,0,67,14,255,245,96,37,255,40,106,35,28,225,39,172,178,106,63,254,60,201,255,18,150,132,0,36,61,34,0,8,166,21,255,254,87,76,255,255,65,254,0,0,67,14,255,240,89,161,255,36,214,214,29,40,147,213,178,106
IntStatus: 3
DMP is good to go! IntStatus: 3

I guess it just barely gets it.

However, I'm planning to use this to stream accelerometer data over BLE so I'm afraid even if the example works, adding more of my own code might take me over the memory limit :(

I'll see what I can do in the meantime :) Thanks!

[espruino-discuss]

Posted at 2020-07-28 by parasquid

I might have spoken too soon; closing and then reopening the Espruino editor gives me a memory error again:

No errors in https://raw.githubusercontent.com/espruino/EspruinoDocs/master/devices/MPU6050_DMP.js. Minifying 34114 bytes to 8589 bytes
No errors. Minifying 14282 bytes to 12744 bytes
>
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v06 (c) 2019 G.Williams
>Resetting MPU6050...
New interpreter error: LOW_MEMORY,MEMORY
OTP bank is valid!
Uncaught Error: Function or String not supplied!
 at line 1 col 353
...r();setTimeout(e,20,a,c,g,h)
                              ^
in function called from system

[gfwilliams]

Posted at 2020-07-28 by @gfwilliams

Ahh - can you check "Modules uploaded as functions" is checked under communication settings, and that you're definitely uploading to flash?

I just uploaded your code to a Pixl here and it went fine (up to the point that I2C failed because I didn't have an MPU6050 attached) with only ~900 vars used.

[espruino-discuss]

Posted at 2020-07-28 by parasquid

Strangely, trying to upload the code again after the memory problem works. I guess I'll work with this for now :)

[espruino-discuss]

Posted at 2020-07-28 by parasquid

Ah I was uploading to RAM. Uploading to flash works :) (I didn't upload to flash because I had to reset the Pixl when the memory issues came up during upload)

And yeah, modules uploaded as functions is checked.

The module seems to be working, and is quite performant :) Thanks again for the help!

[gfwilliams]

Posted at 2020-07-28 by @gfwilliams

No problem. Can you try moving back to the website modules now?

I2C1.setup({scl:A5, sda:A4});
const MPU = require("MPU6050").connect(I2C1);
// 2nd parameter is the fifoRate. The DMP output frequency = 200Hz / (1 + fifoRate)
const DMP = require("MPU6050_DMP").create(MPU, 3);
function pidLoop(args) {
  var data = DMP.getData();
  if(data !== undefined) console.log(DMP.getYawPitchRoll(data));
}
setWatch(pidLoop, D13, { repeat:true, edge:'rising' });

I also found out that the MPU6050 module wasn't minifying quite right, so now it should be a bit faster and use a bit less space too!

[espruino-discuss]

Posted at 2020-07-28 by parasquid

Works well! :)