Storage Library location

[espruino-discuss]

Posted at 2018-04-20 by jonreid

On the MDBT42 currently I use Flash (for a circular logging buffer where some bytes are written twice before erase) and FlashEEPROM for storing a couple of params. I make sure they don't overlap by doing the following:

flashEEPROM = new (require("FlashEEPROM"))();
flashEEPROM.endAddr = flashEEPROM.addr+PAGE_SIZE;
  
flash = require("Flash");
logStartAddress = flashEEPROM.endAddr;
logEndAddress = logStartAddress + NUM_LOG_PAGES*PAGE_SIZE;

I am interested in swapping out flashEEPROM for the new Storage Library as want to store downloaded code to run and it seems a nice way to handle it. Can I get it to run alongside the Flash library ? If not I will just use the Flash lib to store the code.

[espruino-discuss]

Posted at 2018-04-21 by jonreid

I think I have another issue which is available flash space. Currently I have
FlashEEPROM start_address = 393216
Flash start_address = 397312, end_address = 430080

But process.memory().flash_code_start is reported as 442368. So I only have 12kb of space left which is not going to be enough for storing 2 copies of code. So I may have to rethink my strategy a bit if I cant get a little more free flash space.

Is there a way to alter .flash_code_start ? As the only initially saved() code would be my small bootloader

[gfwilliams]

Posted at 2018-04-23 by @gfwilliams

The Storage library uses the flash memory areas that are reserved for saving your code - internally saved code now also uses the storage module, so effectively the storage is being split automatically between saved code and your own data. The FlashEEPROM just tries to find an area of memory that isn't used for anything else - so yes, you should be able to run both at the same time without problems.

I'd definitely say use Storage for your second copy of the code. It should be pretty straightforward to use - and actually if you just eval(require("Storage").read(...)) then it'll be pretty efficient.

All I'd say is watch out if you then delete the second copy of the firmware - doing the eval as above will actually leave the function code in flash memory, so if you then erase flash all the code for the functions in RAM will disappear. If you want to avoid that then you can do eval(require("Storage").read(...)+"") to force the code to be loaded right into RAM though.

[espruino-discuss]

Posted at 2018-04-23 by jonreid

OK sounds good ! So the Storage lib is just using from flash_code_start onwards ? And so it wont clash with the Flash lib I have set to work between 397312 and 430080

[gfwilliams]

Posted at 2018-04-23 by @gfwilliams

So the Storage lib is just using from flash_code_start onwards ?

Yes, that's right. So yes, as long as you stick between flash_binary_end and flash_code_start you should be fine.

[espruino-discuss]

Posted at 2018-04-24 by jonreid

What do you mean by your earlier comment of running from flash that 'it will be pretty efficient' ?

What are the downsides to running from flash vs loading to RAM ? (Apart from the deletion issue you pointed out)

[gfwilliams]

Posted at 2018-04-24 by @gfwilliams

What do you mean by your earlier comment of running from flash that 'it will be pretty efficient' ?

It'll be efficient with RAM - also it's never loading the string of code to be executed into RAM either.

What are the downsides to running from flash vs loading to RAM ? (Apart from the deletion issue you pointed out)

There really aren't many... If you've turned pretokenisation on then it won't have an effect, but otherwise it's great - it just saves loads of RAM because the contents of every function is executed straight from flash.

[espruino-discuss]

Posted at 2018-05-03 by jonreid

OK so I have my MEDBT42 bootloader up and running now. It loads new code in from another module via the serial port, saves to Storage and then eval.

What I started with was keeping a copy of the previous working copy also, but I found then on writing the second, new copy of code on eval the code was corrupt and would not run. The code size is 10.5kB.

I am also saving the bootloader code from the IDE using 'save on send' and I have also observed the bootloader code getting corrupted when saving both copies of main code.

I would not have thought that I am running out of space with my bootloader of 3k and 2x 10.5kb there would be a size issue that would cause the corruption ? Any thoughts ?

[gfwilliams]

Posted at 2018-05-03 by @gfwilliams

It could be what I mentioned before I guess - that when you save new code into storage, the storage library tries to compact, which rearranges things in ROM - and any code that was executed from flash would have had the function code stored in ROM (which would have then changed position).

If you write your bootloader code like this and upload:

eval(`
Your
code
in
here`);

Then at the very least your bootloader will remain intact - and you can use eval(require("Storage").read(...)+"") in the bootloader to evaluate the code you want to run (which should cause the functions to all be loaded into RAM).

[espruino-discuss]

Posted at 2018-05-08 by jonreid

So could the compact/rearrange occur during the first eval ? I am not sure if this is the problem, as even if I reboot and try to eval the code later it still fails. So what I was doing originally was

1. Save bootloader from IDE (not using Save on Send so should be all in RAM ?)
2. Bootloader runs code ver 1 by eval (This was previously written to storage by bootloader)
3. Code ver is running, is notified of software update so reboots to bootloader (E.enableWatchdog(1); while(1);)
4. Bootloader receives code v2 via serial port, writes to Storage
5. Bootloader eval code v2 but fails due to code errors. Even if I reboot it still fails.

If I delete code v1 before writing v2 to Storage, it works. Not sure if that shows something I am missing.

I will try your suggestion of eval to RAM as well

[gfwilliams]

Posted at 2018-05-09 by @gfwilliams

So could the compact/rearrange occur during the first eval?

If you're saving anything to storage then yes. However any function that you define inside that first eval (from flash) will have problems if the flash layout then changes.