Storing usercode in repl-accessible file

[jakeg]

Running .ls shows no files, and .pwd shows we're in root. Where is the usercode which is sent to the Pi Pico via kaluma flash -w stored?

If putting MicroPython on the Pi Pico, the usercode is stored in main.py in the root. It would be very useful if Kaluma did something similar (suggest index.js instead). The .ls command etc would then make more sense.

I'm asking because I'm thinking of flashing files to Kaluma in a similar way to how it works with MicroPython, which is much quicker!

Try https://www.mathsuniverse.com/pico for Kaluma JS and https://www.mathsuniverse.com/pico/python.html for Python. Try flashing a file with both and note how much quicker it is with Python.

With Kaluma JS, the file is sent using a ymodem implementation. With MicroPython, raw repl mode is entered and the code is just pasted in, then saved to main.py by MicroPython itself, then a command is then sent to soft-reset.

So I'm thinking if the usercode is stored in the repl-accessible folder as index.js then we could use a similar method, using fs = require('fs') etc. The ymodem implementation could then be deprecated. My guess is that this would be much faster as well, like it is with MicroPython.

[niklauslee]

The user code is stored directly in flash not in file system. Please see https://kalumajs.org/docs/boards/pico#flash
We wanted to store in filesystem, but JerryScript has no way to load js code from file system.

[jakeg]

Ok, so I found out how it's possible!

Let's consider 2 files: flash.js and index.js.

flash.js is flashed just once, and includes bootstrapping code to read and write to index.js and load it.

index.js has usercode, ie whatever code the user wants to run. We want to be able to send this code to the Pico and load it as fast as possible.

So flash.js includes this (or similar), and only needs to be flashed to the Pico once:

// flash this file once to Kaluma. It in turn loads index.js stored in user-accessible /
let fs = require('fs')

function writeFile (fileName, string) {
  return fs.writeFile(fileName, new TextEncoder().encode(string))
}

function readFile (fileName) {
  return new TextDecoder().decode(fs.readFile(fileName))
}

function loadFile (fileName) {
  try {
    let code = readFile(fileName)
    let func = new Function(code)
    func()
  } catch (e) {
    console.log('No index.js to load or other error')
    console.error(e)
  }
}

if (fs.exists('/index.js')) loadFile('/index.js')

The functions writeFile(), readFile() and loadFile() are now accessible in the REPL. They're all that's needed to put usercode ins index.js and have it loaded instantly and on boot.

I've tried this with very short files and it works perfectly and ridiculously fast. With the flash.js flashed, in the REPL do:

writeFile('/index.js', `setInterval(() => console.log('hi'), 100)`)
.load

... this of course persists a reboot.

When trying with larger files I'm currently getting a REPL buffer overflow but I presume I just need to send ./index.js in chunks or something and append lines as they're received to flash.js's writeFile() instead of writing the whole file at once.

If this can work with larger files, then flash.js can just be part of Kaluma's .uf2 install, meaning it doesn't need to be done once by users.

[jakeg]

Ok so this works! Using this code means the code sent to the Pico runs pretty much instantly (vs several seconds using the ymodem implementation).

// Flash this file once to Kaluma
// It in turn loads index.js, which users can update much quicker

let usercode = (() => {
  let fs = require('fs')
  let file = null
  let code = ''

  function transferStart (fileName = '/index.js') {
    file = fs.open(fileName, 'w')
    console.log('Transferring code...')
  }

  function transferChunk (codeChunk) {
    fs.write(file, new TextEncoder().encode(codeChunk))
  }

  function transferEnd () {
    fs.close(file)
    console.log('... transfer done.')
  }

  function load (fileName = '/index.js') {
    code = new TextDecoder().decode(fs.readFile(fileName))

    try {
      // let code = usercode.load('./index.js')
      ;(1, eval)(code) // need it like this so eval'd code is in global scope
    } catch (e) {
      console.log('Error with user code')
      console.log(e)
    }
    return code
  }

  function show () {
    console.log('\n----------- BEGIN CODE -----------\n')
    console.log(code)
    console.log('------------ END CODE ------------\n')
  }

  return { transferStart, transferChunk, transferEnd, load, show }

})();

usercode.load()

One disadvantage is that if you use the let keyword eg let led = 25 then that variable isn't available in the REPL. But if you use var instead, or create functions normally etc they are accessible in the global namespace.

[jakeg]

I've implemented this in full here: https://www.mathsuniverse.com/pico

Once connected to the Pico, the flash.js file above needs to be installed by pressing the green 'Flash bootloader'. This sends it via YMODEM.

With that bootloader installed, the 'Run on Pico' button is ridiculously fast!

Better would be if this kind of bootloader/flash.js is installed by default as part of the Kaluma .uf2, saving the 'install bootloader' step.

Feel free to 'view source' on the link above - it's just a single ~650 line file with all JS inlined and no external resources.

[jakeg]

I just made a new video showing it in action. https://youtu.be/J1tOiVSxBqY?si=yCRjoMEp8UXaIZVN&t=69 (jumps to 1min9s to see how fast it is).

Using ymodem: ~4s
Using the bootloader: ~50ms