build system for espruino

[espruino-discuss]

Posted at 2017-01-08 by CriscoCrusader

For the project I'm doing, I'm writing each layer as reusable modules. The layers are:

1. User interface (general purpose library for connecting buttons and such)
2. Timer and event emitter
3. State machine
4. Curve generation and output processing
5. Output transmission (I2C)

I need to know how these should best be assembled. I've read the modules page but I still have questions.

Does the web IDE perform all the minification? Do I stand to gain any improvements/optimizations by using uglify-js2?

Is CommonJS the most efficient approach for Espruino builds using multiple custom modules? If you look at the output generated by a CommonJS system like Browserify, you'll see that a lot of junk has to be wrapped around each module to recreate the CommonJS system within a concatenated file.

When it comes time to sell Espruino Picos flashed with my code, how do I setup the production process?

[espruino-discuss]

Posted at 2017-01-08 by CriscoCrusader

Looks like webpack works really well when using the optimization flag: --optimize-minimize. This allows me to use npm link to bring in each of the modules I'm writing, then build, then flash to the board.

[espruino-discuss]

Posted at 2017-01-08 by @joakim

Have you had a look at Rollup? It claims to produce smaller bundles than Webpack and Browserify, and it has an uglify plugin.

http://rollupjs.org/

[gfwilliams]

Posted at 2017-01-09 by @gfwilliams

If you're trying to get the minimum possible size I'd look at some process that makes one single JS file from your many files - something that has tree-shaking so can remove unused code.

The module implementation inside Espruino is quite efficient, assuming you just use it with exports and without the general-purpose function(){} wrappers that get added for some libraries.

However at the end of the day, all the names of exported functions have to stay the same - so some tool that knows enough about the whole project to rename all functions will be better. It just depends if you care about debugging or not.

For production, it depends on the parts you're using and whether you expect their firmware to be up to date. For STM32-based Espruino boards I'd suggest getting a board as you want it, then using stm32loader.py to load a firmware image from the board, and to then write it to all your production boards.

That way you update the Espruino firmware and your software all at once.

[espruino-discuss]

Posted at 2017-01-09 by CriscoCrusader

Seems the only way to run my code is minified, because there's no line breaks. The IDE runs each line separately, which breaks the script.

[gfwilliams]

Posted at 2017-01-10 by @gfwilliams

How are you uploading? the command-line IDE should tweak any code formatting to make it work.

Also if you use K&R-style formatting (with the opening bracket on the same line) everything will just work even if you upload the raw characters.

edit: if you're using a command-line app, you need to use the proper espruino one - several others have been made, but generally they don't check over the code and reformat the lines as described above (they also tend to miss out quite a lot of other steps too)

[espruino-discuss]

Posted at 2017-01-10 by CriscoCrusader

Tonight I got as far as loading the build result on the board using the espruino npm module. I spent a few hours trying to get it to drop me into the espruino REPL right after the code is loaded, like what happens when I connect from the command line, espruino -p /dev/cu.usbmodem1421, but I haven't been able to get it working.

I copied the terminal() function from espruino-cli.js, but when Espruino.Core.Serial.open() is called, this error is thrown:

COM:  /dev/cu.usbmodem1421
Connected Ok
undefined:92
    connection.write(a, callback);
              ^

TypeError: Cannot read property 'write' of undefined
    at Object.writeSerial [as write] (eval at loadJS (/Users/rob/projects/espruino2/node_modules/espruino/index.js:10:15), <anonymous>:92:15)
    at writeSerialWorker (eval at loadJS (/Users/rob/projects/espruino2/node_modules/espruino/index.js:10:15), <anonymous>:228:19)
    at Object.writeSerial [as write] (eval at loadJS (/Users/rob/projects/espruino2/node_modules/espruino/index.js:10:15), <anonymous>:245:5)
    at Object.getEspruinoPrompt (eval at loadJS (/Users/rob/projects/espruino2/node_modules/espruino/index.js:10:15), <anonymous>:206:26)
    at Object.executeExpression (eval at loadJS (/Users/rob/projects/espruino2/node_modules/espruino/index.js:10:15), <anonymous>:270:27)
    at Timeout.queryBoardProcess [as _onTimeout] (eval at loadJS (/Users/rob/projects/espruino2/node_modules/espruino/index.js:10:15), <anonymous>:39:25)
    at ontimeout (timers.js:372:18)
    at tryOnTimeout (timers.js:237:5)
    at Timer.listOnTimeout (timers.js:207:5)

I can see that it's invoking open() on the device (openSerial, serial_nodeserial.js), but I haven't figured out what's happening there and why connection is undefined in my use case.

FWIW, calling Espruino.Core.Serial.isConnected() before Espruino.Core.Serial.open() returns false.

[gfwilliams]

Posted at 2017-01-10 by @gfwilliams

I spent a few hours trying to get it to drop me into the espruino REPL right after the code is loaded

Try espruino -p /dev/cu.usbmodem1421 -w yourfile?

[espruino-discuss]

Posted at 2017-01-10 by CriscoCrusader

Hi @gordon. Execution from the command-line works fine. I'm working with the espruino module as an import in my build script: https://github.com/stokebrain/morra-espruino1/blob/master/build.js

The trouble starts with Espruino.Core.Serial.open() in openConnection().

[gfwilliams]

Posted at 2017-01-10 by @gfwilliams

I don't have time to help you out with that at the moment I'm afraid.

I guess worst case you could just execute espruino -p /dev/cu.usbmodem1421 -w yourfile as a separate process

[espruino-discuss]

Posted at 2017-01-10 by @joakim

Why can't you use the espruino npm module directly to send your code? Or is there some reason you need to get into the REPL afterwards?

Anyway, I'm actually making a Rollup plugin for Espruino. I hadn't thought of using Rollup for bundling Espruino projects before, but it makes perfect sense. It prefers ES modules (import … from, export default, etc), supports treeshaking, has a nice selection of plugins and produces no overhead as long as format is es (the default). For simple Espruino projects without any external dependencies it results in very nice code (before uglification of course). The result is a flattening of all your modules (treeshaken) into one file (one scope) without any wrappers/loaders.

https://github.com/joakimstai/rollup-plugin-espruino

Note: I haven't actually tested it with an Espruino device yet, as I haven't got one on me :)

[espruino-discuss]

Posted at 2017-01-11 by CriscoCrusader

This is tough, and frustrating. I can see that connection is defined after calling Espruino.Core.Serial.open(), but it seems that eval() is running writeSerial() in a scope where connection is undefined. And this is being triggered by the queryBoardProcess which was added as a processor function for the 'connected' event.

Why does this project need to use eval?

@joakim, I am using the espruino npm module directly to send my code. That's what this does: https://github.com/stokebrain/morra-espruino1/blob/master/build.js

I want to enter the terminal after the code is loaded so I can receive the log output from the device as it's running.

[gfwilliams]

Posted at 2017-01-11 by @gfwilliams

Why does this project need to use eval?

It's because the Web IDE was originally made as a website where each file executed in the same scope, and adds itself to the global 'Espruino' variable - I can't just 'require()' everything as-is. Ideally I'd rewrite every file to handle both loading styles, but I don't have the time to do that while also checking it still works fine as a Chrome App, NWjs app, and on the Espruino website.

As-is it does make debugging extremely frustrating, since the filenames aren't correct :(

Looks to me like connection.write is in serial_nodeserial.js. connection shouldn't be global at all - it looks like you're either:

a) trying to write before you're actually connected
b) disconnecting before the code has actually been written
b) somehow managing to have got two separate instances of the EspruinoTools loaded - so one is connected while the other isn't.

There should really be a lot more being written to the console - it might be worth trying to figure out why that's not happening as the extra log messages would probably let you know what's going on.

I still don't understand why you can't just use the existing espruino-cli code though - or at the very least start from that code which does exactly what you want and then strip out the bits you don't want.

[espruino-discuss]

Posted at 2017-01-11 by CriscoCrusader

I still don't understand why you can't just use the existing espruino-cli code though - or at the very least start from that code which does exactly what you want and then strip out the bits you don't want.

That's exactly what I did.
https://github.com/stokebrain/morra-espruino1/blob/master/build.js#L76 and down

The problem seems to be that it doesn't work after having already run Espruino.init(), and Espruino.sendFile(), which is the first step in my script (after all the webpack build stuff). I added a lot of logging statements. I can see that connection is a SerialPort object and should work just fine, right until queryBoardProcess() runs, which is happening because it's listening for the 'connected' event. I don't understand yet what this code does. Will have to work on it tomorrow. Thanks for the explanations.

[espruino-discuss]

Posted at 2017-01-11 by @joakim

@CriscoCrusader Ah, sorry! I only looked for require('espruino') at the top of the code, and thought you used serialport directly. Never mind me then :)

[gfwilliams]

Posted at 2017-01-11 by @gfwilliams

Actually it looks like your problem might be that sendFile ends up closing the connection when it's done: https://github.com/espruino/EspruinoTools/blob/gh-pages/index.js#L134

You might have to copy the code out of it and use it directly.

[espruino-discuss]

Posted at 2017-01-12 by CriscoCrusader

That worked :). I was thinking I would just reopen the connection after it closed.

Thanks so much for your help :)

https://github.com/stokebrain/morra-build/blob/master/main.js