Espruino command-line tool

[gfwilliams]

Posted at 2015-11-18 by @gfwilliams

I'm thinking I'll finally submit the command-line tool for Espruino to NPM, as espruino has become clear now.

... but any thoughts as to what would be a good name for the command? Currently it's espruinotool, but that's pretty longwinded.

What about:

• esp - too short - might get confused with something else?
• esptool - probably not given the ESP8266 programmer is already called that :)
• espr?
• espruino - might get confused with the actual Espruino binary?

[espruino-discuss3]

Posted at 2015-11-18 by Ollie

Last one, 'espruino', for me. Longest of them but semantic and your namespace so won't conflict.

---

Posted at 2015-11-18 by UliMerkel

espruino_cmd, to avoid the conflict with the binary

---

Posted at 2015-11-18 by @gfwilliams

Just uploaded it here - as espruino for now as I didn't see UliMerkel's post when I did it.

... it's early days, so I can always change it later.

---

Posted at 2015-12-25 by tve

It looks like it doesn't support communication over sockets?
How does one run this after checking out the code from github? I'm not familiar with node. Looks like there are a lot of dependencies that need to be installed?

---

Posted at 2015-12-25 by tve

FYI, I added support for node sockets and for a --terminal option that pops into the terminal after uploading a file. The socket support doesn't work for multiple sockets and the terminal support is a gross copy&paste of the existing terminal code, so ti should be included more nicely.
To see the changes use: https://github.com/espruino/EspruinoTools/compare/gh-pages...tve:gh-pages?w=1 to avoid seeing all the whitespace changes (my editor removes trailing spaces automatically).

---

Posted at 2016-01-04 by @gfwilliams

Thanks! The socket thing looks great - and having the ability to force the terminal looks like a good idea too.

I know my code is a bit of a hack, but there's got to be a better way of getting the terminal without that duplication :)

---

Posted at 2016-01-06 by Snerkle

The expr function isn't behaving as I'd expected:

• when I require('espruino') in node and use expr then I get a lot of info on the console and there is no option to suppress this like there is with the cli -quiet option.
• when I use the cli -e option then I never see the output that is returned from the esprino as there is no way to register a callback like there is with require('espruino').expr()

I have a work-around for the first issue, see attached

require('esp-console');
esp = require('espruino');
console.mute();
esp.expr(port, 'getTime()', function(m) { console.info(m); });
console.unmute();

Note the use of console.info in the callback, as console.log is currently mute (by default).
You can mute any version of console using for example console.mute('error');

For the second issue, could you maybe set up a callback of function(m) { console.log(m) }; when espruino -e 'code...' is used, so the output can be seen?

---

Attachments:

• esp-console.js

---

Posted at 2016-01-06 by @gfwilliams

when I require('espruino') in node and use expr then I get a lot of info on the console

Yes, it's a pain - It's under the 'To Do' list on https://github.com/espruino/EspruinoTools - sadly I have my work cut out doing everything else though!

cli -e

At the moment the CLI just sends that command straight to Espruino, so it doesn't really have a way of parsing the result out.

I guess it could change to using .expr() instead (which actually sends 'console.log("<<"+"<<<"+JSON.stringify('+expressionToExecute+')+">>>"+">>")'), but I wonder if that might cause some problems with code that worked previously.

---

Posted at 2016-01-06 by Snerkle

OK. I can do what I need using the module form with my console.mute() work-around.
I plan on moving my build process to WebStorm and use the integrated gulp tools to upload code and run tasks on the espruino.
I really want the git/gitHub integration that WebStorm gives me as I live in fear I'll lose changes/history using the WebIDE. I've also found it doesn't save/cache the code in the code window once it exceeds a certain size (sorry but I didn't investigate too much), and I get inconsistent errors about not being able to find modules when modules require() other modules.

To complete the integration it would be nice to figure out how to give WebStorm what it needs to be aware of the built-in espruino functions/libraries, especially where they differ from standard javaScript/node, e.g. the E. object and all the global h/w functions like digitalWrite(). I look into it and let you know if I find a solution.

---

Posted at 2016-01-06 by @gfwilliams

If WebStorm uses Tern, you might be able to somehow use this tern info file.

... But otherwise you could probably hack something up so that you could generate a huge JS 'stub' file with all the functions and comments in?

---

Posted at 2016-01-06 by Snerkle

Had a look and it appears WebStorm doesn't read tern files. It likes js files that include JSDocs format comments.

Before I launch into parsing the tern file into something else: is tern your source format you maintain my hand, or is there something behind that like a structured database I might find easier to work with?

---

Posted at 2016-01-06 by @gfwilliams

The tern file is generated using https://github.com/espruino/Espruino/blob/master/scripts/build_tern_json.js

However those scripts may (will) change in the future, but the format of the Tern file will stay the same :)

---

Posted at 2016-01-06 by Snerkle

Referring back to post #9 above:

How about a new command line option, say -x that uses .expr() and doesn't impact existing users of -e ?

---

Posted at 2016-01-07 by @gfwilliams

Yes, I think that could work well.

I guess ideally there'd be 'statements' - what happens currently, and expressions which use expr

---

Posted at 2016-01-08 by Snerkle

I've been working away on what I mentioned in post #10 above, and have the two attached files to offer as a result.

Its not the tern based stuff but some other additions to the cli that I thought would be helpful.

These are updates for both:

• https://github.com/espruino/EspruinoTools/blob/gh-pages/plugins/localModules.js
• https://github.com/espruino/EspruinoTools/blob/gh-pages/core/serial_nodeserial.js

I apologise in advance that I haven't got around to doing these a pull-requests on gitHub. I'll get over my laziness soon and start proposing updates that way in future. For now I hope you'll do a diff and choose to integrate the changes if you like them.

These amount to:

• serial_nodeserial.js : applies a filter the to available ports and lists only ports that (hopefully) belong to an espruino (based on USB vendorId and productId)
• localModules.js : implemented the relevant rules for require.resolve() from node so enable a more complete 'require()' experience with with cli.

If I say so myself, I'm living and loving them both.

---

Attachments:

• serial_nodeserial.js
• localModules.js

---

Posted at 2016-01-08 by @gfwilliams

Thanks - I've just pulled in the localModules stuff (it's modules because that's what the Web IDE uses for local projects, so it's a bit more compatible).

For the serial stuff, it will work on all Espruino boards via USB - that kind of detection is really handy, but in reality it stops you using Espruino via a USB-TTL converter (eg for ESP8266, or even normal Espruino boards where you're not using USB), or even via bluetooth.

What'd be better is if all the serial modules could report back the path, but also a boolean saying whether they're sure that it's an Espruino.

Then, if you didn't specify a port it could scan all ports, and if it was sure that one of those was Espruino, it'd just choose it.

---

Posted at 2016-01-08 by Snerkle

OK.
But 2 things:

1. I wrote the localModules code for node so I haven't tested if it is browser compatible
2. The serial stuff doesn't stop you manually providing a port using -p -port it only affects the auto-selection of the 'first' port and limits it to the 'first espruino like port (STM32Fxxx)'

---

Posted at 2016-01-08 by Snerkle

Hmmm, 2 steps forward, 1 step back.
Happy with my mods, I've tried a build on a 'large' project and I'm having serial comms. problems with the espruino-cli dropping characters in both directions.
When I say large, I mean a combined 26Kb of source.

• I have checked it's not the code I am loading: I can capture it with -o and then load this file via the WebIDE successfully
• I have forced Espruino.Config.SERIAL_THROTTLE_SEND = true; and verified this is in effect, and it doesn't solve the problem.

When espruino -v source.js sends the code to the Pico there are dropped characters, even if I leave out the -v the result is the same. If I use espruino in terminal mode and do a dump(); on the full set of code (loaded with the WebIDE) then there are dropped characters in what I see. Again, I don't have these problems with the WebIDE.

Any ideas about what to try next? I currently think its somewhere in the 2 different code paths for serial comms. when you compare the WebIDE with espruino-cli.

---

Posted at 2016-01-08 by @gfwilliams

Well, WebIDE and espruino-cli use basically the same code apart from serial_*. I think someone had problems with this and node-serialport, and then I was pretty sure it went away.

Just checking - do you have an up to date Pico firmware?

---

Posted at 2016-01-08 by Snerkle

I'm using

>process.env
={
  "VERSION": "1v84.145",
  "BUILD_DATE": "Jan  5 2016",
  "BUILD_TIME": "10:13:59",
  "GIT_COMMIT": "2f9e0124fad90ebcf32746a02ff3fd56e24a35f3",
  "BOARD": "PICO_R1_3",
  ...

A build you pointed me at to improve the RTC accuracy without a crystal (under my gitHub personna: dmcnaugh) - as expected its better but not fantastic - averaging +5 seconds every 10 minutes.

From what I can tell I'm not losing characters transmitting to the Pico but who knows. The same file will load then not load, then load. It will almost always load after a power cycle of the Pico, but then again I can make files that won't event do that. If I change the echo(0); to echo(1); in codeWriter.js -> ````writeToEspurino``` then it looks like its the echoed characters that are sometimes being dropped, but I'm not getting syntax errors so I don't think its the transmitted characters. At some point after about 16Kb (changes) of code has been sent then the Pico just freezes and wont talk anymore until it has been power cycled.

---

Posted at 2016-01-08 by @gfwilliams

Hmm, that should be fine then. When the Pico freezes it's probably stuck in the middle of entering a function, with echo off.

Try [Ctrl-C] echo(1) [enter] and that may fix it - you might also be able to see what it was entering.

As you've got the EspruinoTools source handy, you could try this: espruino/espruino-tools#6 (comment)

It's a bit of a hack, but it seemed to work in that case.

---

Posted at 2016-01-09 by Snerkle

Thanks, the code suggested at:

As you've got the EspruinoTools source handy, you could try this: github.com/espruino/espruino-too­ls/issues/6#issuecomment-101716152

solved the problem of sending code to the espruino. Is there a permanent fix in the issues queue? Or is this seen as a problem with node-serialport ?

I tracked down the 'dropped characters' problem when the espruino is sending to the terminal a little further and it looks like nothing is dropped in the transmission but different terminals produce different results. The issue occurs only with very long lines, typical when you dump(); and it is sending the content of a large, multi-line function as a single line of text:

• iTerm (OSX) shows only as many characters as it can to fill one line and then ignores the remainder of the line, and doesn't wrap
• the WebIDE wraps at the righthand end of the terminal and keeps showing everything, over multiple lines (best solution in my opinion)
• the terminal in WebStorm does - I don't know? - but you don't see the start of the line; the end of the line; consecutive characters from somewhere in the middle of the line; and sometimes you don't see anything at all for a long line. This is why I thought characters were being dropped.

So I'll try filing a bug with WebStorm but meanwhile I can't find a simple solution to make terminals wrap at the width of the terminal window.

---

Posted at 2016-01-10 by UliMerkel

An Option to SetTime ?
Wouldn't it be good to have an option to set espruinos clock to the time of the server.
This should be executed before all the other processing is done so we can have the "right" time.

-s, --settime : Similar to the Web IDE Settings -> Communications -> Set Current Time

---

Posted at 2016-01-11 by @gfwilliams

Glad that fixed it. I guess it might be worth changing the serial-node.js file to include that by default if it's still an issue?

Are you running under MacOS then? I guess it could be a node-serialport issue for OSX and I don't think I see it under Linux.

An Option to SetTime ?

Actually this exists already, it just needs turning on by setting the correct config options...

---

Posted at 2016-01-11 by Snerkle

Yes I'm on OSX.
The issue I raised on gitHub is also a solution for the this problem.
I got around to forking espruino/Espruino (as dmcnaugh) and have successfully built an image that implements the escape character fix, and I'm not having this issue now. The root cause appears to be that different terminals handle non-printable/non-ascii characters differently, so if you don't send them non-printable/non-ascii characters, then you don't get the problem.

---

Posted at 2016-01-11 by UliMerkel

Actually this exists already, it just needs turning on by setting the correct config options...

Thanks Gordon, seems I have not found it in the documentation I scanned so far.
Can you give me a hint where I can find info about this config options for the command-line tool?

---

Posted at 2016-01-13 by @gfwilliams

@Snerkle - yes, I'll get that fix in. What was Espruino printing that was causing characters to get lost?

Can you give me a hint where I can find info about this config options for the command-line tool?

The tool would need changing. For example a config option is set up here for Baud rate.

All you need it Espruino.Config.SET_TIME_ON_WRITE = true

---

Posted at 2016-01-13 by Snerkle

What was Espruino printing that was causing characters to get lost?

Actually it was the string for the espruino favicon you provided in the http server example titled Favourite Icon on WebServer

If you load that code onto the espruino and then do a dump(); then the long string of the favicon variable is not displayed as it is entered (where all non-ascii values are escaped), instead non-ascii values are displayed as 8-bit (unicode UTF-8 I assume) characters.

---

Posted at 2018-06-20 by user73560

Hi @gfwilliams,

With 1v96 version of the PUCKJS firmware I was able to do following:

1. Produce hex file from the java script
2. Merge the firmware hex with the hex file from above

Once the hex file from 2) was uploaded I had a firmware with uploaded code from 1)

This is not possible with PUCKJS firmware 1v99.50 version, is there any reason for that ?

---

Posted at 2018-06-21 by @gfwilliams

Yes - around version 1v99 Espruino changed the way code is stored so it uses a mini filesystem, allowing you access to store your own data as well using require('Storage').

However if you update the command-line tools to the latest version then they'll produce hex files in the new format.

---

Posted at 2018-06-21 by user73560

Hi @gfwilliams,

Where can we find 'Storage' module ? Any examples using it ?

---

Posted at 2018-06-21 by @gfwilliams

http://www.espruino.com/Reference#Storage - or use the Espruino site's search box :)