Processing incoming serial data - what's the smart way to do this in js?

[espruino-discuss]

Posted at 2019-02-26 by DrAzzy

I know I used to know how to do this, but I've been in Arduino land a lot lately, and now this is puzzling me... I am sure there's a right way.

I have serial data coming in. Each message will consist of a number of characters terminated with a newline.
What I do on Arduino is load the characters one at a time into the incoming message buffer, and if the character is a newline, instead of doing that, I send the buffer off for processing.

But on Espruino, I might get multiple characters at once - which means I could get the end of one message and the start of the next. What's the smart way to handle this? I'm sure it's a common thing that people have done, and settled on the obvious solution.

Is there really no better way than using one of the substring functions to cut one character at a time off of the incoming data, and then apply the arduino-style logic? This seems uglier than I'd expect for Espruino.

My initial reaction is that handling serial data on Espruino is much harder than in Arduino land, where you can just pick off a character at a time and easily process it - but I suspect I'm thinking about it wrong.

Thanks

[espruino-discuss]

Posted at 2019-02-26 by @allObjects

@drazzy, take a look at the GPS module... it does that the Espruino way: constantly receiving and adding to the buffer, and as long as a line is found it removes that line from the buffer and processes it. Latter could also be made asynchronous, so if there is something coming in while one line is processed, that this incoming (urgent) thing is handled first, such as adding new chars to the buffer that is then checked against line ends. The (async) processing loop ends when no complete line is found anymore. A watchdog could be implemented that if when no line end comes in within a certain time, flushing is triggered, because most likely the (eventually) remaining data in the buffer is garbage...

I vaguely recall that we did some of this 'over buffer looping'.... nowI recall: it was sending out stuff without using the drain w/ callback option... conceptually, it is very similar.

[gfwilliams]

Posted at 2019-02-27 by @gfwilliams

As @allObjects suggests, the GPS module is a good example - just use indexOf and substr to search for newlines and split the string. (or you could use str.split("\n") and deal with the array that comes out the end of it).

If you were dealing with particularly long lines you could use indexOf on the data coming in rather than the whole line, but for <50 byte packets the way that's used there is going to work great and is reasonably simple.