Accelerometer Based Gesture Detection for Puck.js v2?

[espruino-discuss]

Posted at 2020-07-29 by user113948

Has anyone been successful in detecting specific gestures using the accelerometer on the Puck.js v2?

I know the Bangle.js seems to use Tensorflow Lite to recognize certain gestures, if you create a model to train them. Does the Puck.js v2 have the ability to load and recognize the same gesture model?

I'm looking for something where I can train a model to recognize simple gestures (forward, left, right, up, down, draw a Z, draw a circle) and then load it onto the Puck.

It looks like the Puck.js v2 has similar specs to the Bangle.js

Bangle.js
64MHz nRF52832 ARM Cortex-M4
64kB RAM 512kB on-chip flash, 4MB external flash

Puck.js v2
64MHz nRF52832 ARM Cortex-M4
64kB RAM, 512kB Flash

If that isn't possible, what's the best way to go about this just using the built in accelerometer? (I've been playing with require("puckjsv2-accel-movement").on(); )

[espruino-discuss]

Posted at 2020-07-30 by @gfwilliams

Yes, it is possible to run Tensorflow on the Puck, but you'll have to build your own firmware.

By default the Puck contains a networking stack so you can connect it to the internet with WiFi/GSM/etc and that uses up enough space that you can't have that and Tensorflow installed.

But compiling your own firmware is pretty straightforward - use the instructions on https://github.com/espruino/Espruino then edit https://github.com/espruino/Espruino/blob/master/boards/PUCKJS.py and replace:

     'NET',
     'GRAPHICS',
     'CRYPTO','SHA256','SHA512',
     'AES',

with

'TENSORFLOW`'

and you're good to go.

Otherwise you could hard-code some gesture recognition by looking at the values in the accel event. Detecting big changes in acceleration in each axis (eg big +X to big -X) would be a good start I reckon.

There's also the gyro, which could be good for detecting twistsing motion

---

Posted at 2021-12-28 by ThomasVikström

So, I have 3 Puck.js (through Kickstarter, so I guess V2) lying around unused. Does anyone have the above mentioned firmware built? I know it's possible to build it myself, but using a Win10 computer it seems it isn't straightforward, and it also seems time consuming, so if anyone one has a ready made firmware I'd be happy to receive it : )