ESP32 S3 AI Acceleration Support

[Rumidom]

Acording to espressif ESP32-S3 has AI Acceleration Support

AI Acceleration Support
ESP32-S3 has additional support for vector instructions in the MCU, which provides acceleration for neural network computing and signal processing workloads. Developers can take advantage of these vector instructions through ESP-DSP and ESP-NN libraries to optimize their applications. ESP-WHO and ESP-Skainet SDKs will also support this acceleration.

is there any way of accessing those instructions via micropython? so far I only found resources on how to access it using their oficial library would be really cool to be able to inference models on it especially using python

[Rumidom]

it seems like the esp-dsp library implements vector operations in c not sure how hard it would be to port those over to micropython, I think I'll try doing that and maybe create a simple micrograd/tinygrad library on top of it. But just the inference part, it would be awesome to be able to run tinygrad models on a esp32

[jonnor]

The easiest would be to create some MicroPython C modules that exposes APIs for the relevant functionality. For inspiration, see how emlearn-micropython works - it implements classical ML models in C and MicroPython APIs.

[Rumidom]

according to this 10297 you cant access esp-idf in native code modules, so I guess the esp-dsp library wouldn't work ? is there a way to write esp32 assembly code as a .mpy module for micropython?

[jonnor]

That limitation is only for dynamically loaded C modules (.mpy files). A user module which is built together with the firmware does not have that limitation.

[Rumidom]

I found this example on how to use the ESP32 S3 SIMD instructions on arduino

[NowLoadY]

Is AI inference possible on the ESP32-S3 using MicroPython? Yes, that's what I'm expecting.😀

[NowLoadY]

with acceleration) haha

[jonnor]

If anyone would add ESP S3 acceleration to the TinyMaix C library, I would be happy to include that as a build configuration in emlearn_cnn (part of emlearn-micropython). It already runs small CNNs quite OK on an ESP32 S3 though, even without additional acceleration. Can do 20 fps of small models, or medium-sized models at more like 1 FPS.

[cnadler86]

I have been working on a Micropython binding for the esp32 DL models available at the moment. They are performing ok if you consider the price for the HW (the face detection at around 10FPS without a good multicore balance at a low image resolution).
Implementing a full binding to the underlying library will requiere much more work, but at least someone can experiment and have fun with them.

[cnadler86]

Here the full benchmark (FPS) for the face detector with the esp32 camera (ov2640) on the S3 with the new jpeg decoder:

FrameSize	fb_count=1	fb_count=2
QQVGA	12.4	18.2
R128x128	12.4	17.6
QCIF	12.4	16.7
HQVGA	10	15.4
R240X240	10	14.3
QVGA	12.4	18.2
CIF	8.3	11.4
HVGA	6.2	10.2
VGA	5	6.9
SVGA	4.1	5.1
XGA	2.5	3.5
HD	2.4	3.1
SXGA	1.8	2.3
UXGA	1.4	1.6

I think the new esp models have such acceleration. I would say the performance is not bad.