Processing audio over I2S choosing the right MCU

[saraverbeecke]

Hi,

Should I go for a mimxrt/Teensy or an ESP32-S3 for audio processing using Micropython? The idea is to create a FX module taking audio in using an external ADC (like 24bit, 44.1 kHz), process it and send it back out using an external DAC (same quality specs) as fast as possible.
Anyone suggestions or experience with both MCU's in audio?
The work of @miketeachman for I2s processing for Micropython is fantastic, now I need to choose the right work horse.
Cheers,
Sara

[2dof]

It is nice article about using esp32 in audio ( using I2S ) ESP32 Audio Input - MAX4466, MAX9814, SPH0645LM4H, INMP441 with [youTube video demonstration] (https://www.youtube.com/watch?v=pPh3_ciEmzs) , so at least You can see how it is work. but its all coded in C/C++ but cover typical hardware problems (noise source, etc)

[saraverbeecke]

Thanks!

[miketeachman]

Assuming you don't need wifi, a Teensy 4.1 would be my choice. The Teensy 4.1 will have at least twice the raw processing power compared to an ESP32-S3. I would select a Teensy 4.1 over a Teensy 4.0 because the Teensy 4.1 includes a built-in SD card slot can use the very fast machine.SDCARD interface.

[saraverbeecke]

Oh ok @miketeachman, thanks for the answer. Do you think the 16 MB RAM on the Teensy 4.1 (when putting the extra 2 RAM chips on the pads on the back) is all accessible in Micropython? As I understand from your examples the DMA buffers are filled and emptied "in the background" so I can still do some asyncio stuff in the meanwhile with the audio data. Is the DMA buffer size linked to the available RAM?

[miketeachman]

Do you think the 16 MB RAM on the Teensy 4.1 (when putting the extra 2 RAM chips on the pads on the back) is all accessible in Micropython?

@robert-hh Do you know the answer?

so I can still do some asyncio stuff in the meanwhile with the audio data

Yes. The audio samples that have been read into the bytearray buffer using await sreader.readinto(buf) can be processed while DMA is filling the internal ibuf.

Is the DMA buffer size linked to the available RAM?

Yes. The internal ibuf is allocated from the MicroPython heap. If you do a gc.mem_free() call before and after the I2S() constructor you will see the heap decrease by the size of the ibuf parameter.

[robert-hh]

Do you think the 16 MB RAM on the Teensy 4.1 (when putting the extra 2 RAM chips on the pads on the back) is all accessible in Micropython?

The external memory slots are not supported by MicroPython. It's on my list, but not done, and I'm not sure whether it will. The external RAM is much slower than the internal RAM.

[saraverbeecke]

@robert-hh I'm thinking of building a custom board using a STM32H7XX... Any ideas to add external RAM to this one that is accessible on Micropython without tremendous effort? If you hookup a RAM package like the APS6404, is it automagically usable in Micropython (I mean for the I2S application will it work as additional DMA buffering?)

[peterhinch]

@miketeachman @robert-hh I would question whether MP was capable of the throughput required for implementing FX in real time. I wrote an fir filtering module which did realtime audio frequency filtering on a Pyboard. However the audio sampling was on a single ADC channel, and the code is in assembler. See performance: a 44.1KHz sample rate would be borderline.

Implementing an FIR filter is extremely simple compared to the DSP required for some audio FX. Admittedly modern platforms are faster than a Pyboard 1.1, but I do have questions over whether this can be done without extensive use of C/Asm.

[saraverbeecke]

Wow great info @peterhinch I will definitely look into that! Did you use the onboard ADC or an external one?

[mattytrentini]

Hey Peter, have you had a chance to investigate ARM CMSIS-DSP? Curious how the performance compares for filtering and FFT...

[peterhinch]

@mattytrentini I haven't looked at DSP for a couple of years. When ulab first came out we compared their DFT with mine and theirs came out on top, despite mine being written in inline asm. While I don't claim mine to be optimised, it does show that well-written C can be very fast. The library you cite probably uses the specialist ARM DSP instructions, so I would expect it to be especially fast.

I restrict myself to publishing code which doesn't require users to be able to build MP, so I doubt I'll be using the CMSIS. However for the needs of @saraverbeecke it looks ideal.

@saraverbeecke I used the onboard ADC and DAC.

[miketeachman]

I would question whether MP was capable of the throughput required for implementing FX in real time

I agree. I expect most real-time audio FXs will require a C implementation. The Teensy Audio Library provides examples on effects that are possible using the Teensy development boards.