Feasibility ESP32 with sound.

[PatLub85]

Hi,
I have a machine that rings when it has finished its work, but I can't hear it because it is at the end of my house. I have some ESP32 in my drawer, and I wondered if it would not be possible:
-1: record the ringtone (with INMP441?) and store this file (wav?)
-2: listen to the ambient noises, and compare with this file (how?).
-3: if the comparison is established, send a notification
Do you think this is possible and if so, with what hardware
Thanks in advance

[peterhinch]

Recognising a sound is trickier than you might think. Are there any other indications that it has finished work? An LED going off or some part physically moving? These would be easier to detect.

[PatLub85]

Hi Peters,
no I don't have these indicators.
I thought I could make a signature of the ringtone that would be easier to compare.
However is it possible to do it with so little memory of an ESP32?

[peterhinch]

Comparing sounds is computationally difficult. There can be differences caused by room acoustics and background noise, phase and frequency differences and suchlike. One option might be to compare in the frequency domain using DFT, using correlation to compare the spectra. If you can change the ringtone there may be scope for choosing a tone with an easily recognised spectrum.

It's an interesting topic for experimentation but as far as I know there is no ready-made solution.

[PatLub85]

Maybe if Mike Teachman reads these lines, he might be able to give me a solution.
https://github.com/miketeachman/micropython-i2s-examples

[sophiedegran]

Yes as Peter answer you, it's not very simple with sound..
Anyway, best thing is to see with a db meter difference in your room with ambiance 'noise' and when your machine is beeping.. Remenber that I2S microphone had mosly very bad sensivity, so you have to be very near the machine and the noise of the machine can be interpret as the 'beep' ...
For me, if you have a +10db when the machine beep it can be possible ;)
Waiting for your investigation
ps: working on miketeachman examples and it work well but disapointed by microphones.. can't find if it was due to micropython I2s implementation

[PatLub85]

Thanks for the advice about microphones.
However I am stuck with the algorithm.
How to check if two sound files are identical.
I don't think we do a bit-by-bit verification, but rather by signature, which I don't know how to do.
For the moment my project is on standby. I saw that in Python there is ShazamApi that could meet the need, but I found nothing in Micropython

[scruss]

I have a machine that rings when it has finished its work …

What kind of machine? Does it do anything else but make the noise when it's finished: an LED that comes on, or goes off, or …?

[Josverl]

• detecting the change/drop in current would be my first thought
  but perhaps training a simple model to recognize (the end of) a spin-cycle would be possible.

@jonnor might have an opinion on that ( https://github.com/jonnor/machinehearing/ )

[jonnor]

For a washing machine, I would recommend trying out using vibration monitoring, using an accelerometer. Because the machine stopping to vibrate is a very clear, persistent indicator that it is done. And, unlike sound, not so influenced by the surrounding environment. It can also be done using periodic sampling (say 1 second per 1 minute), so that one can have very long battery life.
I would recommend computing the RMS values of the accelerometer data after subtracting the gravity vector. Probably you can just do a simple threshold on that to detect off vs on.
There are some optimized RMS implementations at https://github.com/emlearn/emlearn-micropython/blob/master/examples/soundlevel_iir/soundlevel.py#L43
But for accelerometer data the standard formulation will also do just great.

[Josverl]

yes, fully agree - I was remembering later that , some time ago , after learning about MicroPython, a few engineers I worked with at a steel mill were able to build their own sensor to detect that the steel spooler was running out of alignment based on a motion sensor. They managed put this together with a M5 ESP32 device in under a week , while they had been looking to solve this detection problem with a PLC for a few years (based on the screeching sound it made just before it tore itself to bits).
Making the M5 survive in that harsh environment took some more time...

[PatLub85]

My machine has a final drying cycle and the pressure sensor will, I think, be ineffective.
On the other hand, the link https://github.com/jonnor/machinehearing/ is extremely documented and I have weeks to try to understand everything.
I need a library that records, then makes a signature and compares this signature with another.
Thank you all for the different solutions proposed.
I'm not sure I'll succeed, but I'll try

[Josverl]

in my book To try is to succeed

[jonnor]

As mentioned in another comment, I believe that accelerometer vibration monitoring is the best approach for this. But here is some information on how to do it with sound :)

Detecting an indicator sound (triggered at end of cycle) means being able to detect a short sound that is repeated only a few times. This is what is known as a sound event, and the task of is known as Sound Event Detection. It is often a bit trickier than general audio classification, because one cannot aggregate many detections in order to keep false detections down. An indicator sound is designed to be clearly audible (by humans), which hopefully makes this task on the simpler side of these kinds of problems. But depending on the other sounds in the environment, it could get tricky still.

The simplest approach would be template-based, to compare the signature of the sound-of-interest to a set of known templates (known examples). But this needs to be done in a reasonable feature space. The "beeping" is a tonal sound, so I would compute the FFT to get the frequency spectrum. The FFT length will need to be set appropriately, around the length of each beep. Hopefully it sticks out very clearly there as one or a few peaks. If so, then Euclidean distance to a set of such templates might work. In emlearn-micropython there is a K Nearest Neighbors classifier that can be used for this. As well as an efficient FFT implementation. https://github.com/emlearn/emlearn-micropython

Capturing some sound of the beeps of interest, and the ambient sound around it would be the first step. This can be done using the I2S examples by Mike.

[peterhinch]

I agree with the idea of vibration monitoring. I would expect the maximum vibration to be in the horizontal axis perpendicular to the axis of rotation (side to side motion caused by load imbalance). It may be that measuring that will be sufficient.