Low Latency Bluetooth Integration?

[baesek]

Hi! First thank you very much for sharing your work!
I am planning to make a Sound workshop for teenagers. Each of them gets a ESP32 Audio Kit (ES8388) and some triggers in their shoes to trigger sounds while walking. Sadly the headphone output is really weak and the speaker output with mobile speakers doesnt sound good. thats why I thought it might be possible to either 1. transmit the audio from the boards to a bluetooth receiver or 2.transmit a trigger signal to a mainboard which then plays back a sound.

1. I am using Marcel Licence esp32_midi_sampler as a starting point to play back sounds. Then I tried to use your library ESP32-A2DP and use it to send audio via bluetooth. I could somehow get it working using a fifo which is filled with all the audio data and when the callback get_data_channels() is called it reads the amount of data (defined by the channel_len parameter which gets passed to the callback) out of it. It works but, I get strange Crackling Sounds and noises every now and then. Do you see the mistake I make?
   I have some questions, sorry if I now just stack them here:
   Is this callback called when the Bluetooth receiving device sends a request, or is it called from your code ?
   Is the channel_len always the same or is it kind of changing to compensate communication lags?
   I plan to try to improve latency as soon as I get the basic SBC Bluetooth connection working.
   Is it easily possible to change the bluetooth codec to aptx low latency or even something with less latency? like aptx live?
   Is it possible to decrease the latency by just transmitting a mono signal?

This is the important code pieces with which i try to explain my scenario:

// Bluetooth output
#include <CircularBuffer.h>
#include "BluetoothA2DPSource.h"
BluetoothA2DPSource a2dp_source;
...
static CircularBuffer<int16_t,SAMPLE_BUFFER_SIZE * 32> fl_sample_bt; // SAMPLE_BUFFER_SIZE = 64
static CircularBuffer<int16_t,SAMPLE_BUFFER_SIZE * 21> fr_sample_bt;

#ifdef BLUETOOTH_ENABLED 
// callback used by A2DP to provide the sound data
int32_t get_data_channels(Frame *frame, int32_t channel_len) {
    //audio_task();
    // channel_len = fl_sample_bt.size();
    for (int sample = 0; sample < channel_len; ++sample) {
        // update the frame with converted buffer values to int16_t 
        //frame[sample].channel1 = int16_t(fl_sample[sample] * 16383.0f);
        //rame[sample].channel2 = int16_t(fr_sample[sample] * 16383.0f);
        frame[sample].channel1 = fl_sample_bt.pop();
        frame[sample].channel2 = fr_sample_bt.pop();
    }
    return channel_len;
}
#endif
...
void setup()
{
    // put your setup code here, to run once:
    delay(500);
#ifdef BLUETOOTH_ENABLED    
    a2dp_source.start("FeinTech-ABT00101", get_data_channels);  
    a2dp_source.set_volume(100);
#endif
...

void loop() 
{
audio_task();
...
}
...
inline void audio_task()
{
    ...
    /*
     * apply master output gain
     */
    for (int n = 0; n < SAMPLE_BUFFER_SIZE; n++)
    {
        /* apply master_output_gain */
        fl_sample[n] *= master_output_gain;
        fr_sample[n] *= master_output_gain;
#ifdef BLUETOOTH_ENABLED
        fl_sample_bt.unshift(int16_t(fl_sample[n] * 16383.0f));
        fr_sample_bt.unshift(int16_t(fr_sample[n] * 16383.0f));
#endif 
        /* output signal to vu meter */
        *vuOutL = max(*vuOutL, absf(fl_sample[n]));
        *vuOutR = max(*vuOutR, absf(fr_sample[n]));
    }
    ...
    Audio_Output(fl_sample, fr_sample);
    ...

It works somehow, but the sound is kind of distorted...

EDIT:
If I set the FIFO size to much bigger:

static CircularBuffer<int16_t,SAMPLE_BUFFER_SIZE * 32> fl_sample_bt;
static CircularBuffer<int16_t,SAMPLE_BUFFER_SIZE * 32> fr_sample_bt;

2. Now to my seconds question. I thought it might be also an idea to just transmit the triggers via bluetooth with much less latency and then there is one Audio Kit just receiving triggers and playing the corresponding sounds? Do you think that's easily possible with the ESP32-A2DP library? Can you help me?

Thank you for any help, i am somehow trying to make it a great fun for the young people in the workshop, but my knowledge is limited... : (

-paul

[pschatzmann]

You have a lot of questions and I am not sure if I can answer them all. However I would recommend that you have a look at my Arduiono-Audio-Tools library. It has a lot of examples for the AudioKit and you can also use most other examples just replacing the Audio Sink! The documentation can be found in the Wiki. I was quite happy with the quality and level of the audio that is coming from the speakers!

I am note sure what your issue is, but potentially it could be any of the following

• buffer underruns because you can't provide data fast enough or the buffer is too small
• synchronization issues between the 2 cores

The root cause of the problem is that your audio source does not support a pulling API: I think the AudioEffects from the Audio Tools should be a better fit!

Looking at your purpose, my recommendation would be not to use BluetoothA2DPSource and stick with something which has a low risk of failure. With it's high sample rate it puts the processor at the edge and there are plenty of stumbling blocks. The BluetoothA2DPSink is working very reliable however.

Is this callback called when the Bluetooth receiving device sends a request, or is it called from your code ?

It is the Espressif IDF framework which calls this

Is the channel_len always the same or is it kind of changing to compensate communication lags?

Looking at the API I would expect that there is some logic, but I am not sure. It is also not documented by Espressif! In my examples I tried to always provide the requested number of bytes and with this the requested size stayed the same.

I plan to try to improve latency as soon as I get the basic SBC Bluetooth connection working.
Is it easily possible to change the bluetooth codec to aptx low latency or even something with less latency? like aptx live?

I have to rely on what Arduino and finally the included Espressif IDF is providing and they currently do not support aptx. I think there is some open source implementation (of a cloned IDF version which support APTX), but you would need to integrate this into Arduino and build your own Arduino Implementation. That's definitely not something I am interested in.

Is it possible to decrease the latency by just transmitting a mono signal?

No - A2DP is stereo!

transmit the triggers

You can basically use any of the following protocols

• Bluetooth Serial
• IP
• UDP
• ESP-Now

In my audio-tools I started to add some examples - mainly with the purpose to transmit audio. But in the end it does not matter what the payload is. Apart from this you will find plenty of good tutorials on the Internet.

After some a second thought however I am not sure if this is a good idea in a school setting where potentially many users might send a request at the same time to the same resource!

[baesek]

Thank you very much for your answer, it is getting a little bit more clear to me now.

buffer underruns because you can't provide data fast enough or the buffer is too small

It seems to me that somehow the buffer underun could be the fault. But a first try to set my buffer size which was 64 to 128 which is the requested channel_len did not solve the problem. but it decreased it a little bit...

Is the AudioEffects Library you mentioned able to play wav files from sd card almost instantly so you could use it as a drum sampler? If it can be used musically, I would like to try it.

synchronization issues between the 2 cores
I dont know anything about this, I dont even know if the code is using both cores because the Arduino settings say: Events on: Core 1 and Arduino on: Core 1...

Looking at your purpose, my recommendation would be not to use BluetoothA2DPSource and stick with something which has a low risk of failure. With it's high sample rate it puts the processor at the edge and there are plenty of stumbling blocks. The BluetoothA2DPSink is working very reliable however.

I dont understand this. How can I use the Sink to send audio to a speaker?

After some a second thought however I am not sure if this is a good idea in a school setting where potentially many users might send a request at the same time to the same resource!

Mh. I wanna try UDP to send one byte in less than a millisecond as soon as soon as I have a sedond audio Kit board to try it. I have no experience what protocol, (bluetooth serial, udp, esp-now, or sth else) is suited best for these kind of triggers and 6 children walking around 10m away from a receiving device. What is your feeling ? What could work?

Thank you for helping!
-paul

[pschatzmann]

• I am usually using an NBuffer with 1024 X 5 buffers from the audio tools.
• You can find out the core by calling xPortGetCoreID() in the callback.
• You are right BluetoothA2DPSink does not help for your purpose. But I think it would be cool if you show that the AudioKit can be a Bluetooth Speaker.
• I suggest to use UDP: There you just need to know the ip address of the receiving device or use ESP-Now if you need to control who can communicate with whon?