Playing an audio-file on one core stops animation on other core

[ld21]

Hi community

I'm using Core 0 on an ESP32-WROOM-32E to continously calculate and update a 8x8 WS2812-Display and Core 1 to play WAV-files after a button-press. The setup (using a mutex/semaphore, to prevent the audioplayer from disturbing the updating of the WS2812-Leds) works – but when the audioplayer starts (and ends) playing a sound, the ESP32 seems to be blocked for a short while (interestingly both cores) (Video here). I think it has something to do with audioplayer.begin(); (and the closing of the file) which take a lot of time (could that process be speeded up?).

Does anybody understand where the pausing in the beginning and end of playing a flie is coming from?

Thanks, Dani

/* 
 * Versuch, auf einem Core des ESP32 eine WS2812b-Animation laufen zu lassen und
 * auf dem anderen Core (verzögerungsfrei) ein Soundfile per I2S auszugeben
 * Ich folgte für die Core-Verteilung mit https://www.youtube.com/watch?v=k_D_Qu0cgu8
 * für den Semaphore/Mutex-Einsatz https://wolles-elektronikkiste.de/freertos-mit-esp32-und-arduino-nutzen#mutex
 * Funktioniert (nicht) auf einem ESP32 mit
 * - 1 Taste an GND und den Pins 16
 * - 1000µF Kondensator zwischen GND und 5V
 * - 1 8x8 RGB Neopixel-Matrix über 330 Ohm Widerstand an Pin 27 befestigt
 * - I2S-Verstärker/Decoder an Pins 15 (WS), 14 (BCK) und 25 (DATA)
 * Sound-Dateien liegen auf einem SPIFFS-System im WAV-Format
 * Problem: Sound verzögert die Anzeige der WS2812b-Marix
 */

#include <Adafruit_GFX.h>
#include <Adafruit_NeoMatrix.h>
#include <Adafruit_NeoPixel.h>

#define NEOPIXEL_PIN 27
#define SHOOT_PIN 16

Adafruit_NeoMatrix matrix = Adafruit_NeoMatrix(8, 8, NEOPIXEL_PIN,
  NEO_MATRIX_BOTTOM     + NEO_MATRIX_RIGHT +
  NEO_MATRIX_ROWS + NEO_MATRIX_PROGRESSIVE,
  NEO_RGB            + NEO_KHZ800);

#include "AudioTools.h"
#include "AudioLibs/AudioSourceSPIFFS.h"
#include "AudioCodecs/CodecWAV.h"
const char *startFilePath="/";
const char* ext="wav";
AudioSourceSPIFFS source(startFilePath, ext);
I2SStream i2s;
WAVDecoder decoder;
AudioPlayer audioplayer(source, i2s, decoder);

SemaphoreHandle_t mutex; // Create handle

struct button{
  uint8_t pin;
};

button shoot = {.pin = SHOOT_PIN};

int x[] = {0,1,2,3,4,5,6,7,7,7,7,7,7,7,7,6,5,4,3,2,1,0,0,0,0,0,0,0};
int y[] = {0,0,0,0,0,0,0,0,1,2,3,4,5,6,7,7,7,7,7,7,7,7,6,5,4,3,2,1};

unsigned long long next_frame;               // Millis, wann es wieder Zeit ist, ein Frame zu rechnen und darzustellen
unsigned int millis_between_frames = 100;    // Millis zwischen einzelnen Frames (--> 10 FPS)

TaskHandle_t Task0; // https://www.youtube.com/watch?v=k_D_Qu0cgu8
unsigned long long counter = 0;

unsigned int ecke2 = random(4,14);
unsigned int ecke3 = random(4,14);

void setup() {
  Serial.begin(57600);             // Verbindung mit dem Compi für's Debugging
  pinMode(shoot.pin, INPUT_PULLUP);

  mutex = xSemaphoreCreateMutex(); //Create the mutex object

  matrix.begin();                  // Matrix wird initialisiert
  matrix.setBrightness(40);

  // Audio-Setup
  // AudioLogger::instance().begin(Serial, AudioLogger::Debug); // Info, Warning, Error or Debug
  auto cfg = i2s.defaultConfig();
  // Standard-I2S-Pins stehen unter: https://github.com/pschatzmann/arduino-audio-tools/blob/main/src/AudioConfig.h
  cfg.pin_ws = 15;
  cfg.pin_bck = 14;
  cfg.pin_data = 25;
  // _____________________
  cfg.sample_rate = 8000;
  cfg.channels = 1;
  i2s.begin(cfg);
  // Ende Audio-Setup

  xTaskCreatePinnedToCore( // Matrix-Animation
    task_for_core_0,  /* aufgerufene Funktion*/
    "Task0",          /* Name des Tasks (String)*/
    32000,             /* Stack-Reservation */
    NULL,             /* Parameter */
    1,                /* Priorität des Tasks (1 = Normal, 2 = Höher als normal) */
    &Task0,           /* Task-Handle (siehe oben deklariert) */
    0                 /* Core, auf dem der Task laufen soll */
  );
}

void loop() { // Sound läuft auf Core 1
  xSemaphoreTake(mutex,portMAX_DELAY); // Take the mutex
  audioplayer.copy();
  if(!audioplayer.isActive()){
    if(digitalRead(shoot.pin) == LOW){
      source.setFileFilter("treffer.wav");             // Tonfile setzen
      audioplayer.begin(); 
    }
  }
  xSemaphoreGive(mutex); // Releases the mutex
}

void task_for_core_0(void * parameter){ // Alles mit dem Display läuft auf Core 0; Kontinuierlich wird das 3eck gedreht
  for(;;){
    if(millis() > next_frame){          // Wenn es Zeit für ein neues Frame ist
      next_frame = millis() + millis_between_frames;  // Wird die Zeit für das nächste Frame berechnet
      matrix.fillScreen(0);            // Display-RAM gelöscht
      counter++;
      matrix.drawTriangle(x[(counter)%28],y[(counter)%28],x[(counter+ecke2)%28],y[(counter+ecke2)%28],x[(counter+ecke2+ecke3)%28],y[(counter+ecke2+ecke3)%28],matrix.Color(0,0,255));
      matrix.writePixel(3,3,matrix.Color(255,0,0));
      xSemaphoreTake(mutex,portMAX_DELAY); // Take the mutex
      matrix.show();                   // Display-RAM zum Display schicken
      xSemaphoreGive(mutex); // Releases the mutex
    }
  }
}

[pschatzmann]

I don't understand why you want to use a Mutex. I don't see any shared variables that you need to protect or am I missing something ?

With the Mutex you stop the GUI rendering whenever some Audio is played and you stop the Audio from playing whenever someting is rendered to the Screen.

[ld21]

Agreed, this seems to be weird. However leaving the mutex away, the start and ending of the audio-playing do not only make the other core pause, but also disturb the rendering of the display as can be seen in this video. So there seems to be a shared resource somehow...?

[pschatzmann]

Not sure why it is doing that. The only potential issue that I see in your code is a missing delay in your task_for_core_0