I2S recoding to SD card choppy (with esp32-s3 and INMP441)

[gohai]

Hello @pschatzmann et al,

Could you please tell me what I am doing wrong in the code below. This captures speech, but the recording has audible glitches at irregular intervals - sample recording here.

This is with an esp32-s3 and INMP441, with latest arduino-esp32 and arduino-audio-tools from GitHub.

#include <AudioTools.h>
#include <FS.h>
#include <SD_MMC.h>

#define MIC_SD 2
#define MIC_WS 3
#define MIC_SCK 4
#define SD_CLK 12
#define SD_CMD 11
#define SD_D0 13

File file;
I2SStream micStream;
//ConverterFillLeftAndRight<int16_t> left_and_right(RightIsEmpty);
//ConverterStream<int16_t> converter(micStream, left_and_right);
EncodedAudioStream encoder(&file, new WAVEncoder());
StreamCopy copier;

bool recording = false;
uint32_t startTime;

void setup() {
  Serial.begin(115200);

  initSd();
  startRecording("/test.wav");
  recording = true;
  startTime = millis();
}

void loop() {
  if (recording) {
    copier.copy();

    if (millis()-startTime > 5000) {
      stopRecording();
      recording = false;
      Serial.println("Finished");
    }
  }
}

bool initSd() {
  if (!SD_MMC.setPins(SD_CLK, SD_CMD, SD_D0)) {
    Serial.println("SD_MMC.setPins() failed");
    return false;
  }
  if (!SD_MMC.begin("/sdcard", true)) {
    Serial.println("SD_MMC.begin() failed");
    return false;
  }
  return true;
}

bool startRecording(const char* fn) {
  file = SD_MMC.open(fn, FILE_WRITE);
  if (!file) {
    Serial.println("SD_MMC.open() failed");
    return false;
  }

  auto cfg = micStream.defaultConfig(RX_MODE);
  cfg.sample_rate = 22050;
  cfg.channels = 1;
  cfg.channel_format = I2SChannelSelect::Left;
  cfg.bits_per_sample = 16;
  cfg.pin_bck = MIC_SCK;
  cfg.pin_ws = MIC_WS;
  cfg.pin_data = MIC_SD;

  if (!micStream.begin(cfg)) {
    Serial.println("micStream.begin() failed");
    file.close();
    return false;
  }
  if (!encoder.begin(cfg)) {
    Serial.println("encoder.begin() failed");
    file.close();
    return false;
  }
  //copier.begin(encoder, converter);  // didn't make a difference
  copier.begin(encoder, micStream);
  return true;
}

void stopRecording() {
  micStream.end();
  encoder.end();
  copier.end();
  file.close();
}

Thank you for your time and patience.

[pschatzmann]

Please follow the recommendations diven here!

[gohai]

The waveform looks clean when I plot the microphone data via serial, @pschatzmann . (Screen recording) What could make a difference when writing it to the SD card instead?

Here the code I used, for reference:

#include "AudioTools.h"

#define MIC_SD 2
#define MIC_WS 3
#define MIC_SCK 4

I2SStream i2sStream;
CsvOutput<int32_t> csvOutput(Serial);
StreamCopy copier(csvOutput, i2sStream);

void setup(void) {
    Serial.begin(115200);
    AudioToolsLogger.begin(Serial, AudioToolsLogLevel::Info);
    
    auto cfg = i2sStream.defaultConfig(RX_MODE);
    cfg.sample_rate = 22050;
    cfg.channels = 1;
    cfg.channel_format = I2SChannelSelect::Left;
    cfg.bits_per_sample = 16;
    cfg.pin_bck = MIC_SCK;
    cfg.pin_ws = MIC_WS;
    cfg.pin_data = MIC_SD;
    i2sStream.begin(cfg);

    csvOutput.begin(cfg);
}

void loop() {
    copier.copy();
}

[pschatzmann]

Then test the SD write speed: it needs to be higher than the generated i2s data!
You can try to increase the SPI frequency...

[gohai]

I did most of my testing on 3.3.0, but I also tried with 3.2.1 and 3.1.3 once. (3.1.3 showed some unrelated issue, which made me believe that at least my downgrading had worked.)

[pschatzmann]

Did you try with the same parameters like in the Python example (pins, BUFFER_LENGTH_IN_BYTES, SPI frequency) ?

I will try to do a test with 3.3.0 on an AudioKit later today...

[gohai]

It's on slightly different hardware (S2 rather than S3), but I used arbitrary GPIOs pins for I2S for both.

BUFFER_LENGTH_IN_BYTES ... how would I set the equivalent in arduino-audio-tools? (Increasing I2S_BUFFER_COUNT to 16-20 and/or I2S_BUFFER_SIZE to 1024 in AudioToolsConfig.h didn't make a difference when I tried. Is there anything else? I tried giving the copier a larger buffer size at some point as well.)

[pschatzmann]

cfg.buffer.size = 1024; cfg.buffer_count = 40;

Not sure what SD API the Python example is using: Did you try with the SD.h library instead of SDMMC ?

Your test skech sine recording sketch showed that the I2S was working properly: so the issue must come from the interaction of I2S and the SD...

[gohai]

Now using the same hardware (S2) and same pins, and it still glitches for me with Arduino 🤨 Increasing the buffer size and count in the way you described didn't fix it either, unfortunately.

On the S2 I ended up using SD, since it lacks the MMC hardware peripheral.

Here is my current test sketch fwiw:

#include <AudioTools.h>
#include <FS.h>
#include <SD.h>
#include <SPI.h>

#define MIC_SD 2
#define MIC_WS 3
#define MIC_SCK 4
#define SD_SCK 14
#define SD_MOSI 13
#define SD_MISO 12
#define SD_CS 11

File file;
I2SStream micStream;
EncodedAudioStream encoder(&file, new WAVEncoder());
StreamCopy copier;

bool recording = false;
uint32_t startTime;

void setup() {
  Serial.begin(115200);

  initSd();
  startRecording("/test.wav");
  recording = true;
  startTime = millis();
}

void loop() {
  if (recording) {
    copier.copy();

    if (millis()-startTime > 5000) {
      stopRecording();
      recording = false;
      Serial.println("Finished");
    }
  }
}

bool initSd() {
  // using SPI on ESP32-S2
  if (!SPI.begin(SD_SCK, SD_MISO, SD_MOSI, SD_CS)) {
    Serial.println("SPI.begin() failed");
    return false;
  }
  if (!SD.begin(SD_CS)) {
    Serial.println("SD.begin() failed");
    return false;
  }
  return true;
}

bool startRecording(const char* fn) {
  file = SD.open(fn, FILE_WRITE);
  if (!file) {
    Serial.println("SD.open() failed");
    return false;
  }

  auto cfg = micStream.defaultConfig(RX_MODE);
  cfg.sample_rate = 22050;
  cfg.channels = 1;
  cfg.channel_format = I2SChannelSelect::Left;
  cfg.bits_per_sample = 16;
  cfg.pin_bck = MIC_SCK;
  cfg.pin_ws = MIC_WS;
  cfg.pin_data = MIC_SD;
  cfg.buffer_size = 1024;
  cfg.buffer_count = 40;

  if (!micStream.begin(cfg)) {
    Serial.println("micStream.begin() failed");
    file.close();
    return false;
  }
  if (!encoder.begin(cfg)) {
    Serial.println("encoder.begin() failed");
    file.close();
    return false;
  }
  copier.begin(encoder, micStream);
  return true;
}

void stopRecording() {
  micStream.end();
  encoder.end();
  copier.end();
  file.close();
}

[pschatzmann]

I did some tests (both with ESP 3.3.0 and ESP 3.2.1 ) with the AudioKit using this sketch and could not reproduce your issue:

The AudioBoardStream is just an I2SStream that also sets up the codec chip.

ps. I also made some tests with your sketch and ended up with empty files. To correct this, I made sure

• to delete the file before the processing with SD.remove(fn);
• commented out the cfg.buffer_size and cfg.buffer_count
• changed stopRecording() to

void stopRecording() {
  // close file first and print file size
  file.close();
  file = SD.open(fn, FILE_READ);
  Serial.print("File Size: ");
  Serial.println(file.size());
  file.close();

  micStream.end();
  encoder.end();
  copier.end();
}