Support for microWakeWord

[johnboiles]

I'm trying to get the voice-assistant working with microWakeWord. I'm close. It works mostly but the speaker audio is very choppy and the microphone isn't as reliable as it is when running on my M5Stack Atom Echo (using the official esphome yaml for that device).

Here's what I have so far. It's a merge of the latest version of the s3-box yaml and the M5Stack-provided non-microWakeWord version.

I bet it's most likely something about not configuring the microphone and speaker as esp-adf audio devices. Even for the official file for the S3 Box this comes from a draft PR.

cores3-voice-assistant.yaml

substitutions:
  name: core-s3-voice
  friendly_name: Core S3 Voice Assistant
  loading_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/loading_320_240.png
  idle_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/idle_320_240.png
  listening_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/listening_320_240.png
  thinking_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/thinking_320_240.png
  replying_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/replying_320_240.png
  error_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/error_320_240.png
  timer_finished_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/timer_finished_320_240.png

  loading_illustration_background_color: "000000"
  idle_illustration_background_color: "000000"
  listening_illustration_background_color: "FFFFFF"
  thinking_illustration_background_color: "FFFFFF"
  replying_illustration_background_color: "FFFFFF"
  error_illustration_background_color: "000000"

  voice_assist_idle_phase_id: "1"
  voice_assist_listening_phase_id: "2"
  voice_assist_thinking_phase_id: "3"
  voice_assist_replying_phase_id: "4"
  voice_assist_not_ready_phase_id: "10"
  voice_assist_error_phase_id: "11"
  voice_assist_muted_phase_id: "12"
  voice_assist_timer_finished_phase_id: "20"

  # These unique characters have been extracted from every test file of every language available on https://github.com/home-assistant/intents (14 March 2024)
  allowed_characters: " !#%'()+,-./0123456789:;<>?@ABCDEFGHIJKLMNOPQRSTUVWYZ[]_abcdefghijklmnopqrstuvwxyz{|}°²³µ¿ÁÂÄÅÉÖÚßàáâãäåæçèéêëìíîðñòóôõöøùúûüýþāăąćčďĐđēėęěğĮįıļľŁłńňőřśšťũūůűųźŻżŽžơưșțΆΈΌΐΑΒΓΔΕΖΗΘΚΜΝΠΡΣΤΥΦάέήίαβγδεζηθικλμνξοπρςστυφχψωϊόύώАБВГДЕЖЗИКЛМНОПРСТУХЦЧШЪЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяёђєіїјљњћאבגדהוזחטיכלםמןנסעפץצקרשת،ءآأإئابةتجحخدذرزسشصضطظعغفقكلمنهوىيٹپچڈکگںھہیےংকচতধনফবযরলশষসచయలഅആഇഈഉഎഓകഗങചജഞടഡണതദധനപഫബഭമയരറലളവശസഹൺൻർൽൾაბგდევზთილმნოპრსტუფქყშჩცძჭხạảấầẩậắặẹẽếềểệỉịọỏốồổỗộớờởợụủứừửữựỳ—、一上不个中为主乾了些亮人任低佔何作供依侧係個側偵充光入全关冇冷几切到制前動區卧厅厨及口另右吊后吗启吸呀咗哪唔問啟嗎嘅嘛器圍在场執場外多大始安定客室家密寵对將小少左已帘常幫幾库度庫廊廚廳开式後恆感態成我戲戶户房所扇手打执把拔换掉控插摄整斯新明是景暗更最會有未本模機檯櫃欄次正氏水沒没洗活派温測源溫漏潮激濕灯為無煙照熱燈燥物狀玄现現瓦用發的盞目着睡私空窗立笛管節簾籬紅線红罐置聚聲脚腦腳臥色节著行衣解設調請謝警设调走路車车运連遊運過道邊部都量鎖锁門閂閉開關门闭除隱離電震霧面音頂題顏颜風风食餅餵가간감갔강개거게겨결경고공과관그금급기길깥꺼껐꼽나난내네놀누는능니다닫담대더데도동됐되된됨둡드든등디때떤뜨라래러렇렌려로료른를리림링마많명몇모무문물뭐바밝방배변보부불블빨뽑사산상색서설성세센션소쇼수스습시신실싱아안않알았애야어얼업없었에여연열옆오온완외왼요운움워원위으은을음의이인일임입있작잠장재전절정제져조족종주줄중줘지직진짐쪽차창천최추출충치침커컴켜켰쿠크키탁탄태탬터텔통트튼티파팬퍼폰표퓨플핑한함해했행혀현화활후휴힘，？"

  micro_wake_word_model: okay_nabu

esphome:
  name: ${name}
  friendly_name: ${friendly_name}
  min_version: 2024.7.0
  name_add_mac_suffix: false
  platformio_options:
    board_build.flash_mode: dio
  libraries:
    - m5stack/M5GFX@^0.1.11
    - m5stack/M5Unified@^0.1.11
  on_boot:
    priority: 600
    then:
      - script.execute: draw_display
      - delay: 30s
      - if:
          condition:
            lambda: return id(init_in_progress);
          then:
            - lambda: id(init_in_progress) = false;
            - script.execute: draw_display

esp32:
  board: esp32-s3-devkitc-1
  flash_size: 16MB
  framework:
    type: esp-idf
    sdkconfig_options:
      CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240: "y"
      CONFIG_ESP32S3_DATA_CACHE_64KB: "y"
      CONFIG_ESP32S3_DATA_CACHE_LINE_64B: "y"
      CONFIG_AUDIO_BOARD_CUSTOM: "y"
      CONFIG_ESP32_S3_BOX_3_BOARD: "y"

psram:
  mode: quad
  speed: 80MHz

external_components:
  - source:
      type: git
      url: https://github.com/m5stack/M5CoreS3-Esphome
    components: [ board_m5cores3, m5cores3_audio, m5cores3_display ]
    refresh: 0s
  - source: github://pr#5230
    components: esp_adf
    refresh: 0s
  - source: github://jesserockz/esphome-components
    components: [file]
    refresh: 0s

api:
  encryption:
    # Replace the key by your API key
    key: "YOURKEYHERE"
  on_client_connected:
    - script.execute: draw_display
  on_client_disconnected:
    - script.execute: draw_display

ota:
  - platform: esphome
    password: "YOURPASSWORDHERE"
    id: common_ota

logger:
  hardware_uart: USB_SERIAL_JTAG

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  fast_connect: true # Required for hidden AP
  on_connect:
    - script.execute: draw_display
  on_disconnect:
    - script.execute: draw_display

# button:
#   - platform: factory_reset
#     id: factory_reset_btn
#     internal: true

# binary_sensor:
#   - platform: gpio
#     pin:
#       number: GPIO0
#       mode: INPUT_PULLUP
#       inverted: true
#     id: top_left_button
#     internal: true
#     on_multi_click:
#       - timing:
#           - ON for at least 50ms
#           - OFF for at least 50ms
#         then:
#           - switch.turn_off: timer_ringing
#       - timing:
#           - ON for at least 10s
#         then:
#           - button.press: factory_reset_btn

# output:
#   - platform: ledc
#     pin: GPIO47
#     id: backlight_output

# light:
#   - platform: monochromatic
#     id: led
#     name: Screen
#     icon: "mdi:television"
#     entity_category: config
#     output: backlight_output
#     restore_mode: RESTORE_DEFAULT_ON
#     default_transition_length: 250ms

# What is the difference in the esp_adf stuff for the box3 vs the cores3
# esp_adf:
#   board: esp32s3box3

# microphone:
#   - platform: esp_adf
#     id: box_mic

# speaker:
#   - platform: esp_adf
#     id: box_speaker

board_m5cores3:
m5cores3_audio:
  id: m5cores3_audio_1

microphone:
  - platform: m5cores3_audio
    m5cores3_audio_id: m5cores3_audio_1
    id: box_mic
    adc_type: external
    i2s_din_pin: 14
    pdm: false

speaker:
  - platform: m5cores3_audio
    m5cores3_audio_id: m5cores3_audio_1
    id: box_speaker
    dac_type: external
    i2s_dout_pin: 13
    mode: mono

micro_wake_word:
  models:
    - ${micro_wake_word_model}
  on_wake_word_detected:
    - voice_assistant.start:
        wake_word: !lambda return wake_word;

voice_assistant:
  id: va
  microphone: box_mic
  speaker: box_speaker
  # esp_adf only
  # noise_suppression_level: 2
  # auto_gain: 31dBFS
  # volume_multiplier: 2.0
  # vad_threshold: 3
  on_listening:
    - lambda: id(voice_assistant_phase) = ${voice_assist_listening_phase_id};
    - text_sensor.template.publish:
        id: text_request
        state: "..."
    - text_sensor.template.publish:
        id: text_response
        state: "..."
    - script.execute: draw_display
  on_stt_vad_end:
    - lambda: id(voice_assistant_phase) = ${voice_assist_thinking_phase_id};
    - script.execute: draw_display
  on_stt_end:
    - text_sensor.template.publish:
        id: text_request
        state: !lambda return x;
    - script.execute: draw_display
  on_tts_start:
    - text_sensor.template.publish:
        id: text_response
        state: !lambda return x;
  on_tts_stream_start:
    - lambda: id(voice_assistant_phase) = ${voice_assist_replying_phase_id};
    - script.execute: draw_display
  on_tts_stream_end:
    - if:
        condition:
          switch.is_off: mute
        then:
          - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
        else:
          - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
    - script.execute: draw_display
  on_end:
    - wait_until:
        not:
          voice_assistant.is_running:
    - if:
        condition:
          and:
            - switch.is_off: mute
            - lambda: return id(wake_word_engine_location).state == "On device";
            - lambda: return id(voice_assistant_phase) != ${voice_assist_timer_finished_phase_id};
        then:
          - micro_wake_word.start:
  on_error:
    - if:
        condition:
          lambda: return !id(init_in_progress);
        then:
          - lambda: id(voice_assistant_phase) = ${voice_assist_error_phase_id};
          - script.execute: draw_display
          - delay: 1s
          - if:
              condition:
                switch.is_off: mute
              then:
                - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
              else:
                - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
          - script.execute: draw_display
  on_client_connected:
    - lambda: id(init_in_progress) = false;
    - script.execute: start_voice_assistant
    - script.execute: draw_display
  on_client_disconnected:
    - script.execute: stop_voice_assistant
    - script.execute: draw_display
  on_timer_started:
    - script.execute: draw_display
  on_timer_cancelled:
    - script.execute: draw_display
  on_timer_updated:
    - script.execute: draw_display
  on_timer_tick:
    - script.execute: draw_display
  on_timer_finished:
    - script.execute: stop_voice_assistant
    - lambda: id(voice_assistant_phase) = ${voice_assist_timer_finished_phase_id};
    - switch.turn_on: timer_ringing
    - script.execute: draw_display
    - wait_until:
        not:
          microphone.is_capturing:
    - while:
        condition:
          switch.is_on: timer_ringing
        then:
          - lambda: id(box_speaker).play(id(timer_finished_wave_file), sizeof(id(timer_finished_wave_file)));
          - delay: 1s
    - wait_until:
        not:
          speaker.is_playing:
    - switch.turn_off: timer_ringing
    - script.execute: start_voice_assistant
    - script.execute: draw_display

script:
  - id: draw_display
    then:
      - if:
          condition:
            lambda: return !id(init_in_progress);
          then:
            - if:
                condition:
                  wifi.connected:
                then:
                  - if:
                      condition:
                        api.connected:
                      then:
                        - lambda: |
                            switch(id(voice_assistant_phase)) {
                              case ${voice_assist_listening_phase_id}:
                                id(s3_box_lcd).show_page(listening_page);
                                id(s3_box_lcd).update();
                                break;
                              case ${voice_assist_thinking_phase_id}:
                                id(s3_box_lcd).show_page(thinking_page);
                                id(s3_box_lcd).update();
                                break;
                              case ${voice_assist_replying_phase_id}:
                                id(s3_box_lcd).show_page(replying_page);
                                id(s3_box_lcd).update();
                                break;
                              case ${voice_assist_error_phase_id}:
                                id(s3_box_lcd).show_page(error_page);
                                id(s3_box_lcd).update();
                                break;
                              case ${voice_assist_muted_phase_id}:
                                id(s3_box_lcd).show_page(muted_page);
                                id(s3_box_lcd).update();
                                break;
                              case ${voice_assist_not_ready_phase_id}:
                                id(s3_box_lcd).show_page(no_ha_page);
                                id(s3_box_lcd).update();
                                break;
                              case ${voice_assist_timer_finished_phase_id}:
                                id(s3_box_lcd).show_page(timer_finished_page);
                                id(s3_box_lcd).update();
                                break;
                              default:
                                id(s3_box_lcd).show_page(idle_page);
                                id(s3_box_lcd).update();
                            }
                      else:
                        - display.page.show: no_ha_page
                        - component.update: s3_box_lcd
                else:
                  - display.page.show: no_wifi_page
                  - component.update: s3_box_lcd
          else:
            - display.page.show: initializing_page
            - component.update: s3_box_lcd

  - id: fetch_first_active_timer
    then:
      - lambda: |
          const auto timers = id(va).get_timers();
          auto output_timer = timers.begin()->second;
          for (auto &iterable_timer : timers) {
            if (iterable_timer.second.is_active && iterable_timer.second.seconds_left <= output_timer.seconds_left) {
              output_timer = iterable_timer.second;
            }
          }
          id(global_first_active_timer) = output_timer;
  - id: check_if_timers_active
    then:
      - lambda: |
          const auto timers = id(va).get_timers();
          bool output = false;
          if (timers.size() > 0) {
            for (auto &iterable_timer : timers) {
              if(iterable_timer.second.is_active) {
                output = true;
              }
            }
          }
          id(global_is_timer_active) = output;
  - id: fetch_first_timer
    then:
      - lambda: |
          const auto timers = id(va).get_timers();
          auto output_timer = timers.begin()->second;
          for (auto &iterable_timer : timers) {
            if (iterable_timer.second.seconds_left <= output_timer.seconds_left) {
              output_timer = iterable_timer.second;
            }
          }
          id(global_first_timer) = output_timer;
  - id: check_if_timers
    then:
      - lambda: |
          const auto timers = id(va).get_timers();
          bool output = false;
          if (timers.size() > 0) {
            output = true;
          }
          id(global_is_timer) = output;

  - id: draw_timer_timeline
    then:
      - lambda: |
          id(check_if_timers_active).execute();
          id(check_if_timers).execute();
          if (id(global_is_timer_active)){
            id(fetch_first_active_timer).execute();
            int active_pixels = round( 320 * id(global_first_active_timer).seconds_left / max(id(global_first_active_timer).total_seconds , static_cast<uint32_t>(1)) );
            if (active_pixels > 0){
              id(s3_box_lcd).filled_rectangle(0 , 225 , 320 , 15 , Color::WHITE );
              id(s3_box_lcd).filled_rectangle(0 , 226 , active_pixels , 13 , id(active_timer_color) );
            }
          } else if (id(global_is_timer)){
            id(fetch_first_timer).execute();
            int active_pixels = round( 320 * id(global_first_timer).seconds_left / max(id(global_first_timer).total_seconds , static_cast<uint32_t>(1)));
            if (active_pixels > 0){
              id(s3_box_lcd).filled_rectangle(0 , 225 , 320 , 15 , Color::WHITE );
              id(s3_box_lcd).filled_rectangle(0 , 226 , active_pixels , 13 , id(paused_timer_color) );
            }
          }
  - id: draw_active_timer_widget
    then:
      - lambda: |
          id(check_if_timers_active).execute();
          if (id(global_is_timer_active)){
            id(s3_box_lcd).filled_rectangle(80 , 40 , 160 , 50 , Color::WHITE );
            id(s3_box_lcd).rectangle(80 , 40 , 160 , 50 , Color::BLACK );

            id(fetch_first_active_timer).execute();
            int hours_left = floor(id(global_first_active_timer).seconds_left / 3600);
            int minutes_left = floor((id(global_first_active_timer).seconds_left - hours_left * 3600) / 60);
            int seconds_left = id(global_first_active_timer).seconds_left - hours_left * 3600 - minutes_left * 60 ;
            auto display_hours = (hours_left < 10 ? "0" : "") + std::to_string(hours_left);
            auto display_minute = (minutes_left < 10 ? "0" : "") + std::to_string(minutes_left);
            auto display_seconds = (seconds_left  < 10 ? "0" : "") + std::to_string(seconds_left) ;

            std::string display_string = "";
            if (hours_left > 0) {
              display_string = display_hours + ":" + display_minute;
            } else {
              display_string = display_minute + ":" + display_seconds;
            }
            id(s3_box_lcd).printf(120, 47, id(font_timer), Color::BLACK, "%s", display_string.c_str());
          }

  - id: start_voice_assistant
    then:
      - if:
          condition:
            switch.is_off: mute
          then:
            - if:
                condition:
                  lambda: return id(wake_word_engine_location).state == "In Home Assistant";
                then:
                  - lambda: id(va).set_use_wake_word(true);
                  - voice_assistant.start_continuous:
            - if:
                condition:
                  lambda: return id(wake_word_engine_location).state == "On device";
                then:
                  - lambda: id(va).set_use_wake_word(false);
                  - micro_wake_word.start
            - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
          else:
            - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};

  - id: stop_voice_assistant
    then:
      - if:
          condition:
            lambda: return id(wake_word_engine_location).state == "In Home Assistant";
          then:
            - lambda: id(va).set_use_wake_word(false);
            - voice_assistant.stop:
      - if:
          condition:
            lambda: return id(wake_word_engine_location).state == "On device";
          then:
            - voice_assistant.stop:
            - micro_wake_word.stop:
      - lambda: id(voice_assistant_phase) = ${voice_assist_not_ready_phase_id};

switch:
  - platform: template
    name: Mute
    id: mute
    icon: "mdi:microphone-off"
    optimistic: true
    restore_mode: RESTORE_DEFAULT_OFF
    entity_category: config
    on_turn_off:
      - if:
          condition:
            lambda: return !id(init_in_progress);
          then:
            - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
            - if:
                condition:
                  not:
                    - voice_assistant.is_running
                then:
                  - if:
                      condition:
                        lambda: return id(wake_word_engine_location).state == "In Home Assistant";
                      then:
                        - lambda: id(va).set_use_wake_word(true);
                        - voice_assistant.start_continuous
                  - if:
                      condition:
                        lambda: return id(wake_word_engine_location).state == "On device";
                      then:
                        - lambda: id(va).set_use_wake_word(false);
                        - micro_wake_word.start
            - script.execute: draw_display
    on_turn_on:
      - if:
          condition:
            lambda: return !id(init_in_progress);
          then:
            - lambda: id(va).set_use_wake_word(false);
            - voice_assistant.stop
            - micro_wake_word.stop
            - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
            - script.execute: draw_display

  - platform: template
    id: timer_ringing
    optimistic: true
    internal: true
    restore_mode: ALWAYS_OFF
    on_turn_on:
      - delay: 15min
      - switch.turn_off: timer_ringing

select:
  - platform: template
    entity_category: config
    name: Wake word engine location
    id: wake_word_engine_location
    icon: "mdi:account-voice"
    optimistic: true
    restore_value: true
    options:
      - In Home Assistant
      - On device
    initial_option: On device
    on_value:
      - if:
          condition:
            lambda: return !id(init_in_progress);
          then:
            - wait_until:
                lambda: return id(voice_assistant_phase) == ${voice_assist_muted_phase_id} || id(voice_assistant_phase) == ${voice_assist_idle_phase_id};
            - if:
                condition:
                  lambda: return x == "In Home Assistant";
                then:
                  - micro_wake_word.stop
                  - delay: 500ms
                  - if:
                      condition:
                        switch.is_off: mute
                      then:
                        - lambda: id(va).set_use_wake_word(true);
                        - voice_assistant.start_continuous:
            - if:
                condition:
                  lambda: return x == "On device";
                then:
                  - lambda: id(va).set_use_wake_word(false);
                  - voice_assistant.stop
                  - delay: 500ms
                  - if:
                      condition:
                        switch.is_off: mute
                      then:
                        - micro_wake_word.start

globals:
  - id: init_in_progress
    type: bool
    restore_value: false
    initial_value: "true"
  - id: voice_assistant_phase
    type: int
    restore_value: false
    initial_value: ${voice_assist_not_ready_phase_id}
  - id: global_first_active_timer
    type: voice_assistant::Timer
    restore_value: false
  - id: global_is_timer_active
    type: bool
    restore_value: false
  - id: global_first_timer
    type: voice_assistant::Timer
    restore_value: false
  - id: global_is_timer
    type: bool
    restore_value: false

image:
  - file: ${error_illustration_file}
    id: casita_error
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: ${idle_illustration_file}
    id: casita_idle
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: ${listening_illustration_file}
    id: casita_listening
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: ${thinking_illustration_file}
    id: casita_thinking
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: ${replying_illustration_file}
    id: casita_replying
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: ${timer_finished_illustration_file}
    id: casita_timer_finished
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: ${loading_illustration_file}
    id: casita_initializing
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: https://github.com/esphome/wake-word-voice-assistants/raw/main/error_box_illustrations/error-no-wifi.png
    id: error_no_wifi
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: https://github.com/esphome/wake-word-voice-assistants/raw/main/error_box_illustrations/error-no-ha.png
    id: error_no_ha
    resize: 320x240
    type: RGB24
    use_transparency: true

font:
  - file:
      type: gfonts
      family: Figtree
      weight: 300
      italic: true
    glyphs: ${allowed_characters}
    id: font_request
    size: 15
  - file:
      type: gfonts
      family: Figtree
      weight: 300
    glyphs: ${allowed_characters}
    id: font_response
    size: 15
  - file:
      type: gfonts
      family: Figtree
      weight: 300
    glyphs: ${allowed_characters}
    id: font_timer
    size: 30

text_sensor:
  - id: text_request
    platform: template
    on_value:
      lambda: |-
        if(id(text_request).state.length()>32) {
          std::string name = id(text_request).state.c_str();
          std::string truncated = esphome::str_truncate(name.c_str(),31);
          id(text_request).state = (truncated+"...").c_str();
        }

  - id: text_response
    platform: template
    on_value:
      lambda: |-
        if(id(text_response).state.length()>32) {
          std::string name = id(text_response).state.c_str();
          std::string truncated = esphome::str_truncate(name.c_str(),31);
          id(text_response).state = (truncated+"...").c_str();
        }

color:
  - id: idle_color
    hex: ${idle_illustration_background_color}
  - id: listening_color
    hex: ${listening_illustration_background_color}
  - id: thinking_color
    hex: ${thinking_illustration_background_color}
  - id: replying_color
    hex: ${replying_illustration_background_color}
  - id: loading_color
    hex: ${loading_illustration_background_color}
  - id: error_color
    hex: ${error_illustration_background_color}
  - id: active_timer_color
    hex: "26ed3a"
  - id: paused_timer_color
    hex: "3b89e3"

file:
  - id: timer_finished_wave_file
    file: https://github.com/esphome/wake-word-voice-assistants/raw/main/sounds/timer_finished.wav

display:
  - platform: m5cores3_display
    model: ILI9342
    dc_pin: 35
    update_interval: never
    id: s3_box_lcd
    pages:
      - id: idle_page
        lambda: |-
          it.fill(id(idle_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_idle), ImageAlign::CENTER);
          id(draw_timer_timeline).execute();
          id(draw_active_timer_widget).execute();
      - id: listening_page
        lambda: |-
          it.fill(id(listening_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_listening), ImageAlign::CENTER);
          id(draw_timer_timeline).execute();
      - id: thinking_page
        lambda: |-
          it.fill(id(thinking_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_thinking), ImageAlign::CENTER);
          it.filled_rectangle(20 , 20 , 280 , 30 , Color::WHITE );
          it.rectangle(20 , 20 , 280 , 30 , Color::BLACK );
          it.printf(30, 25, id(font_request), Color::BLACK, "%s", id(text_request).state.c_str());

          id(draw_timer_timeline).execute();
      - id: replying_page
        lambda: |-
          it.fill(id(replying_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_replying), ImageAlign::CENTER);
          it.filled_rectangle(20 , 20 , 280 , 30 , Color::WHITE );
          it.rectangle(20 , 20 , 280 , 30 , Color::BLACK );
          it.filled_rectangle(20 , 190 , 280 , 30 , Color::WHITE );
          it.rectangle(20 , 190 , 280 , 30 , Color::BLACK );
          it.printf(30, 25, id(font_request), Color::BLACK, "%s", id(text_request).state.c_str());
          it.printf(30, 195, id(font_response), Color::BLACK, "%s", id(text_response).state.c_str());
          id(draw_timer_timeline).execute();
      - id: timer_finished_page
        lambda: |-
          it.fill(id(idle_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_timer_finished), ImageAlign::CENTER);
      - id: error_page
        lambda: |-
          it.fill(id(error_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_error), ImageAlign::CENTER);
      - id: no_ha_page
        lambda: |-
          it.image((it.get_width() / 2), (it.get_height() / 2), id(error_no_ha), ImageAlign::CENTER);
      - id: no_wifi_page
        lambda: |-
          it.image((it.get_width() / 2), (it.get_height() / 2), id(error_no_wifi), ImageAlign::CENTER);
      - id: initializing_page
        lambda: |-
          it.fill(id(loading_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_initializing), ImageAlign::CENTER);
      - id: muted_page
        lambda: |-
          it.fill(Color::BLACK);
          id(draw_timer_timeline).execute();
          id(draw_active_timer_widget).execute();

[johnboiles]

For reference, this yaml seemed to work much better, though the wake word detection wasn't consistent for me and I don't love how far it's diverging from the official esphome yamls.

[johnboiles]

Ok I've got this working by ditching the m5stack audio components.

esp_adf:

i2s_audio:
  - id: i2s_audio_bus
    i2s_lrclk_pin: GPIO33
    i2s_bclk_pin: GPIO34
    i2s_mclk_pin: GPIO0

microphone:
  - platform: i2s_audio
    id: box_mic
    i2s_din_pin: GPIO14
    adc_type: external
    pdm: false
    bits_per_sample: 16bit
    channel: right

micro_wake_word:
  models:
    - ${micro_wake_word_model}
  on_wake_word_detected:
    - voice_assistant.start:
        wake_word: !lambda return wake_word;

EDIT: Maybe not. I guess there was some initialization that is not happening here. When I power-cycled this it stopped working.

cores3-voice-assistant.yaml

substitutions:
  name: core-s3-voice
  friendly_name: Core S3 Voice Assistant
  loading_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/loading_320_240.png
  idle_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/idle_320_240.png
  listening_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/listening_320_240.png
  thinking_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/thinking_320_240.png
  replying_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/replying_320_240.png
  error_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/error_320_240.png
  timer_finished_illustration_file: https://github.com/esphome/wake-word-voice-assistants/raw/main/casita/timer_finished_320_240.png

  loading_illustration_background_color: "000000"
  idle_illustration_background_color: "000000"
  listening_illustration_background_color: "FFFFFF"
  thinking_illustration_background_color: "FFFFFF"
  replying_illustration_background_color: "FFFFFF"
  error_illustration_background_color: "000000"

  voice_assist_idle_phase_id: "1"
  voice_assist_listening_phase_id: "2"
  voice_assist_thinking_phase_id: "3"
  voice_assist_replying_phase_id: "4"
  voice_assist_not_ready_phase_id: "10"
  voice_assist_error_phase_id: "11"
  voice_assist_muted_phase_id: "12"
  voice_assist_timer_finished_phase_id: "20"

  # These unique characters have been extracted from every test file of every language available on https://github.com/home-assistant/intents (14 March 2024)
  allowed_characters: " !#%'()+,-./0123456789:;<>?@ABCDEFGHIJKLMNOPQRSTUVWYZ[]_abcdefghijklmnopqrstuvwxyz{|}°²³µ¿ÁÂÄÅÉÖÚßàáâãäåæçèéêëìíîðñòóôõöøùúûüýþāăąćčďĐđēėęěğĮįıļľŁłńňőřśšťũūůűųźŻżŽžơưșțΆΈΌΐΑΒΓΔΕΖΗΘΚΜΝΠΡΣΤΥΦάέήίαβγδεζηθικλμνξοπρςστυφχψωϊόύώАБВГДЕЖЗИКЛМНОПРСТУХЦЧШЪЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюяёђєіїјљњћאבגדהוזחטיכלםמןנסעפץצקרשת،ءآأإئابةتجحخدذرزسشصضطظعغفقكلمنهوىيٹپچڈکگںھہیےংকচতধনফবযরলশষসచయలഅആഇഈഉഎഓകഗങചജഞടഡണതദധനപഫബഭമയരറലളവശസഹൺൻർൽൾაბგდევზთილმნოპრსტუფქყშჩცძჭხạảấầẩậắặẹẽếềểệỉịọỏốồổỗộớờởợụủứừửữựỳ—、一上不个中为主乾了些亮人任低佔何作供依侧係個側偵充光入全关冇冷几切到制前動區卧厅厨及口另右吊后吗启吸呀咗哪唔問啟嗎嘅嘛器圍在场執場外多大始安定客室家密寵对將小少左已帘常幫幾库度庫廊廚廳开式後恆感態成我戲戶户房所扇手打执把拔换掉控插摄整斯新明是景暗更最會有未本模機檯櫃欄次正氏水沒没洗活派温測源溫漏潮激濕灯為無煙照熱燈燥物狀玄现現瓦用發的盞目着睡私空窗立笛管節簾籬紅線红罐置聚聲脚腦腳臥色节著行衣解設調請謝警设调走路車车运連遊運過道邊部都量鎖锁門閂閉開關门闭除隱離電震霧面音頂題顏颜風风食餅餵가간감갔강개거게겨결경고공과관그금급기길깥꺼껐꼽나난내네놀누는능니다닫담대더데도동됐되된됨둡드든등디때떤뜨라래러렇렌려로료른를리림링마많명몇모무문물뭐바밝방배변보부불블빨뽑사산상색서설성세센션소쇼수스습시신실싱아안않알았애야어얼업없었에여연열옆오온완외왼요운움워원위으은을음의이인일임입있작잠장재전절정제져조족종주줄중줘지직진짐쪽차창천최추출충치침커컴켜켰쿠크키탁탄태탬터텔통트튼티파팬퍼폰표퓨플핑한함해했행혀현화활후휴힘，？"

  micro_wake_word_model: okay_nabu

esphome:
  name: ${name}
  friendly_name: ${friendly_name}
  min_version: 2024.7.0
  name_add_mac_suffix: false
  platformio_options:
    board_build.flash_mode: dio
  libraries:
    - m5stack/M5GFX@^0.1.11
    - m5stack/M5Unified@^0.1.11
  on_boot:
    priority: 600
    then:
      - script.execute: draw_display
      - delay: 30s
      - if:
          condition:
            lambda: return id(init_in_progress);
          then:
            - lambda: id(init_in_progress) = false;
            - script.execute: draw_display

esp32:
  board: esp32-s3-devkitc-1
  flash_size: 16MB
  framework:
    type: esp-idf
    sdkconfig_options:
      CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240: "y"
      CONFIG_ESP32S3_DATA_CACHE_64KB: "y"
      CONFIG_ESP32S3_DATA_CACHE_LINE_64B: "y"
      CONFIG_AUDIO_BOARD_CUSTOM: "y"
      CONFIG_ESP32_S3_BOX_3_BOARD: "y"

psram:
  mode: quad
  speed: 80MHz

external_components:
  - source:
      type: git
      url: https://github.com/m5stack/M5CoreS3-Esphome
    components: [ board_m5cores3, m5cores3_audio, m5cores3_display ]
    refresh: 0s
  - source: github://pr#5230
    components: esp_adf
    refresh: 0s
  - source: github://jesserockz/esphome-components
    components: [file]
    refresh: 0s

api:
  encryption:
    # Replace the key by your API key
    key: "YOURKEYHERE"
  on_client_connected:
    - script.execute: draw_display
  on_client_disconnected:
    - script.execute: draw_display

ota:
  - platform: esphome
    password: "YOURPASSWORDHERE"
    id: common_ota

logger:
  hardware_uart: USB_SERIAL_JTAG

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  fast_connect: true # Required for hidden AP
  on_connect:
    - script.execute: draw_display
  on_disconnect:
    - script.execute: draw_display

# button:
#   - platform: factory_reset
#     id: factory_reset_btn
#     internal: true

# binary_sensor:
#   - platform: gpio
#     pin:
#       number: GPIO0
#       mode: INPUT_PULLUP
#       inverted: true
#     id: top_left_button
#     internal: true
#     on_multi_click:
#       - timing:
#           - ON for at least 50ms
#           - OFF for at least 50ms
#         then:
#           - switch.turn_off: timer_ringing
#       - timing:
#           - ON for at least 10s
#         then:
#           - button.press: factory_reset_btn

# output:
#   - platform: ledc
#     pin: GPIO47
#     id: backlight_output

# light:
#   - platform: monochromatic
#     id: led
#     name: Screen
#     icon: "mdi:television"
#     entity_category: config
#     output: backlight_output
#     restore_mode: RESTORE_DEFAULT_ON
#     default_transition_length: 250ms

esp_adf:

i2s_audio:
  - id: i2s_audio_bus
    i2s_lrclk_pin: GPIO33
    i2s_bclk_pin: GPIO34
    i2s_mclk_pin: GPIO0

microphone:
  - platform: i2s_audio
    id: box_mic
    i2s_din_pin: GPIO14
    adc_type: external
    pdm: false
    bits_per_sample: 16bit
    channel: right

micro_wake_word:
  models:
    - ${micro_wake_word_model}
  on_wake_word_detected:
    - voice_assistant.start:
        wake_word: !lambda return wake_word;

voice_assistant:
  id: va
  microphone: box_mic
  speaker: box_speaker
  # esp_adf only
  # noise_suppression_level: 2
  # auto_gain: 31dBFS
  # volume_multiplier: 2.0
  # vad_threshold: 3
  on_listening:
    - lambda: id(voice_assistant_phase) = ${voice_assist_listening_phase_id};
    - text_sensor.template.publish:
        id: text_request
        state: "..."
    - text_sensor.template.publish:
        id: text_response
        state: "..."
    - script.execute: draw_display
  on_stt_vad_end:
    - lambda: id(voice_assistant_phase) = ${voice_assist_thinking_phase_id};
    - script.execute: draw_display
  on_stt_end:
    - text_sensor.template.publish:
        id: text_request
        state: !lambda return x;
    - script.execute: draw_display
  on_tts_start:
    - text_sensor.template.publish:
        id: text_response
        state: !lambda return x;
  on_tts_stream_start:
    - lambda: id(voice_assistant_phase) = ${voice_assist_replying_phase_id};
    - script.execute: draw_display
  on_tts_stream_end:
    - if:
        condition:
          switch.is_off: mute
        then:
          - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
        else:
          - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
    - script.execute: draw_display
  on_end:
    - wait_until:
        not:
          voice_assistant.is_running:
    - if:
        condition:
          and:
            - switch.is_off: mute
            - lambda: return id(wake_word_engine_location).state == "On device";
            - lambda: return id(voice_assistant_phase) != ${voice_assist_timer_finished_phase_id};
        then:
          - micro_wake_word.start:
  on_error:
    - if:
        condition:
          lambda: return !id(init_in_progress);
        then:
          - lambda: id(voice_assistant_phase) = ${voice_assist_error_phase_id};
          - script.execute: draw_display
          - delay: 1s
          - if:
              condition:
                switch.is_off: mute
              then:
                - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
              else:
                - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
          - script.execute: draw_display
  on_client_connected:
    - lambda: id(init_in_progress) = false;
    - script.execute: start_voice_assistant
    - script.execute: draw_display
  on_client_disconnected:
    - script.execute: stop_voice_assistant
    - script.execute: draw_display
  on_timer_started:
    - script.execute: draw_display
  on_timer_cancelled:
    - script.execute: draw_display
  on_timer_updated:
    - script.execute: draw_display
  on_timer_tick:
    - script.execute: draw_display
  on_timer_finished:
    - script.execute: stop_voice_assistant
    - lambda: id(voice_assistant_phase) = ${voice_assist_timer_finished_phase_id};
    - switch.turn_on: timer_ringing
    - script.execute: draw_display
    - wait_until:
        not:
          microphone.is_capturing:
    - while:
        condition:
          switch.is_on: timer_ringing
        then:
          - lambda: id(box_speaker).play(id(timer_finished_wave_file), sizeof(id(timer_finished_wave_file)));
          - delay: 1s
    - wait_until:
        not:
          speaker.is_playing:
    - switch.turn_off: timer_ringing
    - script.execute: start_voice_assistant
    - script.execute: draw_display

script:
  - id: draw_display
    then:
      - if:
          condition:
            lambda: return !id(init_in_progress);
          then:
            - if:
                condition:
                  wifi.connected:
                then:
                  - if:
                      condition:
                        api.connected:
                      then:
                        - lambda: |
                            switch(id(voice_assistant_phase)) {
                              case ${voice_assist_listening_phase_id}:
                                id(s3_box_lcd).show_page(listening_page);
                                id(s3_box_lcd).update();
                                break;
                              case ${voice_assist_thinking_phase_id}:
                                id(s3_box_lcd).show_page(thinking_page);
                                id(s3_box_lcd).update();
                                break;
                              case ${voice_assist_replying_phase_id}:
                                id(s3_box_lcd).show_page(replying_page);
                                id(s3_box_lcd).update();
                                break;
                              case ${voice_assist_error_phase_id}:
                                id(s3_box_lcd).show_page(error_page);
                                id(s3_box_lcd).update();
                                break;
                              case ${voice_assist_muted_phase_id}:
                                id(s3_box_lcd).show_page(muted_page);
                                id(s3_box_lcd).update();
                                break;
                              case ${voice_assist_not_ready_phase_id}:
                                id(s3_box_lcd).show_page(no_ha_page);
                                id(s3_box_lcd).update();
                                break;
                              case ${voice_assist_timer_finished_phase_id}:
                                id(s3_box_lcd).show_page(timer_finished_page);
                                id(s3_box_lcd).update();
                                break;
                              default:
                                id(s3_box_lcd).show_page(idle_page);
                                id(s3_box_lcd).update();
                            }
                      else:
                        - display.page.show: no_ha_page
                        - component.update: s3_box_lcd
                else:
                  - display.page.show: no_wifi_page
                  - component.update: s3_box_lcd
          else:
            - display.page.show: initializing_page
            - component.update: s3_box_lcd

  - id: fetch_first_active_timer
    then:
      - lambda: |
          const auto timers = id(va).get_timers();
          auto output_timer = timers.begin()->second;
          for (auto &iterable_timer : timers) {
            if (iterable_timer.second.is_active && iterable_timer.second.seconds_left <= output_timer.seconds_left) {
              output_timer = iterable_timer.second;
            }
          }
          id(global_first_active_timer) = output_timer;
  - id: check_if_timers_active
    then:
      - lambda: |
          const auto timers = id(va).get_timers();
          bool output = false;
          if (timers.size() > 0) {
            for (auto &iterable_timer : timers) {
              if(iterable_timer.second.is_active) {
                output = true;
              }
            }
          }
          id(global_is_timer_active) = output;
  - id: fetch_first_timer
    then:
      - lambda: |
          const auto timers = id(va).get_timers();
          auto output_timer = timers.begin()->second;
          for (auto &iterable_timer : timers) {
            if (iterable_timer.second.seconds_left <= output_timer.seconds_left) {
              output_timer = iterable_timer.second;
            }
          }
          id(global_first_timer) = output_timer;
  - id: check_if_timers
    then:
      - lambda: |
          const auto timers = id(va).get_timers();
          bool output = false;
          if (timers.size() > 0) {
            output = true;
          }
          id(global_is_timer) = output;

  - id: draw_timer_timeline
    then:
      - lambda: |
          id(check_if_timers_active).execute();
          id(check_if_timers).execute();
          if (id(global_is_timer_active)){
            id(fetch_first_active_timer).execute();
            int active_pixels = round( 320 * id(global_first_active_timer).seconds_left / max(id(global_first_active_timer).total_seconds , static_cast<uint32_t>(1)) );
            if (active_pixels > 0){
              id(s3_box_lcd).filled_rectangle(0 , 225 , 320 , 15 , Color::WHITE );
              id(s3_box_lcd).filled_rectangle(0 , 226 , active_pixels , 13 , id(active_timer_color) );
            }
          } else if (id(global_is_timer)){
            id(fetch_first_timer).execute();
            int active_pixels = round( 320 * id(global_first_timer).seconds_left / max(id(global_first_timer).total_seconds , static_cast<uint32_t>(1)));
            if (active_pixels > 0){
              id(s3_box_lcd).filled_rectangle(0 , 225 , 320 , 15 , Color::WHITE );
              id(s3_box_lcd).filled_rectangle(0 , 226 , active_pixels , 13 , id(paused_timer_color) );
            }
          }
  - id: draw_active_timer_widget
    then:
      - lambda: |
          id(check_if_timers_active).execute();
          if (id(global_is_timer_active)){
            id(s3_box_lcd).filled_rectangle(80 , 40 , 160 , 50 , Color::WHITE );
            id(s3_box_lcd).rectangle(80 , 40 , 160 , 50 , Color::BLACK );

            id(fetch_first_active_timer).execute();
            int hours_left = floor(id(global_first_active_timer).seconds_left / 3600);
            int minutes_left = floor((id(global_first_active_timer).seconds_left - hours_left * 3600) / 60);
            int seconds_left = id(global_first_active_timer).seconds_left - hours_left * 3600 - minutes_left * 60 ;
            auto display_hours = (hours_left < 10 ? "0" : "") + std::to_string(hours_left);
            auto display_minute = (minutes_left < 10 ? "0" : "") + std::to_string(minutes_left);
            auto display_seconds = (seconds_left  < 10 ? "0" : "") + std::to_string(seconds_left) ;

            std::string display_string = "";
            if (hours_left > 0) {
              display_string = display_hours + ":" + display_minute;
            } else {
              display_string = display_minute + ":" + display_seconds;
            }
            id(s3_box_lcd).printf(120, 47, id(font_timer), Color::BLACK, "%s", display_string.c_str());
          }

  - id: start_voice_assistant
    then:
      - if:
          condition:
            switch.is_off: mute
          then:
            - if:
                condition:
                  lambda: return id(wake_word_engine_location).state == "In Home Assistant";
                then:
                  - lambda: id(va).set_use_wake_word(true);
                  - voice_assistant.start_continuous:
            - if:
                condition:
                  lambda: return id(wake_word_engine_location).state == "On device";
                then:
                  - lambda: id(va).set_use_wake_word(false);
                  - micro_wake_word.start
            - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
          else:
            - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};

  - id: stop_voice_assistant
    then:
      - if:
          condition:
            lambda: return id(wake_word_engine_location).state == "In Home Assistant";
          then:
            - lambda: id(va).set_use_wake_word(false);
            - voice_assistant.stop:
      - if:
          condition:
            lambda: return id(wake_word_engine_location).state == "On device";
          then:
            - voice_assistant.stop:
            - micro_wake_word.stop:
      - lambda: id(voice_assistant_phase) = ${voice_assist_not_ready_phase_id};

switch:
  - platform: template
    name: Mute
    id: mute
    icon: "mdi:microphone-off"
    optimistic: true
    restore_mode: RESTORE_DEFAULT_OFF
    entity_category: config
    on_turn_off:
      - if:
          condition:
            lambda: return !id(init_in_progress);
          then:
            - lambda: id(voice_assistant_phase) = ${voice_assist_idle_phase_id};
            - if:
                condition:
                  not:
                    - voice_assistant.is_running
                then:
                  - if:
                      condition:
                        lambda: return id(wake_word_engine_location).state == "In Home Assistant";
                      then:
                        - lambda: id(va).set_use_wake_word(true);
                        - voice_assistant.start_continuous
                  - if:
                      condition:
                        lambda: return id(wake_word_engine_location).state == "On device";
                      then:
                        - lambda: id(va).set_use_wake_word(false);
                        - micro_wake_word.start
            - script.execute: draw_display
    on_turn_on:
      - if:
          condition:
            lambda: return !id(init_in_progress);
          then:
            - lambda: id(va).set_use_wake_word(false);
            - voice_assistant.stop
            - micro_wake_word.stop
            - lambda: id(voice_assistant_phase) = ${voice_assist_muted_phase_id};
            - script.execute: draw_display

  - platform: template
    id: timer_ringing
    optimistic: true
    internal: true
    restore_mode: ALWAYS_OFF
    on_turn_on:
      - delay: 15min
      - switch.turn_off: timer_ringing

select:
  - platform: template
    entity_category: config
    name: Wake word engine location
    id: wake_word_engine_location
    icon: "mdi:account-voice"
    optimistic: true
    restore_value: true
    options:
      - In Home Assistant
      - On device
    initial_option: On device
    on_value:
      - if:
          condition:
            lambda: return !id(init_in_progress);
          then:
            - wait_until:
                lambda: return id(voice_assistant_phase) == ${voice_assist_muted_phase_id} || id(voice_assistant_phase) == ${voice_assist_idle_phase_id};
            - if:
                condition:
                  lambda: return x == "In Home Assistant";
                then:
                  - micro_wake_word.stop
                  - delay: 500ms
                  - if:
                      condition:
                        switch.is_off: mute
                      then:
                        - lambda: id(va).set_use_wake_word(true);
                        - voice_assistant.start_continuous:
            - if:
                condition:
                  lambda: return x == "On device";
                then:
                  - lambda: id(va).set_use_wake_word(false);
                  - voice_assistant.stop
                  - delay: 500ms
                  - if:
                      condition:
                        switch.is_off: mute
                      then:
                        - micro_wake_word.start

globals:
  - id: init_in_progress
    type: bool
    restore_value: false
    initial_value: "true"
  - id: voice_assistant_phase
    type: int
    restore_value: false
    initial_value: ${voice_assist_not_ready_phase_id}
  - id: global_first_active_timer
    type: voice_assistant::Timer
    restore_value: false
  - id: global_is_timer_active
    type: bool
    restore_value: false
  - id: global_first_timer
    type: voice_assistant::Timer
    restore_value: false
  - id: global_is_timer
    type: bool
    restore_value: false

image:
  - file: ${error_illustration_file}
    id: casita_error
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: ${idle_illustration_file}
    id: casita_idle
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: ${listening_illustration_file}
    id: casita_listening
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: ${thinking_illustration_file}
    id: casita_thinking
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: ${replying_illustration_file}
    id: casita_replying
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: ${timer_finished_illustration_file}
    id: casita_timer_finished
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: ${loading_illustration_file}
    id: casita_initializing
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: https://github.com/esphome/wake-word-voice-assistants/raw/main/error_box_illustrations/error-no-wifi.png
    id: error_no_wifi
    resize: 320x240
    type: RGB24
    use_transparency: true
  - file: https://github.com/esphome/wake-word-voice-assistants/raw/main/error_box_illustrations/error-no-ha.png
    id: error_no_ha
    resize: 320x240
    type: RGB24
    use_transparency: true

font:
  - file:
      type: gfonts
      family: Figtree
      weight: 300
      italic: true
    glyphs: ${allowed_characters}
    id: font_request
    size: 15
  - file:
      type: gfonts
      family: Figtree
      weight: 300
    glyphs: ${allowed_characters}
    id: font_response
    size: 15
  - file:
      type: gfonts
      family: Figtree
      weight: 300
    glyphs: ${allowed_characters}
    id: font_timer
    size: 30

text_sensor:
  - id: text_request
    platform: template
    on_value:
      lambda: |-
        if(id(text_request).state.length()>32) {
          std::string name = id(text_request).state.c_str();
          std::string truncated = esphome::str_truncate(name.c_str(),31);
          id(text_request).state = (truncated+"...").c_str();
        }

  - id: text_response
    platform: template
    on_value:
      lambda: |-
        if(id(text_response).state.length()>32) {
          std::string name = id(text_response).state.c_str();
          std::string truncated = esphome::str_truncate(name.c_str(),31);
          id(text_response).state = (truncated+"...").c_str();
        }

color:
  - id: idle_color
    hex: ${idle_illustration_background_color}
  - id: listening_color
    hex: ${listening_illustration_background_color}
  - id: thinking_color
    hex: ${thinking_illustration_background_color}
  - id: replying_color
    hex: ${replying_illustration_background_color}
  - id: loading_color
    hex: ${loading_illustration_background_color}
  - id: error_color
    hex: ${error_illustration_background_color}
  - id: active_timer_color
    hex: "26ed3a"
  - id: paused_timer_color
    hex: "3b89e3"

file:
  - id: timer_finished_wave_file
    file: https://github.com/esphome/wake-word-voice-assistants/raw/main/sounds/timer_finished.wav

display:
  - platform: m5cores3_display
    model: ILI9342
    dc_pin: 35
    update_interval: never
    id: s3_box_lcd
    pages:
      - id: idle_page
        lambda: |-
          it.fill(id(idle_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_idle), ImageAlign::CENTER);
          id(draw_timer_timeline).execute();
          id(draw_active_timer_widget).execute();
      - id: listening_page
        lambda: |-
          it.fill(id(listening_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_listening), ImageAlign::CENTER);
          id(draw_timer_timeline).execute();
      - id: thinking_page
        lambda: |-
          it.fill(id(thinking_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_thinking), ImageAlign::CENTER);
          it.filled_rectangle(20 , 20 , 280 , 30 , Color::WHITE );
          it.rectangle(20 , 20 , 280 , 30 , Color::BLACK );
          it.printf(30, 25, id(font_request), Color::BLACK, "%s", id(text_request).state.c_str());

          id(draw_timer_timeline).execute();
      - id: replying_page
        lambda: |-
          it.fill(id(replying_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_replying), ImageAlign::CENTER);
          it.filled_rectangle(20 , 20 , 280 , 30 , Color::WHITE );
          it.rectangle(20 , 20 , 280 , 30 , Color::BLACK );
          it.filled_rectangle(20 , 190 , 280 , 30 , Color::WHITE );
          it.rectangle(20 , 190 , 280 , 30 , Color::BLACK );
          it.printf(30, 25, id(font_request), Color::BLACK, "%s", id(text_request).state.c_str());
          it.printf(30, 195, id(font_response), Color::BLACK, "%s", id(text_response).state.c_str());
          id(draw_timer_timeline).execute();
      - id: timer_finished_page
        lambda: |-
          it.fill(id(idle_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_timer_finished), ImageAlign::CENTER);
      - id: error_page
        lambda: |-
          it.fill(id(error_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_error), ImageAlign::CENTER);
      - id: no_ha_page
        lambda: |-
          it.image((it.get_width() / 2), (it.get_height() / 2), id(error_no_ha), ImageAlign::CENTER);
      - id: no_wifi_page
        lambda: |-
          it.image((it.get_width() / 2), (it.get_height() / 2), id(error_no_wifi), ImageAlign::CENTER);
      - id: initializing_page
        lambda: |-
          it.fill(id(loading_color));
          it.image((it.get_width() / 2), (it.get_height() / 2), id(casita_initializing), ImageAlign::CENTER);
      - id: muted_page
        lambda: |-
          it.fill(Color::BLACK);
          id(draw_timer_timeline).execute();
          id(draw_active_timer_widget).execute();

[encodedLife]

This YAML should work. However, the quality is not sufficient, and I suspect there might be an issue with the microphone component. Additionally, it wakes up incorrectly far too often.

[GinAndBacon]

I kept getting the wake word location grayed out, after flashing it went to on device then rebooted. It seemed like something was timing out but I could be wrong. Changed the on_boot to below and works, speaker doesn't seem as good but it's not that great anyways. It's also been a few days since I changed it to use on device wake work using the above with the below changed.

  on_boot:
    priority: 600
    then:
      - delay: 30s            
      - if:
          condition:
            - lambda: return id(init_in_progress);
          then:
            - lambda: id(init_in_progress) = false;      
      - script.execute: draw_display  # Initialize the display
      - delay: 1s  # Wait a moment to ensure the display is ready
      - switch.turn_on: switch_lcd_backlight  # Turn on the backlight switch to set the initial state