Skip to content

Commit f57d6c1

Browse files
cinimlclaude
andcommitted
ble-audio: 再生時の口パクを適応 RMS/dB エンベロープに改善
ピーク振幅の線形スケールだと音楽 (常時高レベル) で口が開きっぱなしに なる問題。3 段で改善: 1. ピーク → RMS エネルギー。瞬間最大ではなくチャンク平均で安定。 2. 線形 → dBFS 対数 + 適応ベースライン。クリップの平均ラウドネスを ~1 秒時定数で slow-track し、「平均からの相対差」で口を駆動 (baseline-2dB→閉, baseline+8dB→開, 定常部≈0.2)。ソースの絶対 レベルに依存せず自己校正するので、圧縮された音楽でも抑揚で動く。 3. 非対称平滑化 (attack 0.95 / release 0.65) で ~21ms チャンクの オンセットにキビキビ追従しつつチャンク境界のチラつきを抑制。 エンベロープ状態はクリップ開始ごとにリセット。 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 49a078a commit f57d6c1

1 file changed

Lines changed: 56 additions & 6 deletions

File tree

main/audio_stream_sink.cpp

Lines changed: 56 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -159,17 +159,65 @@ struct Ring {
159159
}
160160
};
161161

162-
// Drive mouth_open from the chunk peak.
162+
// Mouth-open state. Reset at the start of each playback.
163+
float g_mouth_smoothed = 0.0f;
164+
float g_db_baseline = -30.0f; // slow-tracked average loudness of the clip
165+
bool g_db_baseline_init = false;
166+
167+
// Map chunk loudness to mouth open. A fixed dBFS window still pinned the
168+
// mouth open for music, because mastered music sits at a near-constant
169+
// loud RMS (~-20..-14 dBFS) — there's always *some* sound, so any absolute
170+
// threshold is permanently exceeded. Instead track the clip's running
171+
// average loudness and drive the mouth from how far each chunk sits
172+
// *relative to that baseline*: steady passages → mostly closed, louder
173+
// beats / sung notes → open, quiet gaps → shut. This self-calibrates to
174+
// whatever level the source happens to be at.
163175
void update_mouth(const std::int16_t* samples, std::size_t n)
164176
{
165177
if (g_state == nullptr || n == 0) return;
166-
std::int32_t peak = 0;
178+
179+
double sum_sq = 0.0;
167180
for (std::size_t i = 0; i < n; ++i) {
168-
const std::int32_t v = std::abs(static_cast<std::int32_t>(samples[i]));
169-
if (v > peak) peak = v;
181+
const double s = samples[i];
182+
sum_sq += s * s;
183+
}
184+
const double rms = std::sqrt(sum_sq / static_cast<double>(n)); // 0 .. 32768
185+
const float db = (rms >= 1.0)
186+
? 20.0f * std::log10(static_cast<float>(rms) / 32768.0f)
187+
: -96.0f;
188+
189+
// Track the average loudness, but only over chunks that actually carry
190+
// signal (> -55 dB) so silence doesn't drag the baseline down. Seed it
191+
// from the first real chunk for fast convergence.
192+
constexpr float kSilenceDb = -55.0f;
193+
if (db > kSilenceDb) {
194+
if (!g_db_baseline_init) {
195+
g_db_baseline = db;
196+
g_db_baseline_init = true;
197+
} else {
198+
g_db_baseline += 0.03f * (db - g_db_baseline); // ~1 s time constant
199+
}
170200
}
171-
const float open = std::min(1.0f, static_cast<float>(peak) / 32767.0f * 1.5f);
172-
g_state->mouth_open.store(open, std::memory_order_relaxed);
201+
202+
// Relative window centred on the baseline: baseline-kLowDb maps to
203+
// closed, baseline+kHighDb to fully open. Biased so steady material
204+
// (db ≈ baseline) sits low (~0.2) and only louder-than-average moments
205+
// open the mouth wide.
206+
constexpr float kLowDb = 2.0f; // this far below baseline → closed
207+
constexpr float kHighDb = 8.0f; // this far above baseline → open
208+
// (baseline itself maps to kLowDb/(kLowDb+kHighDb) = 0.2 → mostly shut)
209+
float target = (db - (g_db_baseline - kLowDb)) / (kLowDb + kHighDb);
210+
target = std::clamp(target, 0.0f, 1.0f);
211+
212+
// Asymmetric smoothing: open almost instantly (attack), close a touch
213+
// slower (release). Near-1.0 attack means a loud onset is on-screen
214+
// within a chunk (~21 ms); the release still rounds off the tail so it
215+
// doesn't strobe between chunks.
216+
constexpr float kAttack = 0.95f;
217+
constexpr float kRelease = 0.65f;
218+
const float a = (target > g_mouth_smoothed) ? kAttack : kRelease;
219+
g_mouth_smoothed += a * (target - g_mouth_smoothed);
220+
g_state->mouth_open.store(g_mouth_smoothed, std::memory_order_relaxed);
173221
}
174222

175223
// --- Worker ----------------------------------------------------------
@@ -378,6 +426,8 @@ void worker_task(void* /*arg*/)
378426
M5.Speaker.begin();
379427
i2s_acquired = true;
380428
playback_started = true;
429+
g_mouth_smoothed = 0.0f; // fresh envelope per clip
430+
g_db_baseline_init = false; // re-learn loudness baseline
381431
ESP_LOGI(kTag, "playback started: %u samples pre-rolled @ %u Hz",
382432
static_cast<unsigned>(ring.available_read()),
383433
static_cast<unsigned>(pcm_sample_rate));

0 commit comments

Comments
 (0)