Skip to content

Commit 4a45ccf

Browse files
authored
feat: add Web Audio routing to MediaElementTrack for fades (#318)
* feat: add Web Audio routing to MediaElementTrack for fades Extract fade utilities from playout to core package so they can be shared with media-element-playout. Add optional AudioContext support to MediaElementTrack that routes audio through a Web Audio graph (source → fadeGain → volumeGain → destination) enabling fade in/out effects without Tone.js dependency. Rewrite FadesExample and StylingExample to use MediaElementPlaylistProvider with pre-computed peaks, giving each player instance independent playback (no shared Tone.js Transport singleton). * feat: add showFades prop to MediaElementWaveform Expose fadeIn/fadeOut config via MediaElementDataContext and render FadeOverlay components in MediaElementPlaylist when showFades is enabled. Update FadesExample with show fades checkbox and lower samplesPerPixel for visible fade curve shapes. Make StylingExample theme descriptions explicit about color names and add boldTheme with transparent progress for Extra Wide Bars. * feat: add Tone.js effect bridging to MediaElement example Add a third player to the media-element example demonstrating BitCrusher effect via native→Tone.js bridge pattern. Tone.js is dynamically imported after AudioContext is running to avoid AudioWorklet errors on page load. - Add outputNode getter to MediaElementPlayout for effects wiring - Add EffectWiring component with statechange-based lazy initialization - Update guide, API docs, and llms.txt for audioContext prop, fades, and Tone.js effect bridging * docs: document WaveformPlaylistProvider single-instance constraint Add caution admonition to WaveformPlaylistProvider API doc explaining the shared Tone.js Transport singleton. Add Instances row to MediaElementPlaylistProvider comparison table. Update llms.txt and CLAUDE.md files with Tone.js dynamic import and effect bridging patterns. * fix: address PR review issues for MediaElement fades and effects - Fix partial fade resume to use actual curve interpolation via generateCurve() - Add error handling to wireEffect() with fallback reconnection - Add console.warn diagnostics to all empty catch blocks - Wrap createMediaElementSource() in try-catch with descriptive error - Handle AudioContext.resume() promise rejection - Separate disconnect/reconnect in cleanup with individual try-catch - Fix stale JSDoc (PingPongDelay → BitCrusher) - Deduplicate FadeConfig into @waveform-playlist/core as type alias - Lazy AudioContext creation in FadesExample via useRef - Use String(err) instead of passing error objects to console.warn - Wrap disconnect() in try-catch for connectOutput/disconnectOutput - Add reconnection to guide doc cleanup example * fix: partial fade curve shape, resume race, and guide cleanup - Partial fades now slice the original curve instead of generating a fresh one, preserving the correct curve shape for sCurve/logarithmic types when seeking mid-fade - Await AudioContext.resume() before scheduling fades to prevent fades from being scheduled at currentTime=0 (in the past) - Guide doc now removes statechange listener on cleanup, matching the actual implementation in MediaElementExample - Document why MediaElementExample uses eager AudioContext (single context, always needed) * fix: share single AudioContext across all fade players Multiple AudioContexts per page is wasteful — createMediaElementSource is per-element, not per-context, so all players can share one.
1 parent d947a92 commit 4a45ccf

File tree

24 files changed

+1335
-398
lines changed

24 files changed

+1335
-398
lines changed

CLAUDE.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -350,6 +350,9 @@ const LazyExample = createLazyExample(() =>
350350
27. **Stop Before Clear** — Always call `stop()` before clearing tracks. Clearing React state without stopping Tone.js Transport leaves orphaned audio playing. Use `ClearAllButton` (from `@waveform-playlist/browser`) which handles this automatically via `usePlaylistControls().stop()`.
351351
28. **Tone.js Panner Stereo Preservation**`new Panner(pan)` defaults to `channelCount: 1`, downmixing stereo to mono at 1/√2 gain. Use `new Panner({ pan, channelCount: 2 })` for audio tracks with stereo content. MIDI/SoundFont tracks use mono synths so `channelCount: 1` is fine.
352352
29. **Never Use Tone.js `addAudioWorkletModule`** — Tone.js caches a single `_workletPromise` per context. Only the first URL is loaded; subsequent calls with different URLs are silently skipped. Always use `rawContext.audioWorklet.addModule(url)` directly. `context.createAudioWorkletNode()` is still fine for node creation.
353+
30. **Dynamic Import Tone.js Outside Playout Package**`import * as Tone from 'tone'` eagerly creates a default context with AudioWorklet nodes, which fails before user gesture. Outside the playout package (which handles init via `getGlobalContext()`), use `import type` for types and `await import('tone')` inside effects after AudioContext is running. Call `Tone.setContext(new Tone.Context(audioContext))` to share the native AudioContext.
354+
31. **Tone.js Effect.input Is Not a Native AudioNode**`Effect` subclasses (BitCrusher, etc.) set `this.input = new Tone.Gain(...)` (a Tone.js wrapper). Native `AudioNode.connect(effect.input)` fails with "Overload resolution failed". Use `Tone.Gain` as a bridge: `outputNode.connect(bridge.input)` works because `Tone.Gain.input` IS a native `GainNode`. Then `bridge.chain(effect, destination)` for the Tone chain.
355+
32. **Provider Child Effects and playoutRef Timing** — React runs child effects before parent effects. A child component's `useEffect` accessing `playoutRef.current` will find `null` on first run because the provider's effect hasn't created the playout yet. Add `duration` (from `useMediaElementData()`) as a dependency — it changes from 0 to the actual value when the playout is ready, retriggering the effect.
353356

354357
---
355358

packages/browser/src/MediaElementPlaylistContext.tsx

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ import React, {
99
type ReactNode,
1010
} from 'react';
1111
import { ThemeProvider } from 'styled-components';
12-
import { MediaElementPlayout } from '@waveform-playlist/media-element-playout';
12+
import { MediaElementPlayout, type FadeConfig } from '@waveform-playlist/media-element-playout';
1313
import { type WaveformDataObject } from '@waveform-playlist/core';
1414
import { type WaveformPlaylistTheme, defaultTheme } from '@waveform-playlist/ui-components';
1515
import type { AnnotationData } from '@waveform-playlist/core';
@@ -27,6 +27,10 @@ export interface MediaElementTrackConfig {
2727
waveformData: WaveformDataObject;
2828
/** Track name for display */
2929
name?: string;
30+
/** Fade in configuration (requires audioContext on provider) */
31+
fadeIn?: FadeConfig;
32+
/** Fade out configuration (requires audioContext on provider) */
33+
fadeOut?: FadeConfig;
3034
}
3135

3236
// Context values for animation (high-frequency updates)
@@ -73,6 +77,8 @@ export interface MediaElementDataContextValue {
7377
barWidth: number;
7478
barGap: number;
7579
progressBarWidth: number;
80+
fadeIn?: FadeConfig;
81+
fadeOut?: FadeConfig;
7682
}
7783

7884
// Create contexts
@@ -111,6 +117,16 @@ export interface MediaElementPlaylistProviderProps {
111117
progressBarWidth?: number;
112118
/** Callback when annotations are changed (drag, edit, etc.) */
113119
onAnnotationsChange?: (annotations: AnnotationData[]) => void;
120+
/**
121+
* AudioContext for Web Audio routing (fades, effects).
122+
* When provided, audio routes through Web Audio nodes:
123+
* HTMLAudioElement → MediaElementSourceNode → fadeGain → volumeGain → destination
124+
*
125+
* Without this, playback uses HTMLAudioElement directly (no fades or effects).
126+
* Each provider instance should use its own AudioContext or share one —
127+
* createMediaElementSource() is called once per audio element.
128+
*/
129+
audioContext?: AudioContext;
114130
/** Callback when audio is ready */
115131
onReady?: () => void;
116132
children: ReactNode;
@@ -145,6 +161,7 @@ export const MediaElementPlaylistProvider: React.FC<MediaElementPlaylistProvider
145161
barWidth = 1,
146162
barGap = 0,
147163
progressBarWidth: progressBarWidthProp,
164+
audioContext,
148165
onAnnotationsChange,
149166
onReady,
150167
children,
@@ -235,6 +252,9 @@ export const MediaElementPlaylistProvider: React.FC<MediaElementPlaylistProvider
235252
source: track.source,
236253
peaks: track.waveformData,
237254
name: track.name,
255+
audioContext,
256+
fadeIn: track.fadeIn,
257+
fadeOut: track.fadeOut,
238258
});
239259

240260
// Set up time update callback
@@ -266,6 +286,9 @@ export const MediaElementPlaylistProvider: React.FC<MediaElementPlaylistProvider
266286
track.source,
267287
track.waveformData,
268288
track.name,
289+
track.fadeIn,
290+
track.fadeOut,
291+
audioContext,
269292
initialPlaybackRate,
270293
onReady,
271294
stopAnimationFrameLoop,
@@ -497,6 +520,8 @@ export const MediaElementPlaylistProvider: React.FC<MediaElementPlaylistProvider
497520
barWidth,
498521
barGap,
499522
progressBarWidth,
523+
fadeIn: track.fadeIn,
524+
fadeOut: track.fadeOut,
500525
}),
501526
[
502527
duration,
@@ -509,6 +534,8 @@ export const MediaElementPlaylistProvider: React.FC<MediaElementPlaylistProvider
509534
barWidth,
510535
barGap,
511536
progressBarWidth,
537+
track.fadeIn,
538+
track.fadeOut,
512539
]
513540
);
514541

packages/browser/src/components/MediaElementPlaylist.tsx

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import {
66
Track as TrackComponent,
77
Clip,
88
Selection,
9+
FadeOverlay,
910
PlaylistInfoContext,
1011
DevicePixelRatioProvider,
1112
SmartScale,
@@ -75,6 +76,8 @@ export interface MediaElementPlaylistProps {
7576
onAnnotationUpdate?: OnAnnotationUpdateFn;
7677
/** Custom playhead render function. Receives position, color, and animation refs for smooth 60fps animation. */
7778
renderPlayhead?: RenderPlayheadFunction;
79+
/** Show fade in/out overlays on the waveform. Defaults to false. */
80+
showFades?: boolean;
7881
className?: string;
7982
}
8083

@@ -96,6 +99,7 @@ export const MediaElementPlaylist: React.FC<MediaElementPlaylistProps> = ({
9699
linkEndpoints: linkEndpointsProp = false,
97100
onAnnotationUpdate,
98101
renderPlayhead,
102+
showFades = false,
99103
className,
100104
}) => {
101105
const theme = useTheme() as import('@waveform-playlist/ui-components').WaveformPlaylistTheme;
@@ -117,6 +121,8 @@ export const MediaElementPlaylist: React.FC<MediaElementPlaylistProps> = ({
117121
playoutRef,
118122
barWidth,
119123
barGap,
124+
fadeIn,
125+
fadeOut,
120126
} = useMediaElementData();
121127

122128
const [selectionStart, setSelectionStart] = useState(0);
@@ -327,6 +333,24 @@ export const MediaElementPlaylist: React.FC<MediaElementPlaylistProps> = ({
327333
clipDurationSamples={clip.durationSamples}
328334
/>
329335
))}
336+
{showFades && fadeIn && fadeIn.duration > 0 && (
337+
<FadeOverlay
338+
left={0}
339+
width={Math.floor((fadeIn.duration * sampleRate) / samplesPerPixel)}
340+
type="fadeIn"
341+
curveType={fadeIn.type}
342+
/>
343+
)}
344+
{showFades && fadeOut && fadeOut.duration > 0 && (
345+
<FadeOverlay
346+
left={
347+
width - Math.floor((fadeOut.duration * sampleRate) / samplesPerPixel)
348+
}
349+
width={Math.floor((fadeOut.duration * sampleRate) / samplesPerPixel)}
350+
type="fadeOut"
351+
curveType={fadeOut.type}
352+
/>
353+
)}
330354
</Clip>
331355
);
332356
})}

packages/browser/src/components/MediaElementWaveform.tsx

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,8 @@ export interface MediaElementWaveformProps {
3535
scrollActiveContainer?: 'nearest' | 'all';
3636
/** Custom playhead render function. Receives position, color, and animation refs for smooth 60fps animation. */
3737
renderPlayhead?: RenderPlayheadFunction;
38+
/** Show fade in/out overlays on the waveform. Defaults to false. */
39+
showFades?: boolean;
3840
className?: string;
3941
}
4042

@@ -60,6 +62,7 @@ export const MediaElementWaveform: React.FC<MediaElementWaveformProps> = ({
6062
scrollActivePosition = 'center',
6163
scrollActiveContainer = 'nearest',
6264
renderPlayhead,
65+
showFades = false,
6366
className,
6467
}) => {
6568
const { annotations } = useMediaElementState();
@@ -72,6 +75,7 @@ export const MediaElementWaveform: React.FC<MediaElementWaveformProps> = ({
7275
linkEndpoints={linkEndpoints}
7376
onAnnotationUpdate={onAnnotationUpdate}
7477
renderPlayhead={renderPlayhead}
78+
showFades={showFades}
7579
className={className}
7680
/>
7781
{annotations.length > 0 && (

0 commit comments

Comments
 (0)