Commit c700890
Add ct-voice-input component with real-time audio visualization (commontoolsinc#2138)
* Add ct-voice-input component design document
This design document specifies a voice recording and transcription
component for CommonTools patterns. The component enables:
- Voice recording with MediaRecorder API
- Real-time waveform visualization
- Automatic transcription via API integration
- Reactive cell binding for pattern integration
- Two-component architecture (ct-voice-input + ct-audio-visualizer)
The design includes a phased implementation roadmap starting with v1 MVP
focusing on core recording and transcription functionality, with future
versions adding visual polish and advanced features.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Happy <[email protected]>
* Add ct-voice-input and ct-audio-visualizer components
Implements voice recording and transcription components for CommonTools
patterns, following the design in docs/specs/ct-voice-input-design.md.
**ct-voice-input** - Main voice input component featuring:
- MediaRecorder integration for audio capture
- Microphone permission handling
- Cell binding via CellController for reactive integration
- Recording modes (hold-to-record and toggle)
- Duration tracking with max duration limits
- Error handling for permission and device issues
- Transcription API integration (placeholder)
- Event emission for all recording lifecycle events
**ct-audio-visualizer** - Waveform visualization component:
- Real-time audio frequency visualization using Web Audio API
- SVG-based rendering for flexibility and styling
- Configurable bar count, color, and height
- Automatic resource cleanup on disconnect
Both components follow Common UI v2 patterns:
- Extend BaseElement
- Use theme context for styling
- Include box-sizing resets
- Export types separately
- Proper JSDoc documentation
This is the v1 MVP implementation focusing on core functionality.
Future enhancements will include:
- Discord-style expansion animations
- Audio format conversion (WebM → WAV)
- Real transcription API integration
- Playback functionality
- Message bubble states
packages/ui/src/v2/components/ct-voice-input/ct-voice-input.ts:614
packages/ui/src/v2/components/ct-audio-visualizer/ct-audio-visualizer.ts:169
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Happy <[email protected]>
* Add voice-note test pattern and audio conversion utilities
**voice-note.tsx pattern** - Demo pattern showcasing voice input:
- Hold-to-record voice transcription
- Saves transcribed notes to a list
- Shows latest transcription with duration
- Delete functionality for saved notes
- Clean UI with ct-card and ct-vstack layout
**audio-conversion.ts** - Complete WebM to WAV conversion:
- Uses Web Audio API for decoding
- Resamples to target sample rate (16kHz for transcription)
- Converts stereo to mono
- Encodes as 16-bit PCM WAV format
- Proper WAV header generation
- Linear interpolation for resampling
**ct-voice-input updates**:
- Integrated real audio conversion utility
- Graceful fallback if conversion fails
- Optimized for FAL AI Wizper transcription API
The transcription API endpoint already exists at
/api/ai/voice/transcribe and is fully functional with caching,
FAL AI Wizper integration, and support for timestamped chunks.
Component is now fully functional end-to-end!
packages/patterns/voice-note.tsx:166
packages/ui/src/v2/utils/audio-conversion.ts:203
packages/ui/src/v2/components/ct-voice-input/ct-voice-input.ts:618
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Happy <[email protected]>
* Add JSX type definitions for voice input components
- Add CTVoiceInputElement and CTAudioVisualizerElement interfaces
- Add CTVoiceInputAttributes with all properties and event handlers
- Add CTAudioVisualizerAttributes for the visualizer subcomponent
- Add IntrinsicElements entries for both components
- Create voice-note-simple.tsx demo pattern
- Update voice-note.tsx with handler fixes
This enables TypeScript compilation of patterns using the new
ct-voice-input and ct-audio-visualizer components in JSX.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Happy <[email protected]>
* Fix ct-audio-visualizer animation during recording
The audio visualizer wasn't animating during voice recording, providing
no visual feedback to users. This fixes the issue by:
1. Using Lit's ref directive instead of querySelector for reliable element access
2. Waiting for updateComplete to ensure the element is rendered before accessing it
3. Skipping the first 2 frequency bins to avoid low-frequency noise dominating the visualization
This results in a smooth, balanced waveform animation that provides clear
real-time feedback during recording.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Happy <[email protected]>
* Move ct-voice-input spec to component directory
Following the pattern used by ct-outliner, move the design spec into
the component directory itself rather than docs/specs/.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Happy <[email protected]>
* Address PR review feedback
- Reuse AudioContext per element instead of creating new ones (jsantell)
- Remove unused renderMode property from ct-audio-visualizer
- Fix Content-Type header to use actual blob type when WAV conversion fails
- Update spec.md to reflect actual implementation (duration parameter)
- Fix delete button in voice-note pattern by passing noteId through context
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Happy <[email protected]>
* Add comprehensive tests for audio conversion utilities
Tests cover:
- WAV file format structure validation
- Float to 16-bit PCM conversion with clamping
- Audio resampling with linear interpolation
- Stereo to mono channel mixing
- DataView string writing for WAV headers
All tests pass successfully.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Happy <[email protected]>
* Improve audio conversion tests to use actual exported functions
Address cubic-dev-ai feedback:
- Export helper functions (resample, floatTo16BitPCM, createWavFile) for testing
- Update tests to call actual functions instead of reimplementing logic
- Fix "sample rates match" test to actually call resample() function
- All tests now provide regression coverage for the real utilities
All 12 test steps still pass.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Happy <[email protected]>
* Lint and format
* Remove `spec.md`
* Fix compile errors
* Remove `MouseEvent` type
---------
Co-authored-by: Claude <[email protected]>
Co-authored-by: Happy <[email protected]>
Co-authored-by: Ben Follington <[email protected]>1 parent 963575c commit c700890
File tree
10 files changed
+1581
-0
lines changed- packages
- html/src
- patterns
- ui/src/v2
- components
- ct-audio-visualizer
- ct-voice-input
- utils
10 files changed
+1581
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2906 | 2906 | | |
2907 | 2907 | | |
2908 | 2908 | | |
| 2909 | + | |
| 2910 | + | |
2909 | 2911 | | |
2910 | 2912 | | |
2911 | 2913 | | |
| |||
2941 | 2943 | | |
2942 | 2944 | | |
2943 | 2945 | | |
| 2946 | + | |
| 2947 | + | |
| 2948 | + | |
| 2949 | + | |
| 2950 | + | |
| 2951 | + | |
| 2952 | + | |
| 2953 | + | |
| 2954 | + | |
| 2955 | + | |
| 2956 | + | |
| 2957 | + | |
| 2958 | + | |
| 2959 | + | |
| 2960 | + | |
| 2961 | + | |
| 2962 | + | |
| 2963 | + | |
| 2964 | + | |
| 2965 | + | |
| 2966 | + | |
| 2967 | + | |
| 2968 | + | |
| 2969 | + | |
| 2970 | + | |
| 2971 | + | |
| 2972 | + | |
| 2973 | + | |
| 2974 | + | |
| 2975 | + | |
| 2976 | + | |
| 2977 | + | |
2944 | 2978 | | |
2945 | 2979 | | |
2946 | 2980 | | |
| |||
3889 | 3923 | | |
3890 | 3924 | | |
3891 | 3925 | | |
| 3926 | + | |
| 3927 | + | |
| 3928 | + | |
| 3929 | + | |
| 3930 | + | |
| 3931 | + | |
| 3932 | + | |
| 3933 | + | |
3892 | 3934 | | |
3893 | 3935 | | |
3894 | 3936 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
0 commit comments