|
1 | | -# Sherpa-ONNX Combined WebAssembly Module |
2 | | - |
3 | | -This directory contains a combined WebAssembly module for the Sherpa-ONNX project, which integrates multiple features: |
| 1 | +# Sherpa-ONNX WASM Combined Module |
4 | 2 |
|
| 3 | +This directory contains the WebAssembly (WASM) combined module for Sherpa-ONNX, which includes support for: |
5 | 4 | - Automatic Speech Recognition (ASR) |
6 | 5 | - Voice Activity Detection (VAD) |
7 | | -- Text-to-Speech Synthesis (TTS) |
8 | | -- Speech Enhancement |
| 6 | +- Text-to-Speech (TTS) |
| 7 | +- Keyword Spotting (KWS) |
9 | 8 | - Speaker Diarization |
10 | | -- Keyword Spotting |
11 | | - |
12 | | -## How to Use |
13 | | - |
14 | | -### Loading the Module |
15 | | - |
16 | | -You can use the combined module in two ways: |
17 | | - |
18 | | -#### Option 1: Load Individual Modules (Recommended) |
19 | | - |
20 | | -This approach loads only the components you need: |
21 | | - |
22 | | -```html |
23 | | -<!-- First load the WASM module --> |
24 | | -<script src="sherpa-onnx-wasm-combined.js"></script> |
25 | | - |
26 | | -<!-- Load the core module which is required by all other modules --> |
27 | | -<script src="sherpa-onnx-core.js"></script> |
28 | | - |
29 | | -<!-- Then load only the modules you need --> |
30 | | -<script src="sherpa-onnx-vad.js"></script> |
31 | | -<!-- Add other modules as needed --> |
32 | | - |
33 | | -<script> |
34 | | - // This callback is called when the WASM module is loaded |
35 | | - window.onModuleReady = function() { |
36 | | - // Your initialization code here |
37 | | - console.log("Module ready!"); |
38 | | - }; |
39 | | -</script> |
40 | | -``` |
41 | | - |
42 | | -#### Option 2: Load All Modules via the Combined Loader |
43 | | - |
44 | | -This approach loads all available modules: |
45 | | - |
46 | | -```html |
47 | | -<!-- First load the WASM module --> |
48 | | -<script src="sherpa-onnx-wasm-combined.js"></script> |
49 | | - |
50 | | -<!-- Then load the combined module that will load all other modules --> |
51 | | -<script src="sherpa-onnx-combined.js"></script> |
52 | | - |
53 | | -<script> |
54 | | - // This callback is called when all modules are loaded |
55 | | - window.onSherpaOnnxReady = function() { |
56 | | - // Your initialization code here |
57 | | - console.log("All modules loaded!"); |
58 | | - }; |
59 | | -</script> |
60 | | -``` |
61 | | - |
62 | | -### Module Structure |
63 | | - |
64 | | -The codebase has been organized into modular files: |
65 | | - |
66 | | -- `sherpa-onnx-core.js`: Core functionality, utilities, and file system operations |
67 | | -- `sherpa-onnx-vad.js`: Voice Activity Detection functionality |
68 | | -- `sherpa-onnx-combined.js`: Loader that loads all individual modules |
69 | | - |
70 | | -Additional modules will be added in the future: |
71 | | -- `sherpa-onnx-asr.js`: Automatic Speech Recognition functionality |
72 | | -- `sherpa-onnx-tts.js`: Text-to-Speech functionality |
73 | | -- And more... |
74 | | - |
75 | | -## Demo Application |
| 9 | +- Speech Enhancement |
76 | 10 |
|
77 | | -The included `index.html` demonstrates how to use the combined module. It shows: |
| 11 | +## File Structure |
78 | 12 |
|
79 | | -1. How to load models from URLs |
80 | | -2. How to initialize each component (ASR, VAD, TTS) |
81 | | -3. How to stream audio from the microphone |
82 | | -4. How to get results from each component |
| 13 | +When built, the following files are generated: |
| 14 | +- `sherpa-onnx-wasm-combined.js` - The main JavaScript glue code |
| 15 | +- `sherpa-onnx-wasm-combined.wasm` - The WebAssembly binary |
| 16 | +- `sherpa-onnx-wasm-combined.data` - The preloaded assets (models) |
| 17 | +- JS library files: |
| 18 | + - `sherpa-onnx-core.js` - Core functionality |
| 19 | + - `sherpa-onnx-asr.js` - ASR functionality |
| 20 | + - `sherpa-onnx-vad.js` - VAD functionality |
| 21 | + - `sherpa-onnx-tts.js` - TTS functionality |
| 22 | + - `sherpa-onnx-kws.js` - Keyword Spotting functionality |
| 23 | + - `sherpa-onnx-speaker.js` - Speaker Diarization functionality |
| 24 | + - `sherpa-onnx-enhancement.js` - Speech Enhancement functionality |
| 25 | + - `sherpa-onnx-combined.js` - Combined functionality wrapper |
83 | 26 |
|
84 | | -## Building the Module |
| 27 | +## Building |
85 | 28 |
|
86 | | -The WebAssembly module can be built using the provided build script: |
| 29 | +To build the WASM module: |
87 | 30 |
|
88 | 31 | ```bash |
89 | 32 | cd /path/to/sherpa-onnx |
90 | 33 | ./build-wasm-combined.sh |
91 | 34 | ``` |
92 | 35 |
|
93 | | -The built files will be located in `bin/wasm/combined/` and are also copied to `wasm/combined/`. |
94 | | - |
95 | | -## Setting Up Models |
96 | | - |
97 | | -Before using the demo, you need to set up model files: |
| 36 | +This script will: |
| 37 | +1. Create a `build-wasm-combined` directory |
| 38 | +2. Configure CMake with the necessary options |
| 39 | +3. Build the WASM module |
| 40 | +4. Install the files to `bin/wasm/combined` |
| 41 | +5. Copy the files to the original repo at `wasm/combined` |
98 | 42 |
|
99 | | -```bash |
100 | | -cd /path/to/sherpa-onnx/wasm/combined |
101 | | -./setup-assets.sh |
102 | | -``` |
| 43 | +## Important Notes |
103 | 44 |
|
104 | | -This script will download necessary model files to the `assets/` directory. |
| 45 | +1. **Large Asset Bundle**: The `.data` file can be very large (300MB+) as it contains all preloaded models. For production, consider using dynamic loading of models instead. |
105 | 46 |
|
106 | | -## Testing the Demos |
| 47 | +2. **File Locations**: All files must be in the same directory for the WASM module to work correctly. The `.data` file MUST be in the same directory as the `.js` and `.wasm` files. |
107 | 48 |
|
108 | | -After building the module and setting up the models, you can test the demos using Python's built-in HTTP server: |
| 49 | +3. **Local Testing**: To test locally, run a web server from the `wasm/combined` directory: |
109 | 50 |
|
110 | 51 | ```bash |
111 | 52 | cd /path/to/sherpa-onnx/wasm/combined |
112 | | -python3 -m http.server 8080 |
| 53 | +python -m http.server 8000 |
113 | 54 | ``` |
114 | 55 |
|
115 | | -Then navigate to http://localhost:8080/demos/ in your web browser to access the demos. |
| 56 | +Then open `http://localhost:8000` in your browser. |
116 | 57 |
|
117 | | -## Troubleshooting |
| 58 | +## License |
118 | 59 |
|
119 | | -- **Module load errors**: Ensure the WASM module is loaded before any other scripts |
120 | | -- **Model load errors**: Check the browser console for specific error messages |
121 | | -- **Audio capture issues**: Make sure your browser has permission to access the microphone |
122 | | -- **Performance issues**: Try reducing buffer sizes or using smaller models |
| 60 | +Same as Sherpa-ONNX. |
0 commit comments