Cache silero_vad and prefetch functions #235
Description
To reduce latency between when an instance is created and when it is used, it would be useful to expose a static prefetch function in the MicVad class so that big dependencies can be loaded immediately so when we actually need to create the instance we can do it immediately without waiting for the files to be fetched.
I did this in a dumb way by doing:
const a = await MicVAD.new({
onnxWASMBasePath: 'https://cdn.jsdelivr.net/npm/onnxruntime-web@1.23.0/dist/',
baseAssetPath: 'https://cdn.jsdelivr.net/npm/@ricky0123/vad-web@0.0.27/dist/',
getStream() {
return new Promise(() => {
});
},
});
a.destroy();Which works, as it prefetches the deps, but it has one issue, that the silero_vad_v5.onnx file is fetched every time a new instance of the mic vad is created, which is wasteful and increases latency.
In the default model fetcher file, caching could be added so that if the model was fetched once, further calls return the already fetched value, something like:
const cache = new Map<string, Promise<ArrayBuffer>>();
export const defaultModelFetcher = (path: string): Promise<ArrayBuffer> => {
const cachedPromise = cache.get(path);
if (cachedPromise) {
return cachedPromise;
}
const newPromise = fetch(path)
.then((response) => {
if (!response.ok) {
throw new Error(`Failed to fetch model from ${path}: ${response.statusText}`);
}
return response.arrayBuffer();
})
.catch((error) => {
cache.delete(path);
throw error;
});
cache.set(path, newPromise);
return newPromise;
};