hyparquet has an AsyncBuffer type, but it would be great if it could support Blob out of the box:
- Blob have a
.slice() function that return Promise<Blob>
- They also have a
.arrayBuffer() that returns Promise<ArrayBuffer>
- They have a
.stream() function that return a web stream
It's an interoperable types, returned by the browser when handling user files, or by BunFile from bun, etc (our specifc usecase is for the downloadFile of @huggingface/hub which also implements Blob subclasses like WebBlob, FileBlob, ...)
A wrapper could look like this:
function blobToAsyncBuffer(blob: Blob) {
return {
byteLength: blob.size,
slice(start, end) => blob.slice(star,end).then(blob => blob.arrayBuffer())
}
}but the lib handling blob.stream() natively would be nice as well, avoiding making several range request on a fetch for example, and reading the data as it comes.
hyparquethas anAsyncBuffertype, but it would be great if it could supportBlobout of the box:.slice()function that returnPromise<Blob>.arrayBuffer()that returnsPromise<ArrayBuffer>.stream()function that return a web streamIt's an interoperable types, returned by the browser when handling user files, or by
BunFilefrom bun, etc (our specifc usecase is for thedownloadFileof@huggingface/hubwhich also implementsBlobsubclasses likeWebBlob,FileBlob, ...)A wrapper could look like this:
but the lib handling
blob.stream()natively would be nice as well, avoiding making several range request on afetchfor example, and reading the data as it comes.