Support for compression/decompression library

[KaruroChori]

At this point in time, there is no support for compression/decompression in txiki for what I can tell.
Users can bundle some like pure JS implementation like pako, but since quickjs is not exactly fast, that has a huge impact on performance. Binding zlib directly would be much more preferable from my point of view.

I am probably going to add it as a module myself, but even if the external module proposal is not mainlined I think there is a good case for some compression library to be shipped with txiki.

[saghul]

Yes! I've been thinking about wrapping libarchive. The annoying part is that it doesn't seem to be thread-safe, which means we'd need to run operations serialized in thre thread-pool, and watch out so no 2 things can run in parallel.

[KaruroChori]

Just my two cents on the subject.
From what I can tell, libarchive is quite massive and with a very broad scope. It would make perfect sense as an external module, but for many it would just end up being mostly "bloat" as part of the default distribution.
Something like zlib has a much more narrow focus, it self-describes as thread-safe and comes with all the basic features to support compression/decompression when implementing custom protocols (even just compressing HTTP) or data formats.

[saghul]

Good point.

There is already fflate bundled in, because it's a dependency of the ConpressionStream polyfill. Perhaps we could just expose it.

The reason I thought about libarchive is that in order to implement "tjs bundle" I need to download esbuild which comes in tar.gz form, so I'd need tar support too.

Alternatively I could just re-pack it and that's it...

[KaruroChori]

I would have preferred a native implementation, but that is specific for my use case. For general purpose I think exposing fflate would be best.

I was just wondering, esbuild has a wasm version: https://www.npmjs.com/package/esbuild-wasm
So in theory we can just use it, with the advantage of being portable regardless of the architecture.
Aside from the untarring process, for which we can surely find a workaround.

[saghul]

I thought of that but wasm3 is not very fast and I'm afraid we might not have all the APIs necessary for it to work, but I also haven't tried it yet.

[KaruroChori]

I am just testing it, it does not work because of URL.createObjectURL missing (and potentially more)

[saghul]

The hanks for giving it a try!

[jimmywarting]

just implement standard CompressionStream