Optimize Zstd decompression to reduce memory allocations and improve performance

[Flgado]

Summary

This change optimizes the Zstd decompression logic by switching from io.ReadAll(zstd.NewReader(bytes.NewReader(data))) to using zstd.Decoder.DecodeAll with a pre-allocated buffer.

Motivation

The old approach creates a new bytes.Reader and reads all decompressed data into a freshly allocated buffer, causing many heap allocations and increased GC pressure.

Impact

• Benchmark results show:
  • ~29% reduction in memory allocated per operation.
  • Consistent or slightly improved decompression speed.
  • No increase in allocation count (GC overhead unchanged).

This significantly improves memory efficiency, especially important for large result sets.

Benchmarks Example:

• Old version 10M rows

BenchmarkSpoolingProtocolSpooledSegmentlJsonZstdDecoderQuery-8   	       1	53275675497 ns/op	21708494160 B/op	300044245 allocs/op

• new version 10M row

cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
BenchmarkSpoolingProtocolSpooledSegmentlJsonZstdDecoderQuery-8   	       1	48507167074 ns/op	15496255624 B/op	300038202 allocs/op

The major impact was on io.Read allocation.