v2.0.1

What's Changed

• Prevent GC compaction and unintended sweeps for preventing checksum mismatch by @cosmo0920 in #116

Full Changelog: v2.0.0...v2.0.1

v2.0.0

What's Changed

BREAKING CHANGE

• Fix symbol collision with other native gems using libzstd by @SpringMT in #104
• Remove deprecated methods for v2 by @SpringMT in #108

Improvemnt

• Add versioning policy for v2.0.0+ by @SpringMT in #105
• Implements handling of partial frames by the streaming decompressor by @gucki in #88
• Add tests for decompress_with_pos method by @SpringMT in #106
• Add documentation for decompress_with_pos method by @SpringMT in #107
• V2 by @SpringMT in #109
• Potential fix for code scanning alert no. 10: Workflow does not contain permissions by @SpringMT in #110
• chore(deps): update actions/checkout action to v5 by @renovate[bot] in #111

Bug fix

• Add pending handler to concat 128kb or more data by @cosmo0920 in #114

New Contributors

• @gucki made their first contribution in #88
• @cosmo0920 made their first contribution in #114

Full Changelog: v1.5.7.0...v2.0.0

v1.5.7.1

What's Changed

• Merge pull request #114 from cosmo0920/add-pending-handler-to-concat-… by @SpringMT in #115

Full Changelog: v1.5.7.0...v1.5.7.1

v2.0.0 preview release 1

What's Changed

BREAKING CHANGE

• Fix symbol collision with other native gems using libzstd by @SpringMT in #104
• Remove deprecated methods for v2 by @SpringMT in #108

Improvemnt

• Add versioning policy for v2.0.0+ by @SpringMT in #105
• Implements handling of partial frames by the streaming decompressor by @gucki in #88
• Add tests for decompress_with_pos method by @SpringMT in #106
• Add documentation for decompress_with_pos method by @SpringMT in #107

New Contributors

• @gucki made their first contribution in #88

Full Changelog: v1.5.7.0...v2.0.0.pre.preview1

v1.5.7.0

What's Changed

• feat: update Zstd 1.5.7 by @SpringMT in #103

Full Changelog: v1.5.6.7...v1.5.7.0

v1.5.6.7

What's Changed

• tests: add decompress with large streaming compress data by @SpringMT in #92
• chore(deps): update dependency benchmark-ips to v2.14.0 by @renovate in #94
• Fix deprecation warning in StreamWriter by @zarqman in #96
• feat: add ruby 3.4.x by @SpringMT in #97
• feat: support CDict & DDict for faster operation with repeated dictionary use by @sekiyama58 in #98
• fix: missing argument names in rb_prohibit_copy (build failure on older GCC/Clang) by @flfymoss in #99
• Fix build error with GCC 15 by @Watson1978 in #100
• Fix ddict example in README by @catlee in #101

New Contributors

• @zarqman made their first contribution in #96
• @sekiyama58 made their first contribution in #98
• @flfymoss made their first contribution in #99
• @catlee made their first contribution in #101

Full Changelog: v1.5.6.6...v1.5.6.7

v1.5.6.6

What's Changed

• feat: add "zstd-ruby/stream_writer" and "zstd-ruby/stream_reader" loading by @SpringMT in #85
• fix: decompress with streaming compress binary by @SpringMT in #90

Full Changelog: v1.5.6.5...v1.5.6.6

v1.5.6.5

What's Changed

• feat: unlock GVL for simple compression and decompression by @SpringMT in #84

Improvemnt

• GVL unlocking is limited to streaming, but it is also unlocked for simple compression and decompression.
  • No action required.

Full Changelog: v1.5.6.4...v1.5.6.5

v1.5.6.4

What's Changed

• fix: free dctx for Zstd.decompress by @SpringMT in #81
• fix: compression crash by @SpringMT in #83

Bug Fix

• Fixed a memory leak when using Zstd.decompress().
  In versions 1.5.6.2 and 1.5.6.3, the context (ZSTD_DCtx) does not released after the decompression process was completed. This PR #81 addresses that issue.

• Resolved a crash when using Zstd.compress().
  In versions 1.5.6.2 and 1.5.6.3, Zstd.compress() internally used ZSTD_compressStream2, but it was crashing due to incomplete consumption of the internal buffer when large bytes were received. Zstd.compress() now uses ZSTD_compress2 instead of ZSTD_compressStream2 to ensure it doesn’t crash when processing large bytes.

If you are using versions v1.5.6.2 or v1.5.6.3, please update to version v1.5.6.4.

Full Changelog: v1.5.6.3...v1.5.6.4

v1.5.6.3

What's Changed

• Feature/unlock gvl for streaming compression/decompression by @SpringMT in #79

Full Changelog: v1.5.6.2...v1.5.6.3

Improvemnt

Unlock GVL for streaming compression/decompression

Zstd streaming compression/decompression is a CPU-only process that do not involve the ruby interpreter.
Therefore, it is possible to unlock the GVL, allowing for parallel multiple threads, thus fully utilizing CPU resources.

This program demonstrates the difference:
(in benckmarks)

Re-introduce #53

Streaming Compression

$LOAD_PATH.unshift '../lib'
require 'zstd-ruby'
require 'thread'

GUESSES = (ENV['GUESSES'] || 1000).to_i
THREADS = (ENV['THREADS'] || 1).to_i

p GUESSES: GUESSES, THREADS: THREADS

sample_file_name = ARGV[0]
json_string = File.read("./samples/#{sample_file_name}")

queue = Queue.new
GUESSES.times { queue << json_string }
THREADS.times { queue << nil }
THREADS.times.map {
  Thread.new {
    while str = queue.pop
      stream = Zstd::StreamingCompress.new
      stream << str
      res = stream.flush
      stream << str
      res << stream.finish
    end
  }
}.each(&:join)

Without this patch:

[springmt@MacBook-Pro] (main)✗ % time THREADS=4 bundle exec ruby multi_thread_streaming_comporess.rb city.json
{:GUESSES=>1000, :THREADS=>4}
THREADS=4 bundle exec ruby multi_thread_streaming_comporess.rb city.json  2.83s user 0.29s system 94% cpu 3.299 total

With the patch:

[springmt@MacBook-Pro] (feature/unlock-gvl)✗ % time THREADS=4 bundle exec ruby multi_thread_streaming_comporess.rb city.json
{:GUESSES=>1000, :THREADS=>4}
THREADS=4 bundle exec ruby multi_thread_streaming_comporess.rb city.json  3.33s user 0.36s system 266% cpu 1.385 total

Streaming Decompression

$LOAD_PATH.unshift '../lib'
require 'zstd-ruby'
require 'thread'

GUESSES = (ENV['GUESSES'] || 1000).to_i
THREADS = (ENV['THREADS'] || 1).to_i

p GUESSES: GUESSES, THREADS: THREADS

sample_file_name = ARGV[0]
json_string = File.read("./samples/#{sample_file_name}")
target = Zstd.compress(json_string)

queue = Queue.new
GUESSES.times { queue << target }
THREADS.times { queue << nil }
THREADS.times.map {
  Thread.new {
    while str = queue.pop
      stream = Zstd::StreamingDecompress.new
      stream.decompress(str)
      stream.decompress(str)
    end
  }
}.each(&:join)

Without this patch:

[springmt@MacBook-Pro] (main)✗ % time THREADS=4 bundle exec ruby multi_thread_streaming_decomporess.rb city.json
{:GUESSES=>1000, :THREADS=>4}
THREADS=4 bundle exec ruby multi_thread_streaming_decomporess.rb city.json  2.03s user 0.28s system 93% cpu 2.486 total

With the patch:

[springmt@MacBook-Pro] (feature/unlock-gvl)✗ % time THREADS=4 bundle exec ruby multi_thread_streaming_decomporess.rb city.json
{:GUESSES=>1000, :THREADS=>4}
THREADS=4 bundle exec ruby multi_thread_streaming_decomporess.rb city.json  2.49s user 0.49s system 227% cpu 1.310 total