Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 75 additions & 63 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,14 +53,40 @@ require 'zstd-ruby'
#### Simple Compression

```ruby
compressed_data = Zstd.compress(data)
compressed_data = Zstd.compress(data, level: complession_level) # default compression_level is 3
compressed_data = Zstd.compress(data) # default: 3
compressed_data = Zstd.compress(data, level: 6)
```

#### Compression with Dictionary
### Context-based Compression

For better performance with multiple operations, use reusable contexts:

```ruby
# dictionary is supposed to have been created using `zstd --train`
compressed_using_dict = Zstd.compress("", dict: File.read('dictionary_file'))
# Unified context (recommended)
ctx = Zstd::Context.new(level: 6)
compressed = ctx.compress(data)
original = ctx.decompress(compressed)

# Specialized contexts for memory optimization
cctx = Zstd::CContext.new(level: 6) # Compression-only
dctx = Zstd::DContext.new # Decompression-only
```

### Dictionary Compression

Dictionaries provide better compression for similar data:

```ruby
dictionary = File.read('dictionary_file')

# Using module methods
compressed = Zstd.compress(data, level: 3, dict: dictionary)
original = Zstd.decompress(compressed, dict: dictionary)

# Using contexts for better performance
ctx = Zstd::Context.new(level: 6, dict: dictionary)
compressed = ctx.compress(data)
original = ctx.decompress(compressed)
```

#### Compression with CDict
Expand Down Expand Up @@ -128,16 +154,9 @@ res << stream.finish

### Decompression

#### Simple Decompression

```ruby
data = Zstd.decompress(compressed_data)
```

#### Decompression with Dictionary
```ruby
# dictionary is supposed to have been created using `zstd --train`
Zstd.decompress(compressed_using_dict, dict: File.read('dictionary_file'))
data = Zstd.decompress(compressed_data, dict: dictionary)
```

#### Decompression with DDict
Expand All @@ -157,79 +176,73 @@ result = ''
result << stream.decompress(cstr[0, 10])
result << stream.decompress(cstr[10..-1])
```
## API Reference

### Context Classes

#### `Zstd::Context`
Unified context for both compression and decompression.

#### Streaming Decompression with dictionary
```ruby
cstr = "" # Compressed data
stream = Zstd::StreamingDecompress.new(dict: File.read('dictionary_file'))
result = ''
result << stream.decompress(cstr[0, 10])
result << stream.decompress(cstr[10..-1])
ctx = Zstd::Context.new # Default settings
ctx = Zstd::Context.new(level: 6) # With compression level
ctx = Zstd::Context.new(level: 6, dict: dictionary) # With dictionary
```

DDict can also be specified to `dict:`.
- `compress(data)` → String
- `decompress(compressed_data)` → String

#### Streaming Decompression with Position Tracking

If you need to know how much of the input data was consumed during decompression, you can use the `decompress_with_pos` method:
#### `Zstd::CContext`
Compression-only context for memory optimization.

```ruby
cstr = "" # Compressed data
stream = Zstd::StreamingDecompress.new
result, consumed_bytes = stream.decompress_with_pos(cstr[0, 10])
# result contains the decompressed data
# consumed_bytes contains the number of bytes from input that were processed
cctx = Zstd::CContext.new(level: 6) # With compression level
cctx = Zstd::CContext.new(level: 6, dict: dictionary) # With dictionary
```

This is particularly useful when processing streaming data where you need to track the exact position in the input stream.
- `compress(data)` → String

### Skippable frame
#### `Zstd::DContext`
Decompression-only context for memory optimization.

```ruby
compressed_data_with_skippable_frame = Zstd.write_skippable_frame(compressed_data, "sample data")

Zstd.read_skippable_frame(compressed_data_with_skippable_frame)
# => "sample data"
dctx = Zstd::DContext.new # Default settings
dctx = Zstd::DContext.new(dict: dictionary) # With dictionary
```

### Stream Writer and Reader Wrapper
**EXPERIMENTAL**
- `decompress(compressed_data)` → String

* These features are experimental and may be subject to API changes in future releases.
* There may be performance and compatibility issues, so extensive testing is required before production use.
* If you have any questions, encounter bugs, or have suggestions, please report them via [GitHub issues](https://github.com/SpringMT/zstd-ruby/issues).
### Module Methods

#### Zstd::StreamWriter
#### Compression
- `Zstd.compress(data)` → String (default level 3)
- `Zstd.compress(data, dict: dictionary)` → String

```ruby
require 'stringio'
require 'zstd-ruby'
#### Decompression
- `Zstd.decompress(compressed_data)` → String
- `Zstd.decompress(compressed_data, dict: dictionary)` → String

io = StringIO.new
stream = Zstd::StreamWriter.new(io)
stream.write("abc")
stream.finish
#### Utilities
- `Zstd.zstd_version` → Integer

io.rewind
# Retrieve the compressed data
compressed_data = io.read
```
### Performance Guidelines

#### Zstd::StreamReader
| Use Case | Recommended API | Benefits |
|----------|----------------|----------|
| Single operations | `Zstd.compress/decompress` | Simple, no setup |
| Multiple operations | `Zstd::Context` | 2-3x faster, convenient |
| Specialized needs | `Zstd::CContext/DContext` | Direct API access |

```ruby
require 'stringio'
require 'zstd-ruby' # Add the appropriate require statement if necessary
**Compression Levels:** 1-3 (fast, 3 is default), 9-19 (better compression)

io = StringIO.new(compressed_data)
reader = Zstd::StreamReader.new(io)
## Benchmarks

# Read and output the decompressed data
puts reader.read(10) # 'abc'
puts reader.read(10) # 'def'
puts reader.read(10) # '' (end of data)
```
To test performance on your system:

```bash
cd benchmarks
ruby quick_benchmark.rb # Fast overview of all APIs (recommended)
```

## JRuby
This gem does not support JRuby.
Expand Down Expand Up @@ -266,7 +279,6 @@ To install this gem onto your local machine, run `bundle exec rake install`. To

Bug reports and pull requests are welcome on GitHub at https://github.com/SpringMT/zstd-ruby. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.


## License

The gem is available as open source under the terms of the [BSD-3-Clause License](https://opensource.org/licenses/BSD-3-Clause).
Expand Down
49 changes: 49 additions & 0 deletions benchmarks/context_reuse.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
require 'benchmark/ips'

$LOAD_PATH.unshift '../lib'
require 'zstd-ruby'

# Sample data - size typical of real-world scenarios
data = "Hello, World! " * 1000 # ~13KB string

# Pre-compress data for decompression tests
compressed_data = Zstd.compress(data)

puts "Benchmarking context reuse vs module methods"
puts "Data size: #{data.bytesize} bytes, Compressed: #{compressed_data.bytesize} bytes"
puts

Benchmark.ips do |x|
x.config(time: 3, warmup: 1)

# Compression benchmarks
x.report("Module compress (new context each time)") do
Zstd.compress(data)
end

ctx = Zstd::Context.new
x.report("Context compress (reused)") do
ctx.compress(data)
end

cctx = Zstd::CContext.new
x.report("CContext compress (reused)") do
cctx.compress(data)
end

# Decompression benchmarks
x.report("Module decompress (new context each time)") do
Zstd.decompress(compressed_data)
end

x.report("Context decompress (reused)") do
ctx.decompress(compressed_data)
end

dctx = Zstd::DContext.new
x.report("DContext decompress (reused)") do
dctx.decompress(compressed_data)
end

x.compare!
end
Loading