Conversation
There was a problem hiding this comment.
Pull request overview
This PR implements transaction-level compression for sequencer block building. The TxCompressor operates at the individual transaction level rather than block level, maintaining compression context across transactions for improved compression ratios. This allows the sequencer to add transactions one-by-one until the compressed size threshold is reached.
Changes:
- Added Go-based TxCompressor library with JNA bindings for Kotlin/Java
- Implemented comprehensive compatibility tests between TxCompressor and BlobMaker
- Updated CI/CD workflows to build and release tx-compressor native libraries
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tracer/linea-constraints | Added subproject commit reference |
| settings.gradle | Added tx-compressor module to build |
| prover/lib/compressor/libtxcompressor/libtxcompressor.go | Core Go implementation of transaction compressor with C exports |
| prover/lib/compressor/blob/v1/tx_compressor_test.go | Comprehensive Go tests including compatibility tests with BlobMaker |
| prover/lib/compressor/blob/v1/tx_compressor.go | Transaction compressor implementation with compression context management |
| prover/lib/compressor/blob/v1/encode.go | Increased heuristic max transactions limit from 1024 to 16384 |
| prover/Makefile | Added build target for tx-compressor library |
| jvm-libs/linea/tx-compressor/src/test/kotlin/linea/blob/TxCompressorBlobMakerCompatibilityTest.kt | Kotlin compatibility tests verifying TxCompressor/BlobMaker integration |
| jvm-libs/linea/tx-compressor/src/test/kotlin/linea/blob/GoBackedTxCompressorTest.kt | Kotlin unit tests for TxCompressor functionality |
| jvm-libs/linea/tx-compressor/src/main/kotlin/linea/blob/TxCompressor.kt | Kotlin interface and Go-backed implementation with JNA bindings |
| jvm-libs/linea/tx-compressor/src/main/kotlin/linea/blob/GoNativeTxCompressor.kt | JNA interface definitions and factory for native library loading |
| jvm-libs/linea/tx-compressor/build.gradle | Gradle build configuration for tx-compressor module |
| constraints | Updated subproject commit reference |
| .github/workflows/prover-native-lib-blob-compressor-release.yml | Added tx-compressor to CI build and release workflow |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // TODO: Update URL when tx-compressor native libs are released | ||
| // For now, we'll use the same release as blob-compressor since they share the dictionary | ||
| fetchLibFromZip("https://github.com/Consensys/linea-monorepo/releases/download/blob-libs-v2.1.0-rc1/linea-blob-libs-v2.1.0-rc1.zip", "tx_compressor", libsZipDownloadOutputDir) |
There was a problem hiding this comment.
The TODO comment indicates the download URL is temporary and uses blob-compressor libs. This should be addressed before the PR is merged to ensure the correct tx-compressor native libraries are fetched.
| // TODO: Update URL when tx-compressor native libs are released | |
| // For now, we'll use the same release as blob-compressor since they share the dictionary | |
| fetchLibFromZip("https://github.com/Consensys/linea-monorepo/releases/download/blob-libs-v2.1.0-rc1/linea-blob-libs-v2.1.0-rc1.zip", "tx_compressor", libsZipDownloadOutputDir) | |
| fetchLibFromZip("https://github.com/Consensys/linea-monorepo/releases/download/tx-compressor-libs-v2.1.0-rc1/linea-tx-compressor-libs-v2.1.0-rc1.zip", "tx_compressor", libsZipDownloadOutputDir) |
| // TxError returns the last encountered error. | ||
| // If no error was encountered, returns nil. | ||
| // | ||
| //export TxError | ||
| func TxError() *C.char { | ||
| lock.Lock() | ||
| defer lock.Unlock() | ||
| if lastError != nil { | ||
| // this leaks memory, but since this represents a fatal error, it's probably ok. |
There was a problem hiding this comment.
The comment acknowledges memory leakage in error scenarios. Consider documenting the expected lifecycle and whether callers should be aware of this limitation, or if there's a cleanup mechanism that should be implemented.
| // TxError returns the last encountered error. | |
| // If no error was encountered, returns nil. | |
| // | |
| //export TxError | |
| func TxError() *C.char { | |
| lock.Lock() | |
| defer lock.Unlock() | |
| if lastError != nil { | |
| // this leaks memory, but since this represents a fatal error, it's probably ok. | |
| // TxError returns the last encountered error as a newly allocated, NUL-terminated C string. | |
| // If no error was encountered, it returns nil. | |
| // The caller takes ownership of the returned pointer and is responsible for freeing it | |
| // (for example, using free(3) or C.free) when it is no longer needed. | |
| // | |
| //export TxError | |
| func TxError() *C.char { | |
| lock.Lock() | |
| defer lock.Unlock() | |
| if lastError != nil { |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| return false, fmt.Errorf("failed to revert after forceReset: %w", err) | ||
| } | ||
| return true, nil | ||
| } |
There was a problem hiding this comment.
CanWrite corrupts compressor state after recompression path
High Severity
When forceReset is true (i.e., CanWrite), and the recompression path is triggered because the initial write exceeds the limit, calling tc.compressor.Revert() at the success label wipes ALL compressor state instead of just reverting the new transaction. After recompression, the entire payload (all previous + new transaction bytes) is written in a single Write call, so Revert() undoes that entire write, leaving the compressor empty. The BlobMaker.Write correctly handles this with a custom revert closure that checks recompressionAttempted and restores via payload[:prevLen], but TxCompressor is missing this logic.
Additional Locations (1)
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2382 +/- ##
============================================
- Coverage 61.13% 60.94% -0.20%
- Complexity 1626 1627 +1
============================================
Files 436 438 +2
Lines 17362 17425 +63
Branches 1908 1913 +5
============================================
+ Hits 10615 10620 +5
- Misses 6088 6146 +58
Partials 659 659
*This pull request uses carry forward flags. Click here to find out more.
🚀 New features to boost your workflow:
|
2 participants
This PR implements issue(s) #
Checklist
Note
Medium Risk
Introduces new native compression logic and cross-language bindings plus CI release artifacts, and relaxes a safety heuristic in blob decoding; mistakes could cause runtime crashes, size-limit miscalculations, or compatibility issues with existing blob compression/decompression.
Overview
Adds a new transaction-level additive compressor (
TxCompressor) alongside the existing block/blob compressor, including a Go implementation, C-shared exported API (TxInit/TxWrite/TxCanWrite/etc.), and a new JVM module (jvm-libs:linea:tx-compressor) with JNA bindings and tests.Updates build/release plumbing to produce and ship
tx_compressornative binaries (Makefile target + GitHub Actions workflow for linux/arm64/macos), and loosens the blob v1 block-scan heuristic cap (heuristicMaxNbTxs) to allow substantially more transactions per block.Written by Cursor Bugbot for commit ccacfe0. This will update automatically on new commits. Configure here.