[Breaking] Use non-delta codec for oneof type #146
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Benchmark ResultBenchmark diff with base branchBenchmark result |
tigrannajaryan
changed the title
tigrannajaryan
force-pushed
the
tigran/oneofcodec
branch
7 times, most recently
from
6d65d69 to
7630244
Compare
tigrannajaryan
force-pushed
the
tigran/oneofcodec
branch
2 times, most recently
from
19f11ae to
823b28d
Compare
tigrannajaryan
changed the title
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR changes the oneof codec to use a non-delta encoding approach, replacing the previous delta-based type encoding with direct type encoding using fixed bit counts. This simplifies the codec implementation and improves performance at the cost of a slight increase in uncompressed size.
- Replaces delta encoding with fixed-width bit encoding for oneof types
- Updates specification to reflect the codec change from delta to direct encoding
- Modifies benchmark scripts to reference the correct branch for comparisons
Reviewed Changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| stefgen/templates/java/oneofEncoder.java.tmpl | Template for Java oneof encoders - removes delta encoding and adds fixed bit count encoding |
| stefgen/templates/java/oneofDecoder.java.tmpl | Template for Java oneof decoders - removes delta decoding and adds fixed bit count decoding |
| stefgen/templates/go/oneof.go.tmpl | Template for Go oneof encoder/decoder - removes delta encoding/decoding and adds fixed bit count approach |
| stef-spec/specification.md | Updates specification to document the change from delta to direct type encoding |
| Multiple generated Java files | Generated code implementing the new non-delta codec for various oneof types |
| Multiple generated Go files | Generated code implementing the new non-delta codec for various oneof types |
| benchmarks/scripts/genoldtestfiles.sh | Updates base branch reference for benchmark comparisons |
| .github/workflows/benchmark.yml | Adds test file generation step to benchmark workflow |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
java/src/main/java/com/example/oteltef/ExemplarValueDecoder.java
Outdated
Show resolved
Hide resolved
Resolves #127 This simplifies oneof codec and improves codec speed at the cost of slight increase of uncompressed size. The zstd-compressed size is approximately the same (slightly increased or decresed depending on the dataset). ### Benchmarks Otel schema speed improvement: ``` │ bench_base.txt │ bench_current.txt │ │ sec/point │ sec/point vs base │ SerializeNative/STEF/serialize-10 64.36n ± 1% 63.10n ± 2% -1.96% (p=0.020 n=9) DeserializeNative/STEF/deser-10 24.06n ± 1% 22.70n ± 1% -5.65% (p=0.000 n=9) geomean 39.35n 37.85n -3.82% ``` Otel schema size before the change: ``` File Uncompressed Zstd Bytes hipstershop-otelmetrics.stefz 432716 93702 hostandcollector-otelmetrics.stefz 1258652 83009 astronomy-otelmetrics.stefz 7462303 1652082 ``` After the change: ``` File Uncompressed Zstd Bytes hipstershop-otelmetrics.stefz 441558 95083 hostandcollector-otelmetrics.stefz 1266627 79835 astronomy-otelmetrics.stefz 7557122 1644464 ```
tigrannajaryan
force-pushed
the
tigran/oneofcodec
branch
from
823b28d to
a1f342d
Compare
maheshkanote
approved these changes
Sep 23, 2025
tigrannajaryan
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #127
This simplifies oneof codec and improves codec speed at the cost of slight increase
of uncompressed size. The zstd-compressed size is approximately the same (slightly
increased or decresed depending on the dataset).
Benchmarks
Otel schema speed improvement:
Otel schema size before the change:
After the change:
Other benchmarks show similar patterns for speed and size changes.