-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Change Brotli CompressionLevel.Fastest to use quality 2 instead of 1 #120433
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Co-authored-by: rzikm <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems reasonable to me.
I'd just add a comment for why we don't want to use 0 or 1 in this case.
@copilot address review comments |
Added a comment explaining why quality 2 is used instead of 0 or 1 for CompressionLevel.Fastest. The comment explains that qualities 0 and 1 produce poor compression for incremental writes, often resulting in output larger than the uncompressed input. (48e9b7d) |
…test Co-authored-by: rzikm <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Adjusts the internal Brotli quality mapping for CompressionLevel.Fastest to improve compression effectiveness for incremental (small chunk) writes without requiring external buffering.
- Map CompressionLevel.Fastest from quality 1 to quality 2
- Add explanatory inline comment documenting the rationale
/ba-g Test failures are unrelated |
Original prompt
This section details on the original issue you should resolve
<issue_title>Brotli very inefficient with smaller writes</issue_title>
<issue_description>When writing to a Brotli stream one line at a time, the compressed output is actually larger than the uncompressed input.
Repro:
Output:
Buffering the entire contents in memory then writing it all in a single write avoids the issue, but then that defeats the purpose of using a stream (may as well just use a function that compresses a byte array, and totally avoid streams).</issue_description>
<agent_instructions>Change Brotli to use Quality 2 for CompressionLevel.Fastest.</agent_instructions>
Comments on the Issue (you are @copilot in this section)
@stephentoub > Adding an internal buffer to BrotliStream, as has been done with other streams in the past, would be a good start.This is why BufferedStream exists. With the main exception of FileStream, which has a buffer purely for perf (but where we generally regret the inclusion of the buffer directly), other streams that include buffers typically do so out of necessity (e.g. there's a minimum size required to perform its transformation) rather than for perf. </comment_new>
<comment_new>@stephentoub
There's a difference between being functional and having great performance. Lots of things improve on throughput when input is buffered and processed in larger chunks. For example, every write to a NetworkStream involves a syscall; that doesn't mean we should add a buffer into NetworkStream. If you want your input buffered, you add buffering on top.</comment_new>
<comment_new>@stephentoub
It's doing what the user asked: compressing input at the level requested with the data requested at the time requested. There might be other things that can be improved, but forcing input buffering as part of the stream implementation itself is not the answer. With many compression algorithms, changing the chunk size improves compression ratio; it's not the job of BrotliStream to decide what an acceptable min compression ratio is... its job is to provide a Stream facade on top of the BrotliEncoder/Decoder. And as @svick outlined, changing the compression level improves things, as does using BufferedStream, whose entire reason to exist is to make it easy to layer on buffering to improve performance, whether that performance be about throughout or compression ratios.</comment_new>
<comment_new>@stephentoub
Achieving the optimal level of compression might require buff...
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.