Skip to content

[support-bundle] include for large log files #8771

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
+10 −2
Draft

[support-bundle] include for large log files #8771

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1b682a8
[jj-spr] initial version
papertigers Aug 5, 2025
df99f09
also enable large file in the nexus task
papertigers Aug 5, 2025
6c8413b
Note the compression method for support bundles
papertigers Aug 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion nexus/src/app/background/tasks/support_bundle_collector.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1162,7 +1162,9 @@ fn recursively_add_directory_to_zipfile(

let file_type = entry.file_type()?;
if file_type.is_file() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also: I wonder if we should pick an explicit compression scheme here, so we aren't at the whim of zip changing defaults.

E.g., we expect it'll be supported by customers, maybe we just pick zstd-3 here too

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's probably not a good idea to use zstd for the customer-accessible file because the standard zip tools on all major platforms don't support it. How would we access the file on Illumos, for example?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

according to https://docs.rs/zip/latest/zip/write/struct.FileOptions.html I think the default is "Deflate", and the level is either 24 or 6? https://docs.rs/zip/latest/zip/write/struct.FileOptions.html#method.compression_level

Good point re: zstd - just want to make sure an update of the zip crate doesn't change this for us

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah, agreed it makes sense to define which algorithm we want to use.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry this got put on the back burner for a bit.

I have rebased this PR and added the default compression method to deflate, however I am unsure what we should do about compression level. As @smklein pointed out above it's either 24 or 6 depending on if the zopfli feature is being used. It appears that it's currently included via the "deflate" feature flag to the zip crate. I don't know if we can specify something like no default features and explicitly enable deflate and zopfli.
Thoughts?

let opts = FullFileOptions::default();
let opts = FullFileOptions::default()
.compression_method(zip::CompressionMethod::Deflated)
.large_file(true);
let src = entry.path();

zip.start_file_from_path(dst, opts)?;
Expand Down
8 changes: 7 additions & 1 deletion sled-diagnostics/src/logs.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -720,7 +720,13 @@ fn write_log_to_zip<W: Write + Seek>(
zip_path,
FullFileOptions::default()
.compression_method(zip::CompressionMethod::Zstd)
.compression_level(Some(3)),
.compression_level(Some(3))
// NB: From the docs
// If set to false and the file exceeds the
// limit, an I/O error is thrown and the file is aborted. If set to
// true, readers will require ZIP64 support and if the file does not
// exceed the limit, 20 B are wasted.
.large_file(true),
)?;
if let Err(e) = std::io::copy(&mut src, zip) {
// If we fail here the `ZipWriter` is an unknown state and we are forced
Expand Down
Loading