Skip to content

Fix compress/decompress in LLVM Offloading API #150064

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Oct 6, 2025
Merged

Fix compress/decompress in LLVM Offloading API #150064

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
8fd5968
SWDEV-52811 - fix compress/decompress in LLVM Offloading API
Jun 17, 2025
5a70f7d
Clean-up from review comments.
Aug 12, 2025
bd67241
Resolve PR comments.
Aug 19, 2025
98f59d4
Resolve PR comments part 2
david-salinas Aug 28, 2025
856bb01
Merge branch 'main' into correct-decompress-offloading-api
david-salinas Aug 29, 2025
d3c9a79
Change Offloading API to require a output stream to be passed for dec…
david-salinas Sep 4, 2025
de45503
Refactor API change for user specified output stream
david-salinas Sep 10, 2025
a375975
Merge branch 'main' into correct-decompress-offloading-api
david-salinas Sep 10, 2025
589e898
Resolve build issue and address PR comments.
david-salinas Sep 15, 2025
7ce94fe
Address PR comment.
david-salinas Sep 18, 2025
442a6bd
resolve PR comments
david-salinas Sep 22, 2025
32fe3e7
Merge branch 'main' into correct-decompress-offloading-api
david-salinas Oct 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions llvm/include/llvm/Object/OffloadBundle.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,9 @@ class CompressedOffloadBundle {

static llvm::Expected<std::unique_ptr<llvm::MemoryBuffer>>
compress(llvm::compression::Params P, const llvm::MemoryBuffer &Input,
uint16_t Version, bool Verbose = false);
uint16_t Version, bool Verbose = false, raw_ostream &OutS = errs());
static llvm::Expected<std::unique_ptr<llvm::MemoryBuffer>>
decompress(const llvm::MemoryBuffer &Input, bool Verbose = false);
decompress(const llvm::MemoryBuffer &Input, bool Verbose = false, raw_ostream &OS = errs());
};

/// Bundle entry in binary clang-offload-bundler format.
Expand Down
16 changes: 8 additions & 8 deletions llvm/lib/Object/OffloadBundle.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Error extractOffloadBundle(MemoryBufferRef Contents, uint64_t SectionOffset,
return createFileError(FileName, EC);

Expected<std::unique_ptr<MemoryBuffer>> DecompressedBufferOrErr =
CompressedOffloadBundle::decompress(**CodeOrErr, false);
CompressedOffloadBundle::decompress(**CodeOrErr, false, errs());
if (!DecompressedBufferOrErr)
return createStringError(
inconvertibleErrorCode(),
Expand Down Expand Up @@ -297,7 +297,7 @@ static std::string formatWithCommas(unsigned long long Value) {
Expected<std::unique_ptr<MemoryBuffer>>
CompressedOffloadBundle::compress(compression::Params P,
const MemoryBuffer &Input, uint16_t Version,
bool Verbose) {
bool Verbose, raw_ostream &OutS) {
if (!compression::zstd::isAvailable() && !compression::zlib::isAvailable())
return createStringError("compression not supported.");
Timer HashTimer("Hash Calculation Timer", "Hash calculation time",
Expand Down Expand Up @@ -385,7 +385,7 @@ CompressedOffloadBundle::compress(compression::Params P,
double CompressionTimeSeconds = CompressTimer.getTotalTime().getWallTime();
double CompressionSpeedMBs =
(UncompressedSize64 / (1024.0 * 1024.0)) / CompressionTimeSeconds;
errs() << "Compressed bundle format version: " << Version << "\n"
OutS << "Compressed bundle format version: " << Version << "\n"
<< "Total file size (including headers): "
<< formatWithCommas(TotalFileSize64) << " bytes\n"
<< "Compression method used: " << MethodUsed << "\n"
Expand Down Expand Up @@ -510,7 +510,7 @@ CompressedOffloadBundle::CompressedBundleHeader::tryParse(StringRef Blob) {
}

Expected<std::unique_ptr<MemoryBuffer>>
CompressedOffloadBundle::decompress(const MemoryBuffer &Input, bool Verbose) {
CompressedOffloadBundle::decompress(const MemoryBuffer &Input, bool Verbose, raw_ostream &OS) {
StringRef Blob = Input.getBuffer();

// Check minimum header size (using V1 as it's the smallest).
Expand All @@ -519,7 +519,7 @@ CompressedOffloadBundle::decompress(const MemoryBuffer &Input, bool Verbose) {

if (identify_magic(Blob) != file_magic::offload_bundle_compressed) {
if (Verbose)
errs() << "Uncompressed bundle\n";
OS << "Uncompressed bundle\n";
return MemoryBuffer::getMemBufferCopy(Blob);
}

Expand Down Expand Up @@ -577,11 +577,11 @@ CompressedOffloadBundle::decompress(const MemoryBuffer &Input, bool Verbose) {
double DecompressionSpeedMBs =
(UncompressedSize / (1024.0 * 1024.0)) / DecompressionTimeSeconds;

errs() << "Compressed bundle format version: " << ThisVersion << "\n";
OS << "Compressed bundle format version: " << ThisVersion << "\n";
if (ThisVersion >= 2)
errs() << "Total file size (from header): "
OS << "Total file size (from header): "
<< formatWithCommas(TotalFileSize) << " bytes\n";
errs() << "Decompression method: "
OS << "Decompression method: "
<< (CompressionFormat == compression::Format::Zlib ? "zlib" : "zstd")
<< "\n"
<< "Size before decompression: "
Expand Down
Loading