Skip to content

Add brotli support to FoundationNetworking #83441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Add brotli support to FoundationNetworking #83441

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
44acb66
Add brotli support to FoundationNetworking
zhenchaoli Jul 30, 2025
971f6ea
use path computation helper
zhenchaoli Aug 5, 2025
f384b05
Use BROTLI_DIR
zhenchaoli Aug 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion utils/build.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -720,6 +720,7 @@ enum Project {
SourceKitLSP
SymbolKit
DocC
brotli

LLVM
Runtime
Expand Down Expand Up @@ -2260,6 +2261,21 @@ function Build-Sanitizers([Hashtable] $Platform) {
}
}

function Build-Brotli([Hashtable] $Platform) {
Build-CMakeProject `
-Src $SourceCache\brotli `
-Bin "$(Get-ProjectBinaryCache $Platform brotli)" `
-InstallTo "$BinaryCache\$($Platform.Triple)\usr" `
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to install it? Can we not use it from the build tree?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm following the example of zlib here, and it feels cleaner to rely on build artifacts being in a consistent place for all dependency outputs. It seems all other dependencies already do this, so I think we should be consistent. (Maybe let's do these kind of changes in a separate PR for all dependencies if you feel strongly about it)

Copy link
Member

@compnerd compnerd Aug 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

zlib is the only one doing this because it cannot be used from the build tree (I would love to migrate to that model for zlib as well). We no longer install any library which is not being distributed as a DLL - but that requires switching to the master branch rather than a release.

Copy link
Author

@zhenchaoli zhenchaoli Aug 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be clear, by "use it from the build tree", we'll be referencing .lib object files in brotli build tree like brotli\out\Release\brotlicommon.lib, and header files in its source tree at paths like brotli\c\include\brotli\decode.h?
I could try but I doubt it'll play well with the goal in the other comment which is using Brotli_DIR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, except, the idea with Brotli_DIR is that you tell CMake the directory to look into for BrotliConfig.cmake and it will wire up the headers and libraries appropriately. You don't need to pass flags or paths to the libraries or headers.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I will experiment and report back.

-Platform $Platform `
-UseMSVCCompilers C `
-Defines @{
BUILD_SHARED_LIBS = "NO";
CMAKE_POSITION_INDEPENDENT_CODE = "YES";
CMAKE_SYSTEM_NAME = $Platform.OS.ToString();
}
}


function Build-ZLib([Hashtable] $Platform) {
Build-CMakeProject `
-Src $SourceCache\zlib `
Expand Down Expand Up @@ -2344,7 +2360,7 @@ function Build-CURL([Hashtable] $Platform) {
CURL_CA_BUNDLE = "none";
CURL_CA_FALLBACK = "NO";
CURL_CA_PATH = "none";
CURL_BROTLI = "NO";
CURL_BROTLI = "YES";
CURL_DISABLE_ALTSVC = "NO";
CURL_DISABLE_AWS = "YES";
CURL_DISABLE_BASIC_AUTH = "NO";
Expand Down Expand Up @@ -2422,6 +2438,7 @@ function Build-CURL([Hashtable] $Platform) {
USE_WIN32_LDAP = "NO";
ZLIB_ROOT = "$BinaryCache\$($Platform.Triple)\usr";
ZLIB_LIBRARY = "$BinaryCache\$($Platform.Triple)\usr\lib\zlibstatic.lib";
BROTLI_DIR = "$BinaryCache\$($Platform.Triple)\usr";
})
}

Expand Down Expand Up @@ -2835,6 +2852,7 @@ function Build-Foundation {
"$BinaryCache\$($Platform.Triple)\usr\lib\libz.a"
};
ZLIB_INCLUDE_DIR = "$BinaryCache\$($Platform.Triple)\usr\include";
BROTLI_DIR = "$BinaryCache\$($Platform.Triple)\usr";
dispatch_DIR = $DispatchCMakeModules;
SwiftSyntax_DIR = (Get-ProjectBinaryCache $HostPlatform Compilers);
_SwiftFoundation_SourceDIR = "$SourceCache\swift-foundation";
Expand All @@ -2857,6 +2875,7 @@ function Test-Foundation {
$env:LIBXML_LIBRARY_PATH="$BinaryCache/$($Platform.Triple)/usr/lib"
$env:LIBXML_INCLUDE_PATH="$BinaryCache/$($Platform.Triple)/usr/include/libxml2"
$env:ZLIB_LIBRARY_PATH="$BinaryCache/$($Platform.Triple)/usr/lib"
$env:BROTLI_LIBRARY_PATH="$BinaryCache/$($Platform.Triple)/usr/lib"
$env:CURL_LIBRARY_PATH="$BinaryCache/$($Platform.Triple)/usr/lib"
$env:CURL_INCLUDE_PATH="$BinaryCache/$($Platform.Triple)/usr/include"
Build-SPMProject `
Expand Down Expand Up @@ -3004,6 +3023,7 @@ function Build-SDK([Hashtable] $Platform, [switch] $IncludeMacros = $false) {

# Third Party Dependencies
Invoke-BuildStep Build-ZLib $Platform
Invoke-BuildStep Build-Brotli $Platform
Invoke-BuildStep Build-XML2 $Platform
Invoke-BuildStep Build-CURL $Platform
Invoke-BuildStep Build-LLVM $Platform
Expand Down
7 changes: 7 additions & 0 deletions utils/update_checkout/update-checkout-config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,9 @@
"zlib": {
"remote": { "id": "madler/zlib" }
},
"brotli": {
"remote": { "id": "google/brotli" }
},
"mimalloc": {
"remote": { "id": "microsoft/mimalloc" },
"platforms": [ "Windows" ]
Expand Down Expand Up @@ -180,6 +183,7 @@
"curl": "curl-8_9_1",
"libxml2": "v2.11.5",
"zlib": "v1.3.1",
"brotli": "v1.1.0",
"mimalloc": "v3.0.3",
"swift-subprocess": "development-snapshot-2025-07-21"
}
Expand Down Expand Up @@ -236,6 +240,7 @@
"curl": "curl-8_9_1",
"libxml2": "v2.11.5",
"zlib": "v1.3.1",
"brotli": "v1.1.0",
"mimalloc": "v3.0.1"
}
},
Expand Down Expand Up @@ -498,6 +503,7 @@
"curl": "curl-8_9_1",
"libxml2": "v2.11.5",
"zlib": "v1.3.1",
"brotli": "v1.1.0",
"mimalloc": "v3.0.3",
"swift-subprocess": "development-snapshot-2025-07-21"
}
Expand Down Expand Up @@ -556,6 +562,7 @@
"curl": "curl-8_9_1",
"libxml2": "v2.11.5",
"zlib": "v1.3.1",
"brotli": "v1.1.0",
"mimalloc": "v3.0.3",
"swift-subprocess": "development-snapshot-2025-07-21"
}
Expand Down