Skip to content

Commit 32d5321

Browse files
srilmanscott-routledge2
authored andcommitted
Build linux64 with Bodo Changes
- With Sccache - Upload to Internal Artifactory Apply Bodo Changes Update Bodo Patch Enable SIMD and Upload Disable SIMD from failure Fix and SCCache Add creds Forgot Creds Pass through docker Was gitignored pass through correctly Fix Bug
1 parent f3c70da commit 32d5321

File tree

8 files changed

+364
-30
lines changed

8 files changed

+364
-30
lines changed

.azure-pipelines/azure-pipelines-linux.yml

Lines changed: 19 additions & 24 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

.scripts/build_steps.sh

Lines changed: 3 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

.scripts/run_docker_build.sh

Lines changed: 4 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

.scripts/upload_binary.sh

Lines changed: 20 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

azure-pipelines.yml

Lines changed: 0 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

recipe/build.sh

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,12 @@ fi
4848
# reusable variable for dependencies we cannot yet unvendor
4949
export READ_RECIPE_META_YAML_WHY_NOT=OFF
5050

51+
# Enable SCCache
52+
export SCCACHE_BUCKET=engine-codebuild-cache
53+
export SCCACHE_REGION=us-east-2
54+
export SCCACHE_S3_USE_SSL=true
55+
export SCCACHE_S3_SERVER_SIDE_ENCRYPTION=true
56+
5157
# for available switches see
5258
# https://github.com/apache/arrow/blame/apache-arrow-12.0.0/cpp/cmake_modules/DefineOptions.cmake
5359
# placeholder in ARROW_GDB_INSTALL_DIR must match _la_placeholder in activate.sh
@@ -86,6 +92,7 @@ cmake -GNinja \
8692
-DARROW_SUBSTRAIT=ON \
8793
-DARROW_USE_GLOG=ON \
8894
-DARROW_USE_LD_GOLD=ON \
95+
-DARROW_USE_SCCACHE=ON \
8996
-DARROW_WITH_BROTLI=ON \
9097
-DARROW_WITH_BZ2=ON \
9198
-DARROW_WITH_LZ4=ON \

recipe/meta.yaml

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,8 @@ source:
2424
- patches/0002-skip-NonExistentBucket-test-on-osx.patch
2525
# backport https://github.com/apache/arrow/pull/45859
2626
- patches/0003-MINOR-C-Fix-OTel-1.19-compat-after-silent-breaking-c.patch
27-
27+
# Bodo-specific Changes
28+
- patches/0004-Bodo-Changes.patch
2829
# testing-submodules not part of release tarball
2930
- git_url: https://github.com/apache/arrow-testing.git
3031
git_rev: d2a13712303498963395318a4eb42872e66aead7
@@ -39,6 +40,11 @@ build:
3940
# for cuda support, building with one version is enough to be compatible with
4041
# all later versions, since arrow is only using libcuda, and not libcudart.
4142
skip: true # [cuda_compiler_version not in ("None", cuda_compiler_version_min)]
43+
# arrow promises API- & ABI-compatibility along SemVer, see #1096
44+
script_env:
45+
# Necessary for sccache to find AWS
46+
- AWS_ACCESS_KEY_ID
47+
- AWS_SECRET_ACCESS_KEY
4248

4349
requirements:
4450
build:
@@ -60,6 +66,7 @@ requirements:
6066
- make # [unix]
6167
# where orc ends up looking (because during build, CONDA_PREFIX==BUILD_PREFIX)
6268
- tzdata
69+
- sccache
6370
host:
6471
# for required dependencies, see
6572
# https://github.com/apache/arrow/blob/apache-arrow-11.0.0/cpp/cmake_modules/ThirdpartyToolchain.cmake#L46-L75

0 commit comments

Comments
 (0)