Skip to content

Commit e5b6dc4

Browse files
srilmanIsaacWarren
authored andcommitted
Build linux64 with Bodo Changes
- With Sccache - Upload to Internal Artifactory Apply Bodo Changes Update Bodo Patch Enable SIMD and Upload Disable SIMD from failure Fix and SCCache Add creds Forgot Creds Pass through docker Was gitignored pass through correctly Fix Bug
1 parent 6469215 commit e5b6dc4

File tree

8 files changed

+359
-25
lines changed

8 files changed

+359
-25
lines changed

.azure-pipelines/azure-pipelines-linux.yml

Lines changed: 16 additions & 21 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

.scripts/build_steps.sh

Lines changed: 2 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

.scripts/run_docker_build.sh

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

.scripts/upload_binary.sh

Lines changed: 20 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

azure-pipelines.yml

Lines changed: 0 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

recipe/build.sh

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,12 @@ fi
4848
# reusable variable for dependencies we cannot yet unvendor
4949
export READ_RECIPE_META_YAML_WHY_NOT=OFF
5050

51+
# Enable SCCache
52+
export SCCACHE_BUCKET=engine-codebuild-cache
53+
export SCCACHE_REGION=us-east-2
54+
export SCCACHE_S3_USE_SSL=true
55+
export SCCACHE_S3_SERVER_SIDE_ENCRYPTION=true
56+
5157
# for available switches see
5258
# https://github.com/apache/arrow/blame/apache-arrow-12.0.0/cpp/cmake_modules/DefineOptions.cmake
5359
# placeholder in ARROW_GDB_INSTALL_DIR must match _la_placeholder in activate.sh
@@ -86,6 +92,7 @@ cmake -GNinja \
8692
-DARROW_SUBSTRAIT=ON \
8793
-DARROW_USE_GLOG=ON \
8894
-DARROW_USE_LD_GOLD=ON \
95+
-DARROW_USE_SCCACHE=ON \
8996
-DARROW_WITH_BROTLI=ON \
9097
-DARROW_WITH_BZ2=ON \
9198
-DARROW_WITH_LZ4=ON \

recipe/meta.yaml

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,8 @@ source:
2222
# upstream problems on with s3 tests on osx, see
2323
# https://github.com/apache/arrow/issues/35587
2424
- patches/0002-skip-NonExistentBucket-test-on-osx.patch
25-
25+
# Bodo-specific Changes
26+
- patches/0004-Bodo-Changes.patch
2627
# testing-submodules not part of release tarball
2728
- git_url: https://github.com/apache/arrow-testing.git
2829
git_rev: fbf6b703dc93d17d75fa3664c5aa2c7873ebaf06
@@ -36,6 +37,11 @@ build:
3637
# for cuda support, building with one version is enough to be compatible with
3738
# all later versions, since arrow is only using libcuda, and not libcudart.
3839
skip: true # [cuda_compiler_version not in ("None", cuda_compiler_version_min)]
40+
# arrow promises API- & ABI-compatibility along SemVer, see #1096
41+
script_env:
42+
# Necessary for sccache to find AWS
43+
- AWS_ACCESS_KEY_ID
44+
- AWS_SECRET_ACCESS_KEY
3945

4046
requirements:
4147
build:
@@ -57,6 +63,7 @@ requirements:
5763
- make # [unix]
5864
# where orc ends up looking (because during build, CONDA_PREFIX==BUILD_PREFIX)
5965
- tzdata
66+
- sccache
6067
host:
6168
# for required dependencies, see
6269
# https://github.com/apache/arrow/blob/apache-arrow-11.0.0/cpp/cmake_modules/ThirdpartyToolchain.cmake#L46-L75

0 commit comments

Comments
 (0)