Skip to content

Commit 894b035

Browse files
authored
tighten wheel size limits, expand CI-skipping logic, other small build changes (#804)
Proposes a batch of miscellaneous build / packaging / CI changes. ## Changes ### Tightens wheel size limits Contributes to rapidsai/build-planning#219 CUDA 13 wheels can be as much as 50% smaller than the equivalent CUDA 12 wheels, because of some new compression features in `nvcc`. To ensure surprising package-size growth is caught in CI, this PR tightens the limits in the following ways: * setting different limits based on CUDA major version where package size differs significantly * setting `libcuopt` limits to `{compressed_size} + 50Mi`, rounded to the nearest 5Mi * setting all other limits to `{compressed_size} + 10Mi`, rounded to the nearest 5Mi ### Expands CI-skipping logic Contributes to rapidsai/build-planning#243 Tries to avoid unnecessary CI runs by making the CI-skipping rules finer-grained. For example, PRs that only touch `.pre-commit-config.yaml` should now not require any runners with GPUs 😁 ### Removes some reliance on `pypi.nvidia.com` Contributes to rapidsai/build-planning#241 ```shell git grep -i -E 'pypi\.nvidia\.com' git grep -i -E 'rapids\-init\-pip' ``` And removed/updated all relevant references. This project does not need any wheels from `pypi.nvidia.com` at build-time or runtime, it can safely avoid searching that index. Its own packages are not available on `pypi.org` though, so uses of `pypi.nvidia.com` in docs are preserved here. ### Enforces PEP 639 license metadata in `pyproject.toml` Contributes to rapidsai/pre-commit-hooks#95 Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Ramakrishnap (https://github.com/rgsl888prabhu) URL: #804
1 parent 7c7e77d commit 894b035

19 files changed

+197
-83
lines changed

.github/workflows/pr.yaml

Lines changed: 124 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -111,66 +111,145 @@ jobs:
111111
uses: rapidsai/shared-workflows/.github/workflows/changed-files.yaml@main
112112
with:
113113
files_yaml: |
114-
test_cpp:
114+
build_docs:
115115
- '**'
116-
- '!CONTRIBUTING.md'
117-
- '!README.md'
118-
- '!ci/release/update-version-cuopt.sh'
119-
- '!ci/release/update-version-rapids.sh'
120-
- '!docs/**'
121-
- '!img/**'
122-
- '!notebooks/**'
123-
- '!python/**'
124-
- '!readme_pages/**'
116+
- '!.ai/**'
117+
- '!.coderabbit.yaml'
118+
- '!.github/AGENTS.md'
119+
- '!.github/CODE_OF_CONDUCT.md'
120+
- '!.github/CODEOWNERS'
121+
- '!.github/ISSUE_TEMPLATE/**'
122+
- '!.github/PULL_REQUEST_TEMPLATE.md'
123+
- '!.github/SECURITY.md'
124+
- '!.github/.ai/**'
125+
- '!.github/.coderabbit_review_guide.md'
126+
- '!.github/agents/**'
127+
- '!.github/copy-pr-bot.yaml'
128+
- '!.github/ops-bot.yaml'
129+
- '!.pre-commit-config.yaml'
130+
- '!ci/build_wheel*.sh'
131+
- '!ci/check_style.sh'
132+
- '!ci/release/**'
133+
- '!ci/run_ctests.sh'
134+
- '!ci/run_*.pytests.sh'
135+
- '!ci/test_cpp*.sh'
136+
- '!ci/test_notebooks.sh'
137+
- '!ci/test_python.sh'
138+
- '!ci/test_self_hosted_service.sh'
139+
- '!ci/test_wheel*.sh'
125140
- '!container-builder/**'
126-
- '!helm-chart/**'
141+
- '!helmchart/**'
127142
- '!ngc/**'
128-
- '!omniverse/**'
129-
- '!regression/**'
130-
- '!resources/**'
143+
- '!sonar-project.properties'
144+
- '!sonarqube/**'
131145
- '!ucf/**'
132146
- '!utilities/**'
133-
test_notebooks:
134-
- '**'
135-
- '!CONTRIBUTING.md'
136-
- '!README.md'
137-
- '!ci/release/update-version-cuopt.sh'
138-
- '!ci/release/update-version-rapids.sh'
139-
- '!docs/**'
140-
- '!python/nvcf_client/**'
141-
test_python:
147+
test_cpp:
142148
- '**'
143149
- '!CONTRIBUTING.md'
144150
- '!README.md'
145-
- '!ci/release/update-version-cuopt.sh'
146-
- '!ci/release/update-version-rapids.sh'
151+
- '!.ai/**'
152+
- '!.coderabbit.yaml'
153+
- '!.github/AGENTS.md'
154+
- '!.github/CODE_OF_CONDUCT.md'
155+
- '!.github/CODEOWNERS'
156+
- '!.github/ISSUE_TEMPLATE/**'
157+
- '!.github/PULL_REQUEST_TEMPLATE.md'
158+
- '!.github/SECURITY.md'
159+
- '!.github/.ai/**'
160+
- '!.github/.coderabbit_review_guide.md'
161+
- '!.github/agents/**'
162+
- '!.github/copy-pr-bot.yaml'
163+
- '!.github/ops-bot.yaml'
164+
- '!.pre-commit-config.yaml'
165+
- '!ci/build_docs.sh'
166+
- '!ci/build_python.sh'
167+
- '!ci/build_wheel*.sh'
168+
- '!ci/check_style.sh'
169+
- '!ci/release/**'
170+
- '!ci/test_python.sh'
171+
- '!ci/test_self_hosted_service.sh'
172+
- '!ci/test_wheel*.sh'
173+
- '!container-builder/**'
147174
- '!docs/**'
175+
- '!helmchart/**'
148176
- '!img/**'
177+
- '!ngc/**'
149178
- '!notebooks/**'
150-
- '!python/nvcf_client/**'
151-
test_python_cuopt:
179+
- '!python/**'
180+
- '!sonar-project.properties'
181+
- '!sonarqube/**'
182+
- '!ucf/**'
183+
- '!utilities/**'
184+
test_python_conda:
152185
- '**'
153186
- '!CONTRIBUTING.md'
154187
- '!README.md'
155-
- '!ci/release/update-version-cuopt.sh'
156-
- '!ci/release/update-version-rapids.sh'
188+
- '!.ai/**'
189+
- '!.coderabbit.yaml'
190+
- '!.github/AGENTS.md'
191+
- '!.github/CODE_OF_CONDUCT.md'
192+
- '!.github/CODEOWNERS'
193+
- '!.github/ISSUE_TEMPLATE/**'
194+
- '!.github/PULL_REQUEST_TEMPLATE.md'
195+
- '!.github/SECURITY.md'
196+
- '!.github/.ai/**'
197+
- '!.github/.coderabbit_review_guide.md'
198+
- '!.github/agents/**'
199+
- '!.github/copy-pr-bot.yaml'
200+
- '!.github/ops-bot.yaml'
201+
- '!.pre-commit-config.yaml'
202+
- '!ci/build_docs.sh'
203+
- '!ci/build_wheel*.sh'
204+
- '!ci/check_style.sh'
205+
- '!ci/release/**'
206+
- '!ci/test_self_hosted_service.sh'
207+
- '!ci/test_wheel*.sh'
208+
- '!container-builder/**'
157209
- '!docs/**'
210+
- '!helmchart/**'
158211
- '!img/**'
212+
- '!ngc/**'
159213
- '!notebooks/**'
160-
- '!python/cuopt_self_hosted/**'
161-
- '!python/cuopt_server/**'
162-
- '!python/nvcf_client/**'
163-
test_python_cuopt_server:
214+
- '!sonar-project.properties'
215+
- '!sonarqube/**'
216+
- '!ucf/**'
217+
- '!utilities/**'
218+
test_python_wheels:
164219
- '**'
165220
- '!CONTRIBUTING.md'
166221
- '!README.md'
167-
- '!ci/release/update-version-cuopt.sh'
168-
- '!ci/release/update-version-rapids.sh'
169-
- '!docs/**'
222+
- '!.ai/**'
223+
- '!.coderabbit.yaml'
224+
- '!.github/AGENTS.md'
225+
- '!.github/CODE_OF_CONDUCT.md'
226+
- '!.github/CODEOWNERS'
227+
- '!.github/ISSUE_TEMPLATE/**'
228+
- '!.github/PULL_REQUEST_TEMPLATE.md'
229+
- '!.github/SECURITY.md'
230+
- '!.github/.ai/**'
231+
- '!.github/.coderabbit_review_guide.md'
232+
- '!.github/agents/**'
233+
- '!.github/copy-pr-bot.yaml'
234+
- '!.github/ops-bot.yaml'
235+
- '!.pre-commit-config.yaml'
236+
- '!ci/build_cpp.sh'
237+
- '!ci/build_docs.sh'
238+
- '!ci/build_python.sh'
239+
- '!ci/check_style.sh'
240+
- '!ci/release/**'
241+
- '!ci/run_ctests.sh'
242+
- '!ci/test_python.sh'
243+
- '!conda/**'
244+
- '!container-builder/**'
245+
- '!helmchart/**'
170246
- '!img/**'
247+
- '!ngc/**'
171248
- '!notebooks/**'
172-
- '!python/cuopt_self_hosted/**'
173-
- '!python/nvcf_client/**'
249+
- '!sonar-project.properties'
250+
- '!sonarqube/**'
251+
- '!ucf/**'
252+
- '!utilities/**'
174253
checks:
175254
secrets: inherit
176255
uses: rapidsai/shared-workflows/.github/workflows/checks.yaml@main
@@ -187,7 +266,7 @@ jobs:
187266
conda-cpp-tests:
188267
needs: [conda-cpp-build, changed-files, compute-matrix-filters]
189268
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-tests.yaml@main
190-
#if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_cpp
269+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_cpp
191270
with:
192271
build_type: pull-request
193272
script: ci/test_cpp.sh
@@ -210,7 +289,7 @@ jobs:
210289
conda-python-tests:
211290
needs: [conda-python-build, changed-files, compute-matrix-filters]
212291
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@main
213-
#if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python
292+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_conda
214293
with:
215294
run_codecov: false
216295
build_type: pull-request
@@ -224,9 +303,10 @@ jobs:
224303
script-env-secret-3-key: CUOPT_AWS_SECRET_ACCESS_KEY
225304
script-env-secret-3-value: ${{ secrets.CUOPT_AWS_SECRET_ACCESS_KEY }}
226305
docs-build:
227-
needs: conda-python-build
306+
needs: [conda-python-build, changed-files]
228307
secrets: inherit
229308
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@main
309+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).build_docs
230310
with:
231311
build_type: pull-request
232312
node_type: "gpu-l4-latest-1"
@@ -271,7 +351,7 @@ jobs:
271351
wheel-tests-cuopt:
272352
needs: [wheel-build-cuopt, wheel-build-cuopt-mps-parser, wheel-build-cuopt-sh-client, changed-files, compute-matrix-filters]
273353
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@main
274-
#if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_cuopt
354+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_wheels
275355
with:
276356
build_type: pull-request
277357
script: ci/test_wheel_cuopt.sh
@@ -311,7 +391,7 @@ jobs:
311391
wheel-tests-cuopt-server:
312392
needs: [wheel-build-cuopt, wheel-build-cuopt-server, changed-files, compute-matrix-filters]
313393
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@main
314-
#if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_cuopt_server
394+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_wheels
315395
with:
316396
build_type: pull-request
317397
script: ci/test_wheel_cuopt_server.sh
@@ -327,7 +407,7 @@ jobs:
327407
needs: [wheel-build-cuopt, wheel-build-cuopt-server, changed-files]
328408
secrets: inherit
329409
uses: ./.github/workflows/self_hosted_service_test.yaml
330-
#if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python
410+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_wheels
331411
with:
332412
build_type: pull-request
333413
script: ci/test_self_hosted_service.sh

.pre-commit-config.yaml

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ repos:
5454
- id: shellcheck
5555
args: ["--severity=warning"]
5656
- repo: https://github.com/rapidsai/pre-commit-hooks
57-
rev: v1.2.1
57+
rev: v1.3.3
5858
hooks:
5959
- id: verify-copyright
6060
args: [--fix, --spdx]
@@ -75,6 +75,12 @@ repos:
7575
(?x)^(
7676
^cpp/tests/utilities/cxxopts.hpp
7777
)
78+
- id: verify-pyproject-license
79+
# ignore the top-level pyproject.toml, which doesn't
80+
# have or need a [project] table
81+
exclude: |
82+
(?x)
83+
^pyproject[.]toml$
7884
- repo: local
7985
hooks:
8086
- id: update-versions

ci/build_wheel.sh

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ package_dir=$2
1010

1111
source rapids-configure-sccache
1212
source rapids-date-string
13+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
14+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
1315
source rapids-init-pip
1416

1517
# Update the version to accomdate nightly and release changes for the wheel name
@@ -26,14 +28,13 @@ RAPIDS_PIP_WHEEL_ARGS=(
2628
-v
2729
--no-deps
2830
--disable-pip-version-check
29-
--extra-index-url=https://pypi.nvidia.com
3031
)
3132

3233
# Only use --build-constraint when build isolation is enabled.
3334
#
3435
# Passing '--build-constraint' and '--no-build-isolation` together results in an error from 'pip',
3536
# but we want to keep environment variable PIP_CONSTRAINT set unconditionally.
36-
# PIP_NO_BUILD_ISOLATION=0 means "add --no-build-isolation" (ref: https://github.com/pypa/pip/issues/573
37+
# PIP_NO_BUILD_ISOLATION=0 means "add --no-build-isolation" (ref: https://github.com/pypa/pip/issues/5735)
3738
if [[ "${PIP_NO_BUILD_ISOLATION:-}" != "0" ]]; then
3839
RAPIDS_PIP_WHEEL_ARGS+=(--build-constraint="${PIP_CONSTRAINT}")
3940
fi

ci/build_wheel_cuopt.sh

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@
66

77
set -euo pipefail
88

9+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
10+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
911
source rapids-init-pip
1012

1113
# Install rockylinux repo

ci/build_wheel_cuopt_mps_parser.sh

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,13 @@
11
#!/bin/bash
22

3-
# SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
# SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
44
# SPDX-License-Identifier: Apache-2.0
55

66

77
set -euo pipefail
88

9+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
10+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
911
source rapids-init-pip
1012

1113
package_dir="python/cuopt/cuopt/linear_programming"

ci/build_wheel_cuopt_server.sh

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
#!/bin/bash
22

3-
# SPDX-FileCopyrightText: Copyright (c) 2023-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
44
# SPDX-License-Identifier: Apache-2.0
55

66
set -euo pipefail
77

8+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
9+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
810
source rapids-init-pip
911

1012
package_dir="python/cuopt_server"

ci/build_wheel_cuopt_sh_client.sh

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
#!/bin/bash
22

3-
# SPDX-FileCopyrightText: Copyright (c) 2023-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
44
# SPDX-License-Identifier: Apache-2.0
55

66
set -euo pipefail
77

8+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
9+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
810
source rapids-init-pip
911

1012
package_dir="python/cuopt_self_hosted"

ci/build_wheel_libcuopt.sh

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
#!/bin/bash
2-
# SPDX-FileCopyrightText: Copyright (c) 2023-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
33
# SPDX-License-Identifier: Apache-2.0
44

55
set -euo pipefail
66

7+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
8+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
79
source rapids-init-pip
810

911
package_name="libcuopt"

ci/test_self_hosted_service.sh

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
#!/bin/bash
22

3-
# SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
# SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
44
# SPDX-License-Identifier: Apache-2.0
55

66
set -euo pipefail
77

8+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
9+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
810
source rapids-init-pip
911

1012
# Download the cuopt built in the previous step
@@ -16,7 +18,6 @@ LIBCUOPT_WHEELHOUSE=$(RAPIDS_PY_WHEEL_NAME="libcuopt_${RAPIDS_PY_CUDA_SUFFIX}" r
1618

1719
# echo to expand wildcard before adding `[extra]` requires for pip
1820
rapids-pip-retry install \
19-
--extra-index-url=https://pypi.nvidia.com \
2021
"${CUOPT_MPS_PARSER_WHEELHOUSE}"/cuopt_mps_parser*.whl \
2122
"${CUOPT_WHEELHOUSE}"/cuopt*.whl \
2223
"${LIBCUOPT_WHEELHOUSE}"/libcuopt*.whl \

ci/test_wheel_cuopt.sh

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,14 @@
11
#!/bin/bash
22

3-
# SPDX-FileCopyrightText: Copyright (c) 2023-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
44
# SPDX-License-Identifier: Apache-2.0
55

66
set -euo pipefail
77

88
# sets up a constraints file for 'pip' and puts its location in an exported variable PIP_EXPORT,
99
# so those constraints will affect all future 'pip install' calls
10+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
11+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
1012
source rapids-init-pip
1113

1214
# Download the packages built in the previous step
@@ -27,7 +29,6 @@ EOF
2729

2830
# echo to expand wildcard before adding `[extra]` requires for pip
2931
rapids-pip-retry install \
30-
--extra-index-url=https://pypi.nvidia.com \
3132
--constraint "${PIP_CONSTRAINT}" \
3233
"${CUOPT_MPS_PARSER_WHEELHOUSE}"/cuopt_mps_parser*.whl \
3334
"$(echo "${CUOPT_WHEELHOUSE}"/cuopt*.whl)[test]" \

0 commit comments

Comments
 (0)