chore: enable build speed-ups by default (#14128)

We make the CMake incremental build feature and the artifact download
caching on by default.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Author: P403n1x87

.gitlab/templates/cached-testrunner.yml

@@ -3,8 +3,6 @@
   variables:
     PIP_CACHE_DIR: '${{CI_PROJECT_DIR}}/.cache/pip'
     SCCACHE_DIR: '${{CI_PROJECT_DIR}}/.cache/sccache'
-    DD_CMAKE_INCREMENTAL_BUILD: '1'
-    DD_SETUP_CACHE_DOWNLOADS: '1'
     EXT_CACHE_VENV: '${{CI_PROJECT_DIR}}/.cache/ext_cache_venv'
   before_script: |
     ulimit -c unlimited

docker-compose.yml

@@ -177,8 +177,6 @@ services:
         image: ghcr.io/datadog/dd-trace-py/testrunner:8b69a2610342b333f8832422ffc4f3a9327bed13@sha256:c2d067947ffdb305fc7dc7ff1f8eb7035cfa110bd1199917dd2519eadd166402
         command: bash
         environment:
-          DD_SETUP_CACHE_DOWNLOADS: "1"
-          DD_CMAKE_INCREMENTAL_BUILD: "1"
           DD_FAST_BUILD: "1"
           CMAKE_BUILD_PARALLEL_LEVEL: "12"
           CARGO_BUILD_JOBS: "12"

docs/build_system.rst

@@ -204,3 +204,23 @@ These environment variables modify aspects of the build process.
 
     version_added:
         v2.16.0:
+
+  DD_CMAKE_INCREMENTAL_BUILD:
+    type: Boolean
+    default: True
+
+    description: |
+        Enables support for incremental builds of CMake extensions when doing an in-place install (e.g. editable mode).
+
+    version_added:
+        v3.10.0:
+
+  DD_SETUP_CACHE_DOWNLOADS:
+    type: Boolean
+    default: True
+
+    description: |
+        Caches the download of artifacts needed by the build process.
+
+    version_added:
+        v3.10.0:

scripts/perf-run-scenario

@@ -64,8 +64,6 @@ if [[ -n ${ARTIFACTS} ]]; then
 fi
 CMD+=" -v $(pwd):/src/ \
   -e PROFILE_BENCHMARKS=${PROFILE_BENCHMARKS:-0} \
-  -e DD_SETUP_CACHE_DOWNLOADS=1 \
-  -e DD_CMAKE_INCREMENTAL_BUILD=1 \
   -e DD_FAST_BUILD=1 \
   -e CMAKE_BUILD_PARALLEL_LEVEL=12 \
   -e CARGO_BUILD_JOBS=12 \

setup.py

@@ -226,7 +226,7 @@ def run(self):
 
 class LibraryDownload:
     CACHE_DIR = HERE / ".download_cache"
-    USE_CACHE = os.getenv("DD_SETUP_CACHE_DOWNLOADS", "0").lower() in ("1", "yes", "on", "true")
+    USE_CACHE = os.getenv("DD_SETUP_CACHE_DOWNLOADS", "1").lower() in ("1", "yes", "on", "true")
 
     name = None
     download_dir = Path.cwd()
@@ -392,7 +392,7 @@ def run(self):
 
 
 class CMakeBuild(build_ext):
-    INCREMENTAL = os.getenv("DD_CMAKE_INCREMENTAL_BUILD", "0").lower() in ("1", "yes", "on", "true")
+    INCREMENTAL = os.getenv("DD_CMAKE_INCREMENTAL_BUILD", "1").lower() in ("1", "yes", "on", "true")
 
     @staticmethod
     def try_strip_symbols(so_file):