Only enable CI mode is $CI or $BUILD_NUMBER is set to a non-empty string

Previously, if either env variable was defined, CI mode would be activated,
even if the variable was set to the empty string.
This had some unfortunate consequences:
* The easy and obvious method of setting those variables,
  i.e. `CI="${OTHER_VAR}"`,
  would not work,
  because `$CI` would always end up being defined
  even if `$OTHER_VAR` was undefined.
* The easy and obvious method of clearing those variables at runtime,
  i.e. `CI="" pytest ...`,
  would not work,
  because again `$CI` would still be defined even though it was blank.

Now, at least one of those variables must be set to a non-empty string
in order to enabled CI mode.

Closes #13766

Author: Jordan Macdonald

AUTHORS

@@ -227,6 +227,7 @@ Jon Parise
 Jon Sonesen
 Jonas Obrist
 Jordan Guymon
+Jordan Macdonald
 Jordan Moldow
 Jordan Speicher
 Joseph Hunkeler

changelog/13766.improvement.rst

@@ -0,0 +1,4 @@
+Previously, PyTest would assume it was running in a CI/CD environment
+if either of the environment variables `$CI` or `$BUILD_NUMBER` was defined;
+the value of the defined variable didn't matter, including if the variable was set to the empty string.
+Now, CI mode is only activated if at least one of those variables is defined and set to a *non-empty* string.

doc/en/explanation/ci.rst

@@ -17,8 +17,8 @@ adapt some of its behaviours.
 How CI is detected
 ------------------
 
-Pytest knows it is in a CI environment when either one of these environment variables are set,
-regardless of their value:
+Pytest knows it is in a CI environment
+when either one of these environment variables are set to any value other than the empty string:
 
 * `CI`: used by many CI systems.
 * `BUILD_NUMBER`: used by Jenkins.

doc/en/reference/reference.rst

@@ -1165,11 +1165,11 @@ Environment variables that can be used to change pytest's behavior.
 
 .. envvar:: CI
 
-When set (regardless of value), pytest acknowledges that is running in a CI process. Alternative to ``BUILD_NUMBER`` variable. See also :ref:`ci-pipelines`.
+When set to any value other than the empty string, pytest acknowledges that is running in a CI process. Alternative to ``BUILD_NUMBER`` variable. See also :ref:`ci-pipelines`.
 
 .. envvar:: BUILD_NUMBER
 
-When set (regardless of value), pytest acknowledges that is running in a CI process. Alternative to CI variable. See also :ref:`ci-pipelines`.
+When set to any value other than the empty string, pytest acknowledges that is running in a CI process. Alternative to CI variable. See also :ref:`ci-pipelines`.
 
 .. envvar:: PYTEST_ADDOPTS
 
@@ -2408,7 +2408,7 @@ All the command-line flags can be obtained by running ``pytest --help``::
                             Plugins that must be present for pytest to run
 
     Environment variables:
-      CI                       When set (regardless of value), pytest knows it is running in a CI process and does not truncate summary info
+      CI                       When set to any value other than the empty string, pytest knows it is running in a CI process and does not truncate summary info
       BUILD_NUMBER             Equivalent to CI
       PYTEST_ADDOPTS           Extra command line options
       PYTEST_PLUGINS           Comma-separated plugins to load during startup

src/_pytest/assertion/util.py

@@ -617,5 +617,6 @@ def _notin_text(term: str, text: str, verbose: int = 0) -> list[str]:
 
 def running_on_ci() -> bool:
     """Check if we're currently running on a CI system."""
+    # Only enable CI mode if one of these env variables is defined and is a non-empty string
     env_vars = ["CI", "BUILD_NUMBER"]
-    return any(var in os.environ for var in env_vars)
+    return any(os.environ.get(var) for var in env_vars)

src/_pytest/helpconfig.py

@@ -221,7 +221,7 @@ def showhelp(config: Config) -> None:
     vars = [
         (
             "CI",
-            "When set (regardless of value), pytest knows it is running in a "
+            "When set to any value other than the empty string, pytest knows it is running in a "
             "CI process and does not truncate summary info",
         ),
         ("BUILD_NUMBER", "Equivalent to CI"),

testing/test_faulthandler.py

@@ -79,7 +79,7 @@ def test_disabled():
         pytest.param(
             True,
             marks=pytest.mark.skipif(
-                "CI" in os.environ
+                bool(os.environ.get("CI"))
                 and sys.platform == "linux"
                 and sys.version_info >= (3, 14),
                 reason="sometimes crashes on CI because of truncated outputs (#7022)",