setting pytest logging level at queue time (#33760)

* try setting pytest logging level at queue time

* add small note in docs

Author: kristapratico

doc/eng_sys_checks.md

@@ -7,6 +7,7 @@
   - [The pyproject.toml](#the-pyprojecttoml)
   - [Environment variables important to CI](#environment-variables-important-to-ci)
     - [Atomic Overrides](#atomic-overrides)
+    - [Enable test logging in CI pipelines](#enable-test-logging-in-ci-pipelines)
   - [Analyze Checks](#analyze-checks)
     - [MyPy](#mypy)
     - [Pyright](#pyright)
@@ -167,6 +168,16 @@ The name that you should use is visible based on what the `tox environment` that
 - `AZURE_SERVICEBUS_PYRIGHT=true` <-- enable a check that normally is disabled in `pyproject.toml`
 - `AZURE_CORE_PYLINT=false` <-- disable a check that normally runs
 
+### Enable test logging in CI pipelines
+
+You can enable test logging in a pipeline by setting the queue time variable `PYTEST_LOG_LEVEL` to the desired logging [level](https://docs.python.org/3/library/logging.html#logging-levels). For example,
+
+`PYTEST_LOG_LEVEL=INFO`
+
+This also works locally with tox by setting the `PYTEST_LOG_LEVEL` environment variable. 
+
+Note that if you want DEBUG level logging with sensitive information unredacted in the test logs, then you still must pass `logging_enable=True` into the client(s) being used in tests.
+
 ## Analyze Checks
 
 Analyze job in both nightly CI and pull request validation pipeline runs a set of static analysis using external and internal tools. Following are the list of these static analysis.

eng/tox/tox.ini

@@ -48,7 +48,7 @@ pkgs =
 
 [pytest]
 ignore_args=--ignore=.tox --ignore=build --ignore=.eggs
-default_args = -rsfE --junitxml={tox_root}/test-junit-{envname}.xml --verbose --cov-branch --durations=10 --ignore=azure {[pytest]ignore_args}
+default_args = -rsfE --junitxml={tox_root}/test-junit-{envname}.xml --verbose --cov-branch --durations=10 --ignore=azure {[pytest]ignore_args} --log-cli-level={pytest_log_level}
 
 [testenv]
 parallel_show_output =True

eng/tox/toxfile.py

@@ -1,3 +1,4 @@
+import os
 from logging import getLogger
 from pathlib import Path
 
@@ -17,3 +18,8 @@ def tox_add_core_config(core_conf: CoreConfigSet, state: State):
         next(p for p in Path(core_conf["config_file_path"]).resolve().parents if (p / ".git").exists()),
     )
 
+    core_conf.add_constant(
+        "pytest_log_level",
+        "The log level to use for pytest, supplied as environment or queue time variable PYTEST_LOG_LEVEL",
+        os.getenv("PYTEST_LOG_LEVEL", "51")  # Defaults to no logging
+    )