Skip to content

Commit 86bc452

Browse files
authored
fixed issue in python precommit (#36561)
* fixed issue in python precommit * fixes * resolved comments * removed unused vars * removed unused imports
1 parent 1b34a6d commit 86bc452

File tree

3 files changed

+14
-5
lines changed

3 files changed

+14
-5
lines changed

contributor-docs/python-tips.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -265,7 +265,17 @@ Execute the following code for running tests using tox:
265265

266266
### Running Tests Using gradle
267267

268-
Integration tests suites on Jenkins are configured in groovy files that launch certain gradle tasks ([example](https://github.com/apache/beam/blob/0fd6a044df5b9f26d567e0f9a619a665a0f4043b/.test-infra/jenkins/job_PostCommit_Python.groovy#L43)). You could launch test suites locally by executing the gradle targets directly (for example: `./gradlew :sdks:python:test-suites:dataflow:py39:postCommitPy39`). This option may only be available to committers, as by default the test suites are configured to use the [`apache-beam-testing`](https://github.com/apache/beam/blob/0fd6a044df5b9f26d567e0f9a619a665a0f4043b/sdks/python/scripts/run_integration_test.sh#L70) project.
268+
Integration tests suites on Jenkins are configured in groovy files that launch certain gradle tasks ([example](https://github.com/apache/beam/blob/0fd6a044df5b9f26d567e0f9a619a665a0f4043b/.test-infra/jenkins/job_PostCommit_Python.groovy#L43)). You could launch test suites locally by executing the gradle targets directly (for example: `./gradlew :sdks:python:test-suites:dataflow:py39:postCommitPy39`). This option may only be available to committers, as by default the test suites are configured to use the [`apache-beam-testing`](https://github.com/apache/beam/blob/0fd6a044df5b9f26d567e0f9a619a665a0f4043b/sdks/python/scripts/run_integration_test.sh#L70) project.
269+
270+
### Environment Variables for Test Stability
271+
272+
The following environment variables can be used to improve test stability in CI environments:
273+
274+
**Test execution settings:**
275+
- `PYTEST_XDIST_WORKER_COUNT=1` - Force sequential test execution
276+
- `PYTHONHASHSEED=0` - Ensure deterministic hash behavior
277+
- `OMP_NUM_THREADS=1` - Limit OpenMP threads
278+
- `OPENBLAS_NUM_THREADS=1` - Limit OpenBLAS threads
269279

270280
To run only a subset of tests using this approach, you could adjust the test label in the test (such as [it_postcommit](https://github.com/apache/beam/blob/25e6008e8919c2f31eaebae2662b44e02f9f37a1/sdks/python/apache_beam/io/gcp/pubsub_integration_test.py#L211)) and the [selector](https://github.com/apache/beam/blob/25e6008e8919c2f31eaebae2662b44e02f9f37a1/sdks/python/test-suites/dataflow/common.gradle#L117) where the test suite is defined.
271281

sdks/python/scripts/run_pytest.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -152,4 +152,4 @@ if [[ $status1 != 0 && $status1 != 5 ]]; then
152152
fi
153153
if [[ $status2 != 0 && $status2 != 5 ]]; then
154154
exit $status2
155-
fi
155+
fi

sdks/python/test-suites/tox/common.gradle

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,9 @@ test.dependsOn "testPy${pythonVersionSuffix}Dill"
3434

3535
// toxTask "testPy${pythonVersionSuffix}Dask", "py${pythonVersionSuffix}-dask", "${posargs}"
3636
// test.dependsOn "testPy${pythonVersionSuffix}Dask"
37+
// Since codecoverage reports will always be generated for py39,
38+
// all tests will be exercised.
3739
project.tasks.register("preCommitPy${pythonVersionSuffix}") {
38-
// Since codecoverage reports will always be generated for py39,
39-
// all tests will be exercised.
40-
// dependsOn = ["testPy${pythonVersionSuffix}Cloud", "testPython${pythonVersionSuffix}"]
4140
dependsOn = ["testPy${pythonVersionSuffix}Cloud", "testPython${pythonVersionSuffix}"]
4241

4342
}

0 commit comments

Comments
 (0)