Update GitHub Actions to run Python 3.13 Tests #35056

jrmccluskey · 2025-05-27T18:23:53Z

Updates workflows to publish 3.13 containers and execute test suites against Python 3.13.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

codecov · 2025-05-29T18:34:43Z

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 40.20%. Comparing base (3534960) to head (32b26dd).
⚠️ Report is 5 commits behind head on master.

Files with missing lines	Patch %	Lines
sdks/python/setup.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #35056      +/-   ##
============================================
- Coverage     40.20%   40.20%   -0.01%     
  Complexity     3386     3386              
============================================
  Files          1220     1220              
  Lines        186149   186175      +26     
  Branches       3523     3523              
============================================
+ Hits          74839    74844       +5     
- Misses       107955   107976      +21     
  Partials       3355     3355

Flag	Coverage Δ
python	`40.54% <0.00%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2025-07-29T12:47:07Z

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

github-actions · 2025-09-29T12:46:47Z

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

…sion

jrmccluskey · 2025-10-02T17:21:53Z

The only current hangup is that I cannot get a clean precommit run off of the branch, but the test failures aren't consistent so it's hard to tell if we're really getting breakages or just flakes.

damccorm

Thanks - just had a few minor comments

sdks/python/apache_beam/runners/interactive/testing/integration/tests/screen_diff_test.py

damccorm · 2025-10-02T17:34:30Z

sdks/python/test-suites/gradle.properties

 # (TODO): https://github.com/apache/beam/issues/21971
 # Add python 3.10 to dataflow test-suites
-dataflow_precommit_it_task_py_versions=3.9,3.12
+dataflow_precommit_it_task_py_versions=3.9,3.13


Could be a follow up (non-blocking here) - it would be nice to refactor 3.9 and 3.13 into max_py_version and min_py_version variables

damccorm · 2025-10-02T17:38:32Z

sdks/python/setup.py

              'torch',
              'transformers',
          ],
+          'p313_ml_test': [


Rather than adding a p3XX_ml_test version for every supported python version, could we rename p312_ml_test to something like ml_basic_test? I think ml_test is just this + dill and tensorflow-transform FWIW

I know this is technically breaking, but I would be surprised if anyone is depending on this directly (and we could call this out in CHANGES)

Kind of depends on how we want to go about it, since the datatables dependency does work in 3.12 but does not support 3.13. I'm not sure how many test cases that potentially takes off of 3.12 runs, but I am not opposed to it.

Ok, what if we do something in the middle? We could define a ml_base dependency like we do with some other deps (e.g.

beam/sdks/python/setup.py

Line 159 in be11a3e

dataframe_dependency = [

)

Then we can have:

'ml_test': [ 'datatable', 'dill', 'tensorflow-transformers' ] + ml_base, 'p312_ml_test': [ 'datatable' ] + ml_base, 'p313_ml_test': ml_base

Thoughts?

took a run at it in the latest version

jrmccluskey · 2025-10-02T18:31:08Z

Clean precommit run - https://github.com/apache/beam/actions/runs/18200511639/job/51817950160

damccorm

LGTM once checks pass

github-actions · 2025-10-02T20:37:05Z

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

jrmccluskey · 2025-10-02T20:45:57Z

apache_beam.ml.rag.enrichment.milvus_search_it_test.TestMilvusSearchEnrichment seems to be consistently red on python 3.12.

damccorm · 2025-10-03T15:22:22Z

Remaining test failures look like #36377

jrmccluskey · 2025-10-03T15:25:13Z

sounds good, thank you

Abacn · 2025-10-04T21:46:42Z

.github/workflows/beam_PostCommit_XVR_PythonUsingJavaSQL_Dataflow.yml

        uses: ./.github/actions/setup-environment-action
        with:
-          python-version: 3.12
+          python-version: 3.13


tests failing due to "sh: 1: python3.12: not found" after this change

fixed by #36389

Update GitHub Actions to run Python 3.13 Tests

6295e71

github-actions bot added the build label May 27, 2025

fix container gradle build commands

70b1181

github-actions bot added python docker labels May 27, 2025

jrmccluskey added 6 commits May 27, 2025 14:34

bump cibuildwheel

1763369

bump distlib

b71e869

update gradle.properties

7028c7c

fix postcommit gradle commands

98c473c

remove inference postcommit reference

40f182c

try stepping confest var back for python postcommit

6dac5cb

github-actions bot added stale and removed stale labels Jul 29, 2025

github-actions bot added stale and removed stale labels Sep 29, 2025

Merge branch 'master' into thirteenWorkflows

1516e99

github-actions bot removed the docker label Sep 30, 2025

jrmccluskey added 7 commits September 30, 2025 18:26

fix conftest, CHANGES.md

c636c85

pin apitools

cd0a92d

remove py313-win env, restore ML precommit exclusions to previous ver…

d60fa62

…sion

update postcommit dependency workflow to use 3.13

3cb78e1

restore tox 313-win env

2663609

bump pytest upper bound

346c936

comment out unneccesary golden_size kwarg

2b897ad

github-actions bot added the runners label Oct 1, 2025

update base image requirements

1e844d2

github-actions bot added the docker label Oct 1, 2025

jrmccluskey and others added 6 commits October 1, 2025 17:39

bump cibuildwheel to 2.23.3

82e6f9b

introduce ML image requirements

e1725f2

Merge branch 'master' into thirteenWorkflows

5368c91

add env for py313-ml

ee637cd

remove datatables dep in 313ml

9308dcc

Merge branch 'master' into thirteenWorkflows

b494da3

jrmccluskey marked this pull request as ready for review October 2, 2025 17:20

jrmccluskey requested review from damccorm and tvalentyn October 2, 2025 17:30

damccorm reviewed Oct 2, 2025

View reviewed changes

deleted commented out lines

be11a3e

tvalentyn approved these changes Oct 2, 2025

View reviewed changes

jrmccluskey added 2 commits October 2, 2025 18:44

modify setup.py for ml dependency clarity

ffe239f

remove errant comma

c6e701b

damccorm approved these changes Oct 2, 2025

View reviewed changes

Merge branch 'master' into thirteenWorkflows

32b26dd

jrmccluskey merged commit a03e96a into master Oct 3, 2025
108 of 125 checks passed

liferoad mentioned this pull request Oct 4, 2025

ci: update Python version from 3.12 to 3.13 in Flink pre-commit workflow #36389

Merged

3 tasks

Abacn reviewed Oct 4, 2025

View reviewed changes

tarun-google mentioned this pull request Oct 6, 2025

Update Dev Image tag to latest #36411

Merged

This was referenced Oct 14, 2025

The PostCommit Python Dependency job is flaky #30799

Closed

The PostCommit TransformService Direct job is flaky #30960

Closed

shunping mentioned this pull request Dec 17, 2025

Remove 3.9 from test workflow and add 3.13. apache/beam-starter-python#54

Merged

Update GitHub Actions to run Python 3.13 Tests #35056

Update GitHub Actions to run Python 3.13 Tests #35056

Uh oh!

Conversation

jrmccluskey commented May 27, 2025

GitHub Actions Tests Status (on master branch)

Uh oh!

codecov bot commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Jul 29, 2025

Uh oh!

github-actions bot commented Sep 29, 2025

Uh oh!

jrmccluskey commented Oct 2, 2025

Uh oh!

damccorm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jrmccluskey commented Oct 2, 2025

Uh oh!

damccorm left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 2, 2025

Uh oh!

jrmccluskey commented Oct 2, 2025

Uh oh!

damccorm commented Oct 3, 2025

Uh oh!

jrmccluskey commented Oct 3, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov bot commented May 29, 2025 •

edited

Loading