Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
6295e71
Update GitHub Actions to run Python 3.13 Tests
jrmccluskey May 27, 2025
70b1181
fix container gradle build commands
jrmccluskey May 27, 2025
1763369
bump cibuildwheel
jrmccluskey May 27, 2025
b71e869
bump distlib
jrmccluskey May 27, 2025
7028c7c
update gradle.properties
jrmccluskey May 28, 2025
98c473c
fix postcommit gradle commands
jrmccluskey May 28, 2025
40f182c
remove inference postcommit reference
jrmccluskey May 28, 2025
6dac5cb
try stepping confest var back for python postcommit
jrmccluskey May 29, 2025
1516e99
Merge branch 'master' into thirteenWorkflows
jrmccluskey Sep 30, 2025
c636c85
fix conftest, CHANGES.md
jrmccluskey Sep 30, 2025
cd0a92d
pin apitools
jrmccluskey Sep 30, 2025
d60fa62
remove py313-win env, restore ML precommit exclusions to previous ver…
jrmccluskey Sep 30, 2025
3cb78e1
update postcommit dependency workflow to use 3.13
jrmccluskey Sep 30, 2025
2663609
restore tox 313-win env
jrmccluskey Sep 30, 2025
346c936
bump pytest upper bound
jrmccluskey Sep 30, 2025
2b897ad
comment out unneccesary golden_size kwarg
jrmccluskey Oct 1, 2025
1e844d2
update base image requirements
jrmccluskey Oct 1, 2025
82e6f9b
bump cibuildwheel to 2.23.3
jrmccluskey Oct 1, 2025
e1725f2
introduce ML image requirements
jrmccluskey Oct 1, 2025
5368c91
Merge branch 'master' into thirteenWorkflows
jrmccluskey Oct 1, 2025
ee637cd
add env for py313-ml
jrmccluskey Oct 2, 2025
9308dcc
remove datatables dep in 313ml
jrmccluskey Oct 2, 2025
b494da3
Merge branch 'master' into thirteenWorkflows
jrmccluskey Oct 2, 2025
be11a3e
deleted commented out lines
jrmccluskey Oct 2, 2025
ffe239f
modify setup.py for ml dependency clarity
jrmccluskey Oct 2, 2025
c6e701b
remove errant comma
jrmccluskey Oct 2, 2025
32b26dd
Merge branch 'master' into thirteenWorkflows
jrmccluskey Oct 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/beam_PostCommit_Python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ jobs:
matrix:
job_name: ['beam_PostCommit_Python']
job_phrase: ['Run Python PostCommit']
python_version: ['3.9', '3.10', '3.11', '3.12']
python_version: ['3.9', '3.10', '3.11', '3.12', '3.13']
os: [[self-hosted, ubuntu-20.04, highmem22]]
if: |
github.event_name == 'workflow_dispatch' ||
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PostCommit_Python_Arm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
matrix:
job_name: [beam_PostCommit_Python_Arm]
job_phrase: [Run Python PostCommit Arm]
python_version: ['3.9', '3.10', '3.11', '3.12']
python_version: ['3.9', '3.10', '3.11', '3.12', '3.13']
if: |
github.event_name == 'workflow_dispatch' ||
github.event_name == 'pull_request_target' ||
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PostCommit_Python_Dependency.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ jobs:
matrix:
job_name: ['beam_PostCommit_Python_Dependency']
job_phrase: ['Run Python PostCommit Dependency']
python_version: ['3.9','3.12']
python_version: ['3.9','3.13']
timeout-minutes: 180
if: |
github.event_name == 'workflow_dispatch' ||
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ jobs:
uses: ./.github/actions/setup-environment-action
with:
java-version: default
python-version: 3.12
python-version: 3.13
- name: Run examplesPostCommit script
uses: ./.github/actions/gradle-command-self-hosted-action
with:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_Python_Examples_Direct"]
job_phrase: ["Run Python Examples_Direct"]
python_version: ['3.9','3.10','3.11','3.12']
python_version: ['3.9','3.10','3.11','3.12', '3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_Python_Examples_Flink"]
job_phrase: ["Run Python Examples_Flink"]
python_version: ['3.9', '3.12']
python_version: ['3.9', '3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_Python_Examples_Spark"]
job_phrase: ["Run Python Examples_Spark"]
python_version: ['3.9', '3.12']
python_version: ['3.9', '3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PostCommit_Python_MongoDBIO_IT.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ jobs:
uses: ./.github/actions/setup-environment-action
with:
java-version: default
python-version: 3.12
python-version: 3.13
- name: Run mongodbioIT script
uses: ./.github/actions/gradle-command-self-hosted-action
with:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_Python_ValidatesContainer_Dataflow"]
job_phrase: ["Run Python Dataflow ValidatesContainer"]
python_version: ['3.9','3.10','3.11','3.12']
python_version: ['3.9','3.10','3.11','3.12','3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_Python_ValidatesContainer_Dataflow_With_RC"]
job_phrase: ["Run Python RC Dataflow ValidatesContainer"]
python_version: ['3.9','3.10','3.11','3.12']
python_version: ['3.9','3.10','3.11','3.12','3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_Python_ValidatesRunner_Dataflow"]
job_phrase: ["Run Python Dataflow ValidatesRunner"]
python_version: ['3.9', '3.12']
python_version: ['3.9', '3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_Python_ValidatesRunner_Flink"]
job_phrase: ["Run Python Flink ValidatesRunner"]
python_version: ['3.9', '3.12']
python_version: ['3.9', '3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_Python_ValidatesRunner_Samza"]
job_phrase: ["Run Python Samza ValidatesRunner"]
python_version: ['3.9', '3.12']
python_version: ['3.9', '3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_Python_ValidatesRunner_Spark"]
job_phrase: ["Run Python Spark ValidatesRunner"]
python_version: ['3.9', '3.12']
python_version: ['3.9', '3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ jobs:
with:
python-version: |
3.9
3.12
3.13
- name: run PostCommit Python Xlang Gcp Dataflow script
uses: ./.github/actions/gradle-command-self-hosted-action
with:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ jobs:
with:
python-version: |
3.9
3.12
3.13
- name: Install docker compose
run: |
sudo curl -L https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ jobs:
with:
python-version: |
3.9
3.12
3.13
- name: run PostCommit Python Xlang IO Dataflow script
uses: ./.github/actions/gradle-command-self-hosted-action
with:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ jobs:
with:
python-version: |
3.9
3.12
3.13
- name: run PostCommit Python Xlang IO Direct script
uses: ./.github/actions/gradle-command-self-hosted-action
with:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_TransformService_Direct"]
job_phrase: ["Run TransformService_Direct PostCommit"]
python_version: ['3.9','3.12']
python_version: ['3.9','3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PostCommit_XVR_Direct.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_XVR_Direct"]
job_phrase: ["Run XVR_Direct PostCommit"]
python_version: ['3.9','3.12']
python_version: ['3.9','3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PostCommit_XVR_Flink.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_XVR_Flink"]
job_phrase: ["Run XVR_Flink PostCommit"]
python_version: ['3.9','3.12']
python_version: ['3.9','3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_XVR_JavaUsingPython_Dataflow"]
job_phrase: ["Run XVR_JavaUsingPython_Dataflow PostCommit"]
python_version: ['3.9','3.12']
python_version: ['3.9','3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ jobs:
- name: Setup environment
uses: ./.github/actions/setup-environment-action
with:
python-version: 3.12
python-version: 3.13
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests failing due to "sh: 1: python3.12: not found" after this change

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed by #36389

- name: run PostCommit XVR PythonUsingJavaSQL Dataflow script
uses: ./.github/actions/gradle-command-self-hosted-action
with:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_XVR_PythonUsingJava_Dataflow"]
job_phrase: ["Run XVR_PythonUsingJava_Dataflow PostCommit"]
python_version: ['3.9','3.12']
python_version: ['3.9','3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PostCommit_XVR_Samza.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_XVR_Samza"]
job_phrase: ["Run XVR_Samza PostCommit"]
python_version: ['3.9','3.12']
python_version: ['3.9','3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PostCommit_XVR_Spark3.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
matrix:
job_name: ["beam_PostCommit_XVR_Spark3"]
job_phrase: ["Run XVR_Spark3 PostCommit"]
python_version: ['3.9','3.12']
python_version: ['3.9','3.13']
steps:
- uses: actions/checkout@v4
- name: Setup repository
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PreCommit_Prism_Python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ jobs:
matrix:
job_name: ['beam_PreCommit_Prism_Python']
job_phrase: ['Run Prism_Python PreCommit']
python_version: ['3.9', '3.12']
python_version: ['3.9', '3.13']
if: |
github.event_name == 'push' ||
github.event_name == 'pull_request_target' ||
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PreCommit_Python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ jobs:
matrix:
job_name: ['beam_PreCommit_Python']
job_phrase: ['Run Python PreCommit']
python_version: ['3.9','3.10','3.11','3.12']
python_version: ['3.9','3.10','3.11','3.12','3.13']
if: |
github.event_name == 'push' ||
github.event_name == 'pull_request_target' ||
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PreCommit_PythonDocker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
matrix:
job_name: ["beam_PreCommit_PythonDocker"]
job_phrase: ["Run PythonDocker PreCommit"]
python_version: ['3.9','3.10','3.11','3.12']
python_version: ['3.9','3.10','3.11','3.12','3.13']
if: |
github.event_name == 'push' ||
github.event_name == 'pull_request_target' ||
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PreCommit_Python_Dataframes.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
matrix:
job_name: ['beam_PreCommit_Python_Dataframes']
job_phrase: ['Run Python_Dataframes PreCommit']
python_version: ['3.9','3.10','3.11','3.12']
python_version: ['3.9','3.10','3.11','3.12','3.13']
if: |
github.event_name == 'push' ||
github.event_name == 'pull_request_target' ||
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PreCommit_Python_Examples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:
matrix:
job_name: ['beam_PreCommit_Python_Examples']
job_phrase: ['Run Python_Examples PreCommit']
python_version: ['3.9','3.10','3.11','3.12']
python_version: ['3.9','3.10','3.11','3.12','3.13']
if: |
github.event_name == 'push' ||
github.event_name == 'pull_request_target' ||
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PreCommit_Python_Integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
matrix:
job_name: ['beam_PreCommit_Python_Integration']
job_phrase: ['Run Python_Integration PreCommit']
python_version: ['3.9', '3.12']
python_version: ['3.9', '3.13']
if: |
github.event_name == 'push' ||
github.event_name == 'pull_request_target' ||
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/beam_PreCommit_Python_ML.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,16 +65,16 @@ jobs:
matrix:
job_name: ['beam_PreCommit_Python_ML']
job_phrase: ['Run Python_ML PreCommit']
python_version: ['3.9','3.10','3.11','3.12']
python_version: ['3.9','3.10','3.11','3.12','3.13']
# Run on both self-hosted and GitHub-hosted runners.
# Some tests (marked require_docker_in_docker) can't run on Beam's
# self-hosted runners due to Docker-in-Docker environment constraint.
# These tests will only execute on ubuntu-latest (GitHub-hosted).
# Context: https://github.com/apache/beam/pull/35585.
os: [[self-hosted, ubuntu-20.04, main], [ubuntu-latest]]
exclude:
# Temporary exclude Python 3.9, 3.10, 3.11 from ubuntu-latest. This
# results in pip dependency resolution exceeded maximum depth issue.
# Temporary exclude Python 3.9, 3.10, 3.11, from ubuntu-latest.
# This results in pip dependency resolution exceeded maximum depth issue.
# Context: https://github.com/apache/beam/pull/35816.
- python_version: '3.9'
os: [ubuntu-latest]
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PreCommit_Python_PVR_Flink.yml
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ jobs:
- name: Setup environment
uses: ./.github/actions/setup-environment-action
with:
python-version: 3.12
python-version: 3.13
- name: run Python PVR Flink PreCommit script
uses: ./.github/actions/gradle-command-self-hosted-action
env:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PreCommit_Python_Runners.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
matrix:
job_name: ['beam_PreCommit_Python_Runners']
job_phrase: ['Run Python_Runners PreCommit']
python_version: ['3.9','3.10','3.11','3.12']
python_version: ['3.9','3.10','3.11','3.12','3.13']
if: |
github.event_name == 'push' ||
github.event_name == 'pull_request_target' ||
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beam_PreCommit_Python_Transforms.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:
matrix:
job_name: ['beam_PreCommit_Python_Transforms']
job_phrase: ['Run Python_Transforms PreCommit']
python_version: ['3.9','3.10','3.11','3.12']
python_version: ['3.9','3.10','3.11','3.12','3.13']
if: |
github.event_name == 'push' ||
github.event_name == 'pull_request_target' ||
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/beam_Publish_Beam_SDK_Snapshots.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,10 +67,12 @@ jobs:
- "python:container:py310:docker"
- "python:container:py311:docker"
- "python:container:py312:docker"
- "python:container:py313:docker"
- "python:container:distroless:py39:docker"
- "python:container:distroless:py310:docker"
- "python:container:distroless:py311:docker"
- "python:container:distroless:py312:docker"
- "python:container:distroless:py313:docker"
- "java:expansion-service:container:docker"
steps:
- uses: actions/checkout@v4
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ jobs:
matrix:
job_name: [beam_Python_ValidatesContainer_Dataflow_ARM]
job_phrase: [Run Python ValidatesContainer Dataflow ARM]
python_version: ['3.9','3.10','3.11','3.12']
python_version: ['3.9','3.10','3.11','3.12','3.13']
if: |
github.event_name == 'push' ||
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/build_wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ jobs:
env:
EVENT_NAME: ${{ github.event_name }}
# Keep in sync with py_version matrix value below - if changed, change that as well.
PY_VERSIONS_FULL: "cp39-* cp310-* cp311-* cp312-*"
PY_VERSIONS_FULL: "cp39-* cp310-* cp311-* cp312-* cp313-*"
outputs:
gcp-variables-set: ${{ steps.check_gcp_variables.outputs.gcp-variables-set }}
py-versions-full: ${{ steps.set-py-versions.outputs.py-versions-full }}
Expand Down Expand Up @@ -230,7 +230,7 @@ jobs:
{"os": "ubuntu-20.04", "runner": "ubuntu-22.04", "python": "${{ needs.check_env_variables.outputs.py-versions-test }}", arch: "aarch64" }
]
# Keep in sync (remove asterisks) with PY_VERSIONS_FULL env var above - if changed, change that as well.
py_version: ["cp39-", "cp310-", "cp311-", "cp312-"]
py_version: ["cp39-", "cp310-", "cp311-", "cp312-", "cp313-"]
steps:
- name: Download python source distribution from artifacts
uses: actions/download-artifact@v5
Expand All @@ -252,7 +252,7 @@ jobs:
name: Set up QEMU
- name: Install cibuildwheel
# note: sync cibuildwheel version with gradle task sdks:python:bdistPy* steps
run: pip install cibuildwheel==2.17.0 setuptools
run: pip install cibuildwheel==2.23.3 setuptools
- name: Build wheel
# Only build wheel if it is one of the target versions for this platform, otherwise no-op
if: ${{ contains(matrix.os_python.python, matrix.py_version) }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/python_dependency_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ jobs:
{"py_ver": "3.10", "py_env": "py310" },
{ "py_ver": "3.11", "py_env": "py311" },
{ "py_ver": "3.12", "py_env": "py312" },
{ "py_ver": "3.13", "py_env": "py313" },
]
steps:
- name: Checkout code
Expand Down
Loading
Loading