Skip to content

Commit 279d5ee

Browse files
LuciferYangdongjoon-hyun
LuciferYang
authored and
dongjoon-hyun
committed
[SPARK-50564][PYTHON] Upgrade protobuf Python package to 5.29.1
### What changes were proposed in this pull request? This PR aims to upgrade protobuf Python package from 5.28.3 to 5.29.1. ### Why are the changes needed? To bring the latest bug fixes, such as - Fix a python bug when assign an empty Struct at creation. (protocolbuffers/protobuf#18977) (protocolbuffers/protobuf@bb0d575), this one only introduced since 5.28.x - Fix cord handling in DynamicMessage and oneofs. (protocolbuffers/protobuf@9e8b30c) - Fix packed reflection handling bug in edition 2023. (protocolbuffers/protobuf@4c92328) - Fix ContainerBase::DeepCopy to not modify the source object. (protocolbuffers/protobuf@9fa1f4f) Full release notes as follows: - https://github.com/protocolbuffers/protobuf/releases/tag/v29.0-rc1 - https://github.com/protocolbuffers/protobuf/releases/tag/v29.0-rc2 - https://github.com/protocolbuffers/protobuf/releases/tag/v29.0-rc3 - https://github.com/protocolbuffers/protobuf/releases/tag/v29.0 - https://github.com/protocolbuffers/protobuf/releases/tag/v29.1 - https://pypi.org/project/protobuf/5.29.1/ ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass Github Actions ### Was this patch authored or co-authored using generative AI tooling? No Closes #49170 from LuciferYang/protobuf-python-5.29.1. Lead-authored-by: yangjie01 <yangjie01@baidu.com> Co-authored-by: YangJie <yangjie01@baidu.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent e7a2e4b commit 279d5ee

File tree

12 files changed

+13
-13
lines changed

12 files changed

+13
-13
lines changed

‎.github/workflows/build_and_test.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -340,7 +340,7 @@ jobs:
340340
- name: Install Python packages (Python 3.11)
341341
if: (contains(matrix.modules, 'sql') && !contains(matrix.modules, 'sql-')) || contains(matrix.modules, 'connect')
342342
run: |
343-
python3.11 -m pip install 'numpy>=1.20.0' pyarrow pandas scipy unittest-xml-reporting 'lxml==4.9.4' 'grpcio==1.67.0' 'grpcio-status==1.67.0' 'protobuf==5.28.3'
343+
python3.11 -m pip install 'numpy>=1.20.0' pyarrow pandas scipy unittest-xml-reporting 'lxml==4.9.4' 'grpcio==1.67.0' 'grpcio-status==1.67.0' 'protobuf==5.29.1'
344344
python3.11 -m pip list
345345
# Run the tests.
346346
- name: Run tests
@@ -726,7 +726,7 @@ jobs:
726726
python-version: '3.11'
727727
- name: Install dependencies for Python CodeGen check
728728
run: |
729-
python3.11 -m pip install 'black==23.9.1' 'protobuf==5.28.3' 'mypy==1.8.0' 'mypy-protobuf==3.3.0'
729+
python3.11 -m pip install 'black==23.9.1' 'protobuf==5.29.1' 'mypy==1.8.0' 'mypy-protobuf==3.3.0'
730730
python3.11 -m pip list
731731
- name: Python CodeGen check for branch-3.5
732732
if: inputs.branch == 'branch-3.5'

‎.github/workflows/build_python_connect35.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ jobs:
7070
pip install 'numpy==1.25.1' 'pyarrow==12.0.1' 'pandas<=2.0.3' scipy unittest-xml-reporting plotly>=4.8 'mlflow>=2.3.1' coverage 'matplotlib==3.7.2' openpyxl 'memory-profiler==0.60.0' 'scikit-learn==1.1.*'
7171
7272
# Add Python deps for Spark Connect.
73-
pip install 'grpcio==1.67.0' 'grpcio-status==1.67.0' 'protobuf==5.28.3' 'googleapis-common-protos==1.65.0' 'graphviz==0.20.3'
73+
pip install 'grpcio==1.67.0' 'grpcio-status==1.67.0' 'protobuf==5.29.1' 'googleapis-common-protos==1.65.0' 'graphviz==0.20.3'
7474
7575
# Add torch as a testing dependency for TorchDistributor
7676
pip install 'torch==2.0.1' 'torchvision==0.15.2' torcheval

‎.github/workflows/maven_test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -178,7 +178,7 @@ jobs:
178178
- name: Install Python packages (Python 3.11)
179179
if: (contains(matrix.modules, 'sql#core')) || contains(matrix.modules, 'connect')
180180
run: |
181-
python3.11 -m pip install 'numpy>=1.20.0' pyarrow pandas scipy unittest-xml-reporting 'grpcio==1.67.0' 'grpcio-status==1.67.0' 'protobuf==5.28.3'
181+
python3.11 -m pip install 'numpy>=1.20.0' pyarrow pandas scipy unittest-xml-reporting 'grpcio==1.67.0' 'grpcio-status==1.67.0' 'protobuf==5.29.1'
182182
python3.11 -m pip list
183183
# Run the tests.
184184
- name: Run tests

‎.github/workflows/python_macos_test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,7 @@ jobs:
134134
python${{matrix.python}} -m pip install --ignore-installed 'blinker>=1.6.2'
135135
python${{matrix.python}} -m pip install --ignore-installed 'six==1.16.0'
136136
python${{matrix.python}} -m pip install numpy 'pyarrow>=15.0.0' 'six==1.16.0' 'pandas==2.2.3' scipy 'plotly>=4.8' 'mlflow>=2.8.1' coverage matplotlib openpyxl 'memory-profiler>=0.61.0' 'scikit-learn>=1.3.2' unittest-xml-reporting && \
137-
python${{matrix.python}} -m pip install 'grpcio==1.67.0' 'grpcio-status==1.67.0' 'protobuf==5.28.3' 'googleapis-common-protos==1.65.0' 'graphviz==0.20.3' && \
137+
python${{matrix.python}} -m pip install 'grpcio==1.67.0' 'grpcio-status==1.67.0' 'protobuf==5.29.1' 'googleapis-common-protos==1.65.0' 'graphviz==0.20.3' && \
138138
python${{matrix.python}} -m pip cache purge && \
139139
python${{matrix.python}} -m pip list
140140
# Run the tests.

‎dev/create-release/spark-rm/Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ RUN pypy3 -m pip install numpy 'six==1.16.0' 'pandas==2.2.3' scipy coverage matp
102102

103103
ARG BASIC_PIP_PKGS="numpy pyarrow>=15.0.0 six==1.16.0 pandas==2.2.3 scipy plotly>=4.8 mlflow>=2.8.1 coverage matplotlib openpyxl memory-profiler>=0.61.0 scikit-learn>=1.3.2 twine==3.4.1"
104104
# Python deps for Spark Connect
105-
ARG CONNECT_PIP_PKGS="grpcio==1.67.0 grpcio-status==1.67.0 protobuf==5.28.3 googleapis-common-protos==1.65.0"
105+
ARG CONNECT_PIP_PKGS="grpcio==1.67.0 grpcio-status==1.67.0 protobuf==5.29.1 googleapis-common-protos==1.65.0"
106106

107107
# Install Python 3.10 packages
108108
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10

‎dev/infra/Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ RUN pypy3 -m pip install numpy 'six==1.16.0' 'pandas==2.2.3' scipy coverage matp
9696

9797
ARG BASIC_PIP_PKGS="numpy pyarrow>=18.0.0 six==1.16.0 pandas==2.2.3 scipy plotly>=4.8 mlflow>=2.8.1 coverage matplotlib openpyxl memory-profiler>=0.61.0 scikit-learn>=1.3.2"
9898
# Python deps for Spark Connect
99-
ARG CONNECT_PIP_PKGS="grpcio==1.67.0 grpcio-status==1.67.0 protobuf==5.28.3 googleapis-common-protos==1.65.0 graphviz==0.20.3"
99+
ARG CONNECT_PIP_PKGS="grpcio==1.67.0 grpcio-status==1.67.0 protobuf==5.29.1 googleapis-common-protos==1.65.0 graphviz==0.20.3"
100100

101101
# Install Python 3.10 packages
102102
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10

‎dev/requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ py
6161
grpcio>=1.67.0
6262
grpcio-status>=1.67.0
6363
googleapis-common-protos>=1.65.0
64-
protobuf==5.28.3
64+
protobuf==5.29.1
6565

6666
# Spark Connect python proto generation plugin (optional)
6767
mypy-protobuf==3.3.0

‎dev/spark-test-image/python-309/Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ RUN apt-get update && apt-get install -y \
6969

7070
ARG BASIC_PIP_PKGS="numpy pyarrow>=18.0.0 six==1.16.0 pandas==2.2.3 scipy plotly>=4.8 mlflow>=2.8.1 coverage matplotlib openpyxl memory-profiler>=0.61.0 scikit-learn>=1.3.2"
7171
# Python deps for Spark Connect
72-
ARG CONNECT_PIP_PKGS="grpcio==1.67.0 grpcio-status==1.67.0 protobuf==5.28.3 googleapis-common-protos==1.65.0 graphviz==0.20.3"
72+
ARG CONNECT_PIP_PKGS="grpcio==1.67.0 grpcio-status==1.67.0 protobuf==5.29.1 googleapis-common-protos==1.65.0 graphviz==0.20.3"
7373

7474
# Install Python 3.9 packages
7575
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.9

‎dev/spark-test-image/python-310/Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ RUN apt-get update && apt-get install -y \
6565

6666
ARG BASIC_PIP_PKGS="numpy pyarrow>=18.0.0 six==1.16.0 pandas==2.2.3 scipy plotly>=4.8 mlflow>=2.8.1 coverage matplotlib openpyxl memory-profiler>=0.61.0 scikit-learn>=1.3.2"
6767
# Python deps for Spark Connect
68-
ARG CONNECT_PIP_PKGS="grpcio==1.67.0 grpcio-status==1.67.0 protobuf==5.28.3 googleapis-common-protos==1.65.0 graphviz==0.20.3"
68+
ARG CONNECT_PIP_PKGS="grpcio==1.67.0 grpcio-status==1.67.0 protobuf==5.29.1 googleapis-common-protos==1.65.0 graphviz==0.20.3"
6969

7070
# Install Python 3.10 packages
7171
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10

‎dev/spark-test-image/python-311/Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ RUN apt-get update && apt-get install -y \
6969

7070
ARG BASIC_PIP_PKGS="numpy pyarrow>=18.0.0 six==1.16.0 pandas==2.2.3 scipy plotly>=4.8 mlflow>=2.8.1 coverage matplotlib openpyxl memory-profiler>=0.61.0 scikit-learn>=1.3.2"
7171
# Python deps for Spark Connect
72-
ARG CONNECT_PIP_PKGS="grpcio==1.67.0 grpcio-status==1.67.0 protobuf==5.28.3 googleapis-common-protos==1.65.0 graphviz==0.20.3"
72+
ARG CONNECT_PIP_PKGS="grpcio==1.67.0 grpcio-status==1.67.0 protobuf==5.29.1 googleapis-common-protos==1.65.0 graphviz==0.20.3"
7373

7474
# Install Python 3.11 packages
7575
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.11

0 commit comments

Comments
 (0)