Skip to content

chore: update py image refs to 3.11 #12383

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 27, 2025
+1,945 −1,930
Merged

chore: update py image refs to 3.11 #12383

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5ed16be
chore: update py image refs to 3.11
zazulam Oct 21, 2025
956631c
chore: address sdk test failures
zazulam Oct 21, 2025
7f4e0a6
chore: regen & update stubs for k8s and backend tests
zazulam Oct 23, 2025
afccad8
chore: add regen proto_test files
zazulam Oct 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
9 changes: 9 additions & 0 deletions .github/actions/kfp-k8s/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,12 @@ runs:
if: ${{ steps.generate-kfp-kubernetes-proto-files.outcome == 'success' }}
run: |
pip install -e ./kubernetes_platform/python[dev] --find-links=sdk/python/dist
# testing reinstalling kfp package from source with no deps
- name: Reinstall kfp from source with no deps
id: reinstall-kfp
shell: bash
if: ${{ steps.install-kfp-kubernetes.outcome == 'success' }}
working-directory: ./sdk/python
run: |
pip install . -I --no-deps
Comment on lines +48 to +56
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this since the previous section was not actually using the source install for the kfp sdk, see here for the error prior to this add.

2 changes: 1 addition & 1 deletion backend/metadata_writer/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# ml-metadata package depends on tensorflow package
FROM python:3.9
FROM python:3.11
COPY backend/metadata_writer/requirements.txt /kfp/metadata_writer/
RUN python3 -m pip install -r /kfp/metadata_writer/requirements.txt

Expand Down
2 changes: 1 addition & 1 deletion backend/metadata_writer/update_requirements.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash

# This image should be in sync with Dockerfile.
IMAGE="python:3.9"
IMAGE="python:3.11"
../../hack/update-requirements.sh $IMAGE <requirements.in >requirements.txt
2 changes: 1 addition & 1 deletion backend/src/apiserver/config/testdata/sample_pipeline.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef hello_world():\n print('hello')\n\n"
image: python:3.9
image: python:3.11
pipelineInfo:
name: pipeline-hello-world
root:
Expand Down
12 changes: 6 additions & 6 deletions backend/src/apiserver/resource/resource_manager_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3479,15 +3479,15 @@ spec:
- name: ENABLE_CACHING
valueFrom:
fieldRef: {fieldPath: 'metadata.labels[''pipelines.kubeflow.org/enable_caching'']'}
- {name: KFP_V2_IMAGE, value: 'python:3.9'}
- {name: KFP_V2_IMAGE, value: 'python:3.11'}
- {name: KFP_V2_RUNTIME_INFO, value: '{"inputParameters": {"some_int": {"type":
"INT"}, "uri": {"type": "STRING"}}, "inputArtifacts": {}, "outputParameters":
{"output_parameter_one": {"type": "INT", "path": "/tmp/outputs/output_parameter_one/data"}},
"outputArtifacts": {"output_dataset_one": {"schemaTitle": "system.Dataset",
"instanceSchema": "", "metadataPath": "/tmp/outputs/output_dataset_one/data"}}}'}
envFrom:
- configMapRef: {name: metadata-grpc-configmap, optional: true}
image: python:3.9
image: python:3.11
volumeMounts:
- {mountPath: /kfp-launcher, name: kfp-launcher}
inputs:
Expand Down Expand Up @@ -3581,15 +3581,15 @@ spec:
- name: ENABLE_CACHING
valueFrom:
fieldRef: {fieldPath: 'metadata.labels[''pipelines.kubeflow.org/enable_caching'']'}
- {name: KFP_V2_IMAGE, value: 'python:3.9'}
- {name: KFP_V2_IMAGE, value: 'python:3.11'}
- {name: KFP_V2_RUNTIME_INFO, value: '{"inputParameters": {"num_steps": {"type":
"INT"}}, "inputArtifacts": {"dataset": {"metadataPath": "/tmp/inputs/dataset/data",
"schemaTitle": "system.Dataset", "instanceSchema": ""}}, "outputParameters":
{}, "outputArtifacts": {"model": {"schemaTitle": "system.Model", "instanceSchema":
"", "metadataPath": "/tmp/outputs/model/data"}}}'}
envFrom:
- configMapRef: {name: metadata-grpc-configmap, optional: true}
image: python:3.9
image: python:3.11
volumeMounts:
- {mountPath: /kfp-launcher, name: kfp-launcher}
inputs:
Expand Down Expand Up @@ -4113,7 +4113,7 @@ deploymentSpec:
_parsed_args = vars(_parser.parse_args())

_outputs = hello_world(**_parsed_args)
image: python:3.9
image: python:3.11
pipelineInfo:
name: hello-world
root:
Expand Down Expand Up @@ -4146,7 +4146,7 @@ deploymentSpec:
executors:
exec-hello-world:
container:
image: python:3.9
image: python:3.11
pipelineInfo:
name: pipelines/p1/versions/v1
root:
Expand Down
32 changes: 16 additions & 16 deletions backend/src/apiserver/server/pipeline_upload_server_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,24 +47,24 @@ const (
func TestUploadPipeline(t *testing.T) {
// TODO(v2): when we add a field to distinguish between v1 and v2 template, verify it's in the response
tt := []struct {
name string
spec []byte
name string
spec []byte
apiVersion string
}{{
name: "upload argo workflow YAML",
spec: []byte("apiVersion: argoproj.io/v1alpha1\nkind: Workflow"),
name: "upload argo workflow YAML",
spec: []byte("apiVersion: argoproj.io/v1alpha1\nkind: Workflow"),
apiVersion: "v1beta1",
}, {
name: "upload argo workflow YAML",
spec: []byte("apiVersion: argoproj.io/v1alpha1\nkind: Workflow"),
name: "upload argo workflow YAML",
spec: []byte("apiVersion: argoproj.io/v1alpha1\nkind: Workflow"),
apiVersion: "v2beta1",
}, {
name: "upload pipeline v2 job in proto yaml",
spec: []byte(v2SpecHelloWorld),
name: "upload pipeline v2 job in proto yaml",
spec: []byte(v2SpecHelloWorld),
apiVersion: "v1beta1",
}, {
name: "upload pipeline v2 job in proto yaml",
spec: []byte(v2SpecHelloWorld),
name: "upload pipeline v2 job in proto yaml",
spec: []byte(v2SpecHelloWorld),
apiVersion: "v2beta1",
}}
for _, test := range tt {
Expand Down Expand Up @@ -732,7 +732,7 @@ deploymentSpec:
_parsed_args = vars(_parser.parse_args())

_outputs = hello_world(**_parsed_args)
image: python:3.9
image: python:3.11
pipelineInfo:
name: hello-world
root:
Expand Down Expand Up @@ -789,7 +789,7 @@ deploymentSpec:
_parsed_args = vars(_parser.parse_args())

_outputs = hello_world(**_parsed_args)
image: python:3.9
image: python:3.11
pipelineInfo:
name: hello-world
root:
Expand Down Expand Up @@ -830,7 +830,7 @@ deploymentSpec:
executors:
exec-hello-world:
container:
image: python:3.9
image: python:3.11
pipelineInfo:
name: hello-world-
root:
Expand All @@ -855,7 +855,7 @@ deploymentSpec:
executors:
exec-hello-world:
container:
image: python:3.9
image: python:3.11
pipelineInfo:
name: hEllo-world
root:
Expand All @@ -880,7 +880,7 @@ deploymentSpec:
executors:
exec-hello-world:
container:
image: python:3.9
image: python:3.11
pipelineInfo:
name: more than 128 characters more than 128 characters more than 128 characters more than 128 characters more than 128 characters
root:
Expand All @@ -905,7 +905,7 @@ deploymentSpec:
executors:
exec-hello-world:
container:
image: python:3.9
image: python:3.11
pipelineInfo:
name: hello-worl.d
root:
Expand Down
6 changes: 3 additions & 3 deletions backend/src/apiserver/server/test/pipeline_with_volume.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.9
image: python:3.11
exec-comp-2:
container:
args:
Expand All @@ -92,7 +92,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.9
image: python:3.11
exec-comp-3:
container:
args:
Expand All @@ -118,7 +118,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.9
image: python:3.11
exec-createpvc:
container:
image: argostub/createpvc
Expand Down
2 changes: 1 addition & 1 deletion backend/src/apiserver/server/test/v2-hello-world.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
"program_path=$(mktemp)\nprintf \"%s\" \"$0\" > \"$program_path\"\npython3 -u \"$program_path\" \"$@\"\n",
"def hello_world(text):\n print(text)\n return text\n\nimport argparse\n_parser = argparse.ArgumentParser(prog='Hello world', description='')\n_parser.add_argument(\"--text\", dest=\"text\", type=str, required=True, default=argparse.SUPPRESS)\n_parsed_args = vars(_parser.parse_args())\n\n_outputs = hello_world(**_parsed_args)\n"
],
"image": "python:3.9"
"image": "python:3.11"
}
}
}
Expand Down
14 changes: 7 additions & 7 deletions backend/src/apiserver/server/test/xgboost_sample_pipeline.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,7 @@ deploymentSpec:
\ try:\n os.makedirs(os.path.dirname(output_file))\n except\
\ OSError:\n pass\n with open(output_file, 'w') as f:\n \
\ f.write(_output_serializers[idx](_outputs[idx]))\n"
image: python:3.9
image: python:3.11
exec-xgboost-predict:
container:
args:
Expand Down Expand Up @@ -344,7 +344,7 @@ deploymentSpec:
_parser.add_argument(\"--predictions\", dest=\"predictions_path\", type=_make_parent_dirs_and_return_path,\
\ required=True, default=argparse.SUPPRESS)\n_parsed_args = vars(_parser.parse_args())\n\
\n_outputs = xgboost_predict(**_parsed_args)\n"
image: python:3.9
image: python:3.11
exec-xgboost-predict-2:
container:
args:
Expand Down Expand Up @@ -398,7 +398,7 @@ deploymentSpec:
predictions_path\", type=_make_parent_dirs_and_return_path, required=True,\
\ default=argparse.SUPPRESS)\n_parsed_args = vars(_parser.parse_args())\n\
\n_outputs = xgboost_predict(**_parsed_args)\n"
image: python:3.9
image: python:3.11
exec-xgboost-predict-3:
container:
args:
Expand Down Expand Up @@ -452,7 +452,7 @@ deploymentSpec:
predictions_path\", type=_make_parent_dirs_and_return_path, required=True,\
\ default=argparse.SUPPRESS)\n_parsed_args = vars(_parser.parse_args())\n\
\n_outputs = xgboost_predict(**_parsed_args)\n"
image: python:3.9
image: python:3.11
exec-xgboost-predict-4:
container:
args:
Expand Down Expand Up @@ -503,7 +503,7 @@ deploymentSpec:
_parser.add_argument(\"--predictions\", dest=\"predictions_path\", type=_make_parent_dirs_and_return_path,\
\ required=True, default=argparse.SUPPRESS)\n_parsed_args = vars(_parser.parse_args())\n\
\n_outputs = xgboost_predict(**_parsed_args)\n"
image: python:3.9
image: python:3.11
exec-xgboost-train:
container:
args:
Expand Down Expand Up @@ -620,7 +620,7 @@ deploymentSpec:
, dest=\"model_config_path\", type=_make_parent_dirs_and_return_path, required=True,\
\ default=argparse.SUPPRESS)\n_parsed_args = vars(_parser.parse_args())\n\
\n_outputs = xgboost_train(**_parsed_args)\n"
image: python:3.9
image: python:3.11
exec-xgboost-train-2:
container:
args:
Expand Down Expand Up @@ -737,7 +737,7 @@ deploymentSpec:
, dest=\"model_config_path\", type=_make_parent_dirs_and_return_path, required=True,\
\ default=argparse.SUPPRESS)\n_parsed_args = vars(_parser.parse_args())\n\
\n_outputs = xgboost_train(**_parsed_args)\n"
image: python:3.9
image: python:3.11
pipelineInfo:
name: xgboost-sample-pipeline
root:
Expand Down
2 changes: 1 addition & 1 deletion backend/src/apiserver/storage/pipeline_store_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1936,7 +1936,7 @@ executors:
_parsed_args = vars(_parser.parse_args())

_outputs = hello_world(**_parsed_args)
image: python:3.9
image: python:3.11
pipelineInfo:
name: hello-world
root:
Expand Down
2 changes: 1 addition & 1 deletion backend/src/apiserver/template/testdata/hello_world.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ deploymentSpec:
_parsed_args = vars(_parser.parse_args())

_outputs = hello_world(**_parsed_args)
image: python:3.9
image: python:3.11
pipelineInfo:
name: namespace/n1/pipeline/hello-world
root:
Expand Down
2 changes: 1 addition & 1 deletion backend/src/apiserver/template/testdata/hello_world_schema_2_0_0.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ deploymentSpec:
_parsed_args = vars(_parser.parse_args())

_outputs = hello_world(**_parsed_args)
image: python:3.9
image: python:3.11
pipelineInfo:
name: hello-world
root:
Expand Down
6 changes: 3 additions & 3 deletions backend/src/apiserver/template/testdata/pipeline_with_volume.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.9
image: python:3.11
exec-comp-2:
container:
args:
Expand All @@ -94,7 +94,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.9
image: python:3.11
exec-comp-3:
container:
args:
Expand All @@ -120,7 +120,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.9
image: python:3.11
exec-createpvc:
container:
image: argostub/createpvc
Expand Down
Loading
Loading