Skip to content

Commit 3247f52

Browse files
kunal-511pboydjuliusvonkohout
authored
Tests for Model Catalog (#3262)
* Tests for Model Catalog Signed-off-by: kunal-511 <[email protected]> * fix port forward Signed-off-by: kunal-511 <[email protected]> * fix svc name Signed-off-by: kunal-511 <[email protected]> * remvove the last step Signed-off-by: kunal-511 <[email protected]> * Update tests/model_catalog_install.sh Co-authored-by: Paul Boyd <[email protected]> Signed-off-by: Kunal Dugar <[email protected]> * Update trainer_install.sh Signed-off-by: Julius von Kohout <[email protected]> * Update workflow trigger paths for model catalog tests Signed-off-by: Julius von Kohout <[email protected]> * Update trainer-controller-manager image tag to v2.0.0 Signed-off-by: Julius von Kohout <[email protected]> * Update kustomization.yaml Signed-off-by: Julius von Kohout <[email protected]> * Update kustomization.yaml Signed-off-by: Julius von Kohout <[email protected]> * Update paths in trainer_test.yaml for trainer tests Signed-off-by: Julius von Kohout <[email protected]> * Update model_catalog_test.yaml Signed-off-by: Julius von Kohout <[email protected]> * Add Model Catalog installation and API tests Signed-off-by: Julius von Kohout <[email protected]> --------- Signed-off-by: kunal-511 <[email protected]> Signed-off-by: Kunal Dugar <[email protected]> Signed-off-by: Julius von Kohout <[email protected]> Co-authored-by: Paul Boyd <[email protected]> Co-authored-by: Julius von Kohout <[email protected]>
1 parent 65245b2 commit 3247f52

File tree

8 files changed

+123
-8
lines changed

8 files changed

+123
-8
lines changed

‎.github/workflows/full_kubeflow_integration_test.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,9 @@ jobs:
109109
run: |
110110
kustomize build applications/model-registry/upstream/options/ui/overlays/istio | kubectl apply -n kubeflow -f -
111111
112+
- name: Install KF Model Catalog
113+
run: ./tests/model_catalog_install.sh
114+
112115
- name: Install Spark
113116
run: chmod u+x tests/*.sh && ./tests/spark_install.sh
114117

@@ -231,6 +234,9 @@ jobs:
231234
"localhost:8080/model-registry/api/v1/model_registry?namespace=${KF_PROFILE}" \
232235
-H "Authorization: Bearer ${KF_TOKEN}"
233236
237+
- name: Run Model Catalog API tests
238+
run: ./tests/model_catalog_test.sh
239+
234240
- name: Test Model Registry API with Unauthorized Token
235241
run: |
236242
UNAUTHORIZED_TOKEN=$(kubectl -n test-unauthorized create token test-unauthorized || kubectl -n test-unauthorized create token default)

‎.github/workflows/model_catalog_test.yaml‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
name: Test Model Catalog
2+
on:
3+
pull_request:
4+
paths:
5+
- tests/install_KinD_create_KinD_cluster_install_kustomize.sh
6+
- .github/workflows/model_catalog*
7+
- applications/model-registry/upstream/options/catalog/**
8+
- tests/istio*
9+
- common/istio*/**
10+
11+
permissions:
12+
contents: read
13+
actions: read
14+
15+
jobs:
16+
build-kfmc:
17+
runs-on: ubuntu-latest
18+
steps:
19+
- name: Checkout
20+
uses: actions/checkout@v4
21+
22+
- name: Install KinD, Create KinD cluster and Install kustomize
23+
run: ./tests/install_KinD_create_KinD_cluster_install_kustomize.sh
24+
25+
- name: Install kubectl
26+
run: ./tests/kubectl_install.sh
27+
28+
- name: Create kubeflow namespace
29+
run: kustomize build common/kubeflow-namespace/base | kubectl apply -f -
30+
31+
- name: Install Istio
32+
run: ./tests/istio-cni_install.sh
33+
34+
- name: Install oauth2-proxy
35+
run: ./tests/oauth2-proxy_install.sh
36+
37+
- name: Install cert-manager
38+
run: ./tests/cert_manager_install.sh
39+
40+
- name: Install KF Multi Tenancy
41+
run: ./tests/multi_tenancy_install.sh
42+
43+
- name: Install kubeflow-istio-resources
44+
run: kustomize build common/istio/kubeflow-istio-resources/base | kubectl apply -f -
45+
46+
- name: Create KF Profile
47+
run: kustomize build common/user-namespace/base | kubectl apply -f -
48+
49+
- name: Install KF Model Catalog
50+
run: ./tests/model_catalog_install.sh
51+
52+
- name: Run Model Catalog API tests
53+
run: ./tests/model_catalog_test.sh

‎.github/workflows/trainer_test.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ on:
55
- tests/install_KinD_create_KinD_cluster_install_kustomize.sh
66
- .github/workflows/trainer_test.yaml
77
- applications/trainer/upstream/**
8-
- tests/trainer_job.yaml
8+
- tests/trainer*
99
- tests/istio*
1010
- tests/oauth2-proxy_install.sh
1111
- common/cert-manager/**

‎applications/trainer/upstream/overlays/manager/kustomization.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ resources:
1515
# Update the Kubeflow Trainer controller manager image tag.
1616
images:
1717
- name: ghcr.io/kubeflow/trainer/trainer-controller-manager
18-
newTag: latest
18+
newTag: v2.0.1
1919

2020
# Secret for the Kubeflow Training webhook.
2121
secretGenerator:

‎applications/trainer/upstream/overlays/runtimes/kustomization.yaml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,12 @@ resources:
66
# Update the Kubeflow LLM Trainer image tag.
77
images:
88
- name: ghcr.io/kubeflow/trainer/torchtune-trainer
9-
newTag: latest
9+
newTag: v2.0.1
1010
- name: ghcr.io/kubeflow/trainer/dataset-initializer
11-
newTag: latest
11+
newTag: v2.0.1
1212
- name: ghcr.io/kubeflow/trainer/model-initializer
13-
newTag: latest
13+
newTag: v2.0.1
1414
- name: ghcr.io/kubeflow/trainer/mlx-runtime
15-
newTag: latest
15+
newTag: v2.0.1
1616
- name: ghcr.io/kubeflow/trainer/deepspeed-runtime
17-
newTag: latest
17+
newTag: v2.0.1

‎tests/model_catalog_install.sh‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/bin/bash
2+
set -euxo pipefail
3+
4+
(
5+
cd applications/model-registry/upstream/options/catalog
6+
kustomize build . | kubectl apply -n kubeflow -f -
7+
)
8+
9+
kubectl wait --for=condition=Available deployment/model-catalog-server -n kubeflow --timeout=120s

‎tests/model_catalog_test.sh‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
#!/bin/bash
2+
set -euxo pipefail
3+
4+
5+
if ! kubectl get deployment/model-catalog-server -n kubeflow; then
6+
echo "ERROR: Model Catalog deployment not found"
7+
exit 1
8+
fi
9+
10+
if ! kubectl get svc/model-catalog-model-catalog-service -n kubeflow; then
11+
echo "ERROR: Model Catalog service not found"
12+
exit 1
13+
fi
14+
15+
kubectl get pods -n kubeflow -l component=model-catalog-server
16+
17+
nohup kubectl port-forward svc/model-catalog-model-catalog-service -n kubeflow 8082:8080 &
18+
PORT_FORWARD_PID=$!
19+
20+
MAX_RETRIES=30
21+
RETRY_COUNT=0
22+
while ! curl -s localhost:8082 > /dev/null; do
23+
echo "Waiting for port-forwarding to be ready... ($RETRY_COUNT/$MAX_RETRIES)"
24+
sleep 1
25+
RETRY_COUNT=$((RETRY_COUNT + 1))
26+
if [ $RETRY_COUNT -ge $MAX_RETRIES ]; then
27+
echo "ERROR: Port-forwarding failed to become ready"
28+
kill $PORT_FORWARD_PID 2>/dev/null
29+
exit 1
30+
fi
31+
done
32+
echo "Port-forwarding ready!"
33+
34+
echo "Testing Model Catalog API..."
35+
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" "http://localhost:8082/api/model_catalog/v1alpha1/models")
36+
37+
if [ "$HTTP_CODE" -eq 200 ] || [ "$HTTP_CODE" -eq 404 ]; then
38+
echo "Model Catalog API is responding (HTTP $HTTP_CODE)"
39+
else
40+
echo "ERROR: Model Catalog API returned unexpected status code: $HTTP_CODE"
41+
kill $PORT_FORWARD_PID 2>/dev/null
42+
exit 1
43+
fi
44+
45+
kill $PORT_FORWARD_PID 2>/dev/null
46+
47+

‎tests/trainer_install.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ sleep 5
88
kubectl wait --for condition=established crd/trainjobs.trainer.kubeflow.org --timeout=60s
99

1010
kustomize build overlays | kubectl apply --server-side --force-conflicts -f -
11-
kubectl wait --for=condition=Available deployment/kubeflow-trainer-controller-manager -n kubeflow-system --timeout=180s
11+
kubectl wait --for=condition=Available deployment/kubeflow-trainer-controller-manager -n kubeflow-system --timeout=240s
1212
kubectl get crd jobsets.jobset.x-k8s.io
1313
kubectl wait --for=condition=Available deployment/jobset-controller-manager -n kubeflow-system --timeout=120s
1414

0 commit comments

Comments
 (0)