Skip to content

Commit 9b0b540

Browse files
mritunjaysharma394mritunjaysharma394
authored
feat[packages]: add kubeflow controllers and components (#58003)
<!--- Provide a short summary in the Title above. Examples of good PR titles: * "ruby-3.1: new package" * "haproxy: fix CVE-2014-123456" --> <!-- Please include references to any related issues or delete this section otherwise. --> Fixes: Related: ### Pre-review Checklist <!-- This checklist is mostly useful as a reminder of small things that can easily be forgotten – it is meant as a helpful tool rather than hoops to jump through. At the moment of this PR you have the most information on what all the change will affect, so please take the time to jot it down. Put an `x` in all the items that apply, make notes next to any that haven't been addressed, and remove any items that are not relevant to this PR. --> #### For new package PRs only <!-- remove if unrelated --> - [ ] This PR is marked as fixing a pre-existing package request bug - [ ] Alternatively, the PR is marked as related to a pre-existing package request bug, such as a dependency - [ ] REQUIRED - The package is available under an OSI-approved or FSF-approved license - [ ] REQUIRED - The version of the package is still receiving security updates - [ ] This PR links to the upstream project's support policy (e.g. `endoflife.date`) #### For new version streams <!-- remove if unrelated --> - [ ] The upstream project actually supports multiple concurrent versions. - [ ] Any subpackages include the version string in their package name (e.g. `name: ${{package.name}}-compat`) - [ ] The package (and subpackages) `provides:` logical unversioned forms of the package (e.g. `nodejs`, `nodejs-lts`) - [ ] If non-streamed package names no longer built, open PR to withdraw them (see [WITHDRAWING PACKAGES](https://github.com/wolfi-dev/os/blob/main/WITHDRAWING_PACKAGES.md)) #### For package updates (renames) in the base images <!-- remove if unrelated --> When updating packages part of base images (i.e. cgr.dev/chainguard/wolfi-base or ghcr.io/wolfi-dev/sdk) - [ ] REQUIRED cgr.dev/chainguard/wolfi-base and ghcr.io/wolfi-dev/sdk images successfully build - [ ] REQUIRED cgr.dev/chainguard/wolfi-base and ghcr.io/wolfi-dev/sdk contain no obsolete (no longer built) packages - [ ] Upon launch, does `apk upgrade --latest` successfully upgrades packages or performs no actions #### For security-related PRs <!-- remove if unrelated --> - [ ] The security fix is recorded in the [advisories](https://github.com/wolfi-dev/advisories) repo #### For version bump PRs <!-- remove if unrelated --> - [ ] The `epoch` field is reset to 0 #### For PRs that add patches <!-- remove if unrelated --> - [ ] Patch source is documented --------- Signed-off-by: Mritunjay Sharma <[email protected]>
1 parent 9b76b05 commit 9b0b540

File tree

3 files changed

+401
-0
lines changed

3 files changed

+401
-0
lines changed

‎kubeflow.yaml

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# Please note that kubeflow is in pipeline to move its components and then we might have to refactor this in future updates https://github.com/kubeflow/kubeflow/issues/7549
2+
# Also, similarly in test scripts, they are going to move manifests from apps directory to applications directory in a future release https://github.com/kubeflow/manifests/pull/3167
3+
# Please update the script for CRDs and other manifest then.
4+
# This package contains all go kubeflow controllers and components.
5+
package:
6+
name: kubeflow
7+
version: "1.10.0"
8+
epoch: 0
9+
description: Kubeflow Go Components
10+
copyright:
11+
- license: Apache-2.0
12+
13+
pipeline:
14+
- uses: git-checkout
15+
with:
16+
repository: https://github.com/kubeflow/kubeflow
17+
tag: v${{package.version}}
18+
expected-commit: 90e987bf87d3e7c900926310b00bfa16b59e41eb
19+
20+
data:
21+
- name: controllers
22+
items:
23+
notebook-controller: The controller allows users to create a custom resource "Notebook" (jupyter notebook).
24+
profile-controller: Profile access management provides namespace level isolation
25+
pvcviewer-controller: Using this component, PVC Viewers can easily be created. PVCViewers enable users to open a filebrowser on arbitrary persistent volume claims.
26+
tensorboard-controller: Kubeflow Tensorboard Controller
27+
28+
# not writing descriptions here as their output is different for each
29+
- name: components
30+
items:
31+
access-management: access-management
32+
admission-webhook: webhook
33+
34+
subpackages:
35+
- range: controllers
36+
name: "kubeflow-${{range.key}}"
37+
description: ${{range.value}}
38+
pipeline:
39+
- uses: go/build
40+
with:
41+
packages: .
42+
modroot: components/${{range.key}}
43+
output: manager
44+
test:
45+
environment:
46+
contents:
47+
packages:
48+
- bash
49+
- openssl
50+
pipeline:
51+
- uses: test/kwok/cluster
52+
- name: "Test ${{range.key}} functionality with KWOK"
53+
runs: |
54+
./controller-test.sh "${{range.key}}" "${{package.version}}"
55+
56+
- range: controllers
57+
name: "kubeflow-${{range.key}}-compat"
58+
description: Compat for kubeflow-${{range.key}}
59+
pipeline:
60+
- runs: |
61+
mkdir -p "${{targets.subpkgdir}}"
62+
ln -sf /usr/bin/${{range.key}} ${{targets.subpkgdir}}/${{range.key}}
63+
test:
64+
pipeline:
65+
- runs: |
66+
stat /${{range.key}}
67+
68+
- range: components
69+
name: "kubeflow-${{range.key}}"
70+
description: "kubeflow-${{range.value}}"
71+
pipeline:
72+
- uses: go/build
73+
with:
74+
packages: .
75+
modroot: components/${{range.key}}
76+
output: ${{range.value}}
77+
test:
78+
environment:
79+
contents:
80+
packages:
81+
- bash
82+
- openssl
83+
- curl
84+
- kustomize
85+
pipeline:
86+
- uses: test/kwok/cluster
87+
- name: "Test ${{range.key}} functionality with KWOK"
88+
runs: |
89+
./components-test.sh "${{range.key}}" "${{package.version}}"
90+
91+
- range: components
92+
name: "kubeflow-${{range.key}}-compat"
93+
description: Compat for kubeflow-${{range.key}}
94+
pipeline:
95+
- runs: |
96+
mkdir -p "${{targets.subpkgdir}}"
97+
ln -sf /usr/bin/${{range.key}} ${{targets.subpkgdir}}/${{range.key}}
98+
test:
99+
pipeline:
100+
- runs: |
101+
stat /${{range.key}}
102+
103+
update:
104+
enabled: true
105+
github:
106+
identifier: kubeflow/kubeflow
107+
use-tag: true
108+
# There were some malformed early tags
109+
tag-filter: v1
110+
strip-prefix: v
111+
112+
test:
113+
pipeline:
114+
- uses: test/tw/ldd-check

‎kubeflow/components-test.sh

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
#!/bin/bash
2+
set -euxo pipefail
3+
4+
COMPONENTS="$1" # Component name: access-management | admission-webhook
5+
KUBEFLOW_TAG="$2" # Kubeflow version tag like v$KUBEFLOW_TAG
6+
7+
kubectl create ns test-ns || true
8+
9+
case "$COMPONENTS" in
10+
11+
access-management)
12+
echo "Running Access Management API test..."
13+
kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/v$KUBEFLOW_TAG/apps/profiles/upstream/crd/bases/kubeflow.org_profiles.yaml
14+
15+
echo "Simulating namespace-labels.yaml..."
16+
mkdir -p /etc/profile-controller
17+
cat <<EOF >/etc/profile-controller/namespace-labels.yaml
18+
kubeflow.org/creator: kubeflow
19+
environment: test
20+
EOF
21+
22+
/usr/bin/access-management &
23+
pid=$!
24+
sleep 10
25+
26+
echo "Testing profile creation"
27+
curl -s -w '%{http_code}\n' -o /dev/null -X POST -H "Content-Type: application/json" \
28+
-d '{"metadata":{"name":"user1"},"spec":{"owner":{"kind":"User","name":"[email protected]"}}}' \
29+
http://127.0.0.1:8081/kfam/v1/profiles
30+
31+
32+
echo "Confirming profile exists"
33+
kubectl get profile
34+
35+
kill $pid
36+
;;
37+
38+
admission-webhook)
39+
# Namespace and CRD
40+
kubectl create ns kubeflow || true
41+
mkdir -p /tmp/webhook-certs
42+
openssl req -x509 -newkey rsa:4096 -days 365 -nodes -keyout /tmp/webhook-certs/key.pem -out /tmp/webhook-certs/cert.pem -subj "/CN=localhost"
43+
44+
45+
kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/v$KUBEFLOW_TAG/apps/admission-webhook/upstream/base/crd.yaml
46+
47+
# Launch webhook
48+
/usr/bin/webhook -tlsCertFile /tmp/webhook-certs/cert.pem -tlsKeyFile /tmp/webhook-certs/key.pem &
49+
50+
pid=$!
51+
52+
sleep 5
53+
54+
echo "Fetching manifests..."
55+
mkdir -p /tmp/admission-webhook
56+
cd /tmp/admission-webhook
57+
58+
kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/v$KUBEFLOW_TAG/apps/admission-webhook/upstream/base/crd.yaml
59+
60+
echo "Creating namespace..."
61+
kubectl create ns kubeflow || true
62+
63+
echo "Deploying Service..."
64+
kubectl apply -f - <<EOF
65+
apiVersion: v1
66+
kind: Service
67+
metadata:
68+
name: poddefault-webhook
69+
namespace: kubeflow
70+
spec:
71+
ports:
72+
- port: 443
73+
targetPort: 8443
74+
protocol: TCP
75+
selector:
76+
app: poddefault-webhook
77+
EOF
78+
79+
echo "Applying MutatingWebhookConfiguration..."
80+
kubectl apply -f - <<EOF
81+
apiVersion: admissionregistration.k8s.io/v1
82+
kind: MutatingWebhookConfiguration
83+
metadata:
84+
name: mutating-webhook-configuration
85+
webhooks:
86+
- admissionReviewVersions:
87+
- v1beta1
88+
- v1
89+
clientConfig:
90+
caBundle: ""
91+
service:
92+
name: poddefault-webhook
93+
namespace: kubeflow
94+
path: /apply-poddefault
95+
sideEffects: None
96+
failurePolicy: Fail
97+
name: poddefault-webhook.kubeflow.org
98+
namespaceSelector:
99+
matchLabels:
100+
app.kubernetes.io/part-of: kubeflow-profile
101+
rules:
102+
- apiGroups:
103+
- ""
104+
apiVersions:
105+
- v1
106+
operations:
107+
- CREATE
108+
resources:
109+
- pods
110+
EOF
111+
112+
echo "Testing PodDefault CR..."
113+
kubectl apply -f - <<EOF
114+
apiVersion: kubeflow.org/v1alpha1
115+
kind: PodDefault
116+
metadata:
117+
name: add-gcp-secret
118+
namespace: kubeflow
119+
spec:
120+
selector:
121+
matchLabels:
122+
add-gcp-secret: "true"
123+
desc: "Add GCP credential"
124+
volumeMounts:
125+
- name: secret-volume
126+
mountPath: /secret/gcp
127+
volumes:
128+
- name: secret-volume
129+
secret:
130+
secretName: gcp-secret
131+
EOF
132+
133+
echo "Listing PodDefaults..."
134+
kubectl get poddefaults -n kubeflow
135+
136+
kill $pid
137+
;;
138+
139+
*)
140+
echo "No functional test defined for $COMPONENTS"
141+
exit 0
142+
;;
143+
esac

0 commit comments

Comments
 (0)