Skip to content

Commit ad03d9a

Browse files
ckadnerckadner
authored
doc: Add developer instructions (#57)
Add documentation that describes how to update the code, build a new image and test the code changes in a ModelMesh Serving deployment. --------- Signed-off-by: Christian Kadner <[email protected]>
1 parent 6eb3f77 commit ad03d9a

File tree

4 files changed

+198
-13
lines changed

4 files changed

+198
-13
lines changed

‎README.md

Lines changed: 181 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
11
[![Build](https://github.com/kserve/modelmesh-runtime-adapter/actions/workflows/build.yml/badge.svg?branch=main)](https://github.com/kserve/modelmesh-runtime-adapter/actions/workflows/build.yml)
22

3-
# Modelmesh Runtime Adapter
3+
# ModelMesh Runtime Adapter
44

5-
This repo contains the unified puller/runtime-adapter image of the sidecar containers which run in the modelmesh-serving model server Pods. See the main [modelmesh-serving](https://github.com/kserve/modelmesh-serving) repo for more details.
5+
This repo contains the unified puller/runtime-adapter image of the sidecar containers
6+
which run in the ModelMesh Serving model server Pods. Take a look at the main
7+
[ModelMesh Serving](https://github.com/kserve/modelmesh-serving) repo for more details.
68

79
Logical subcomponents within the image:
810

@@ -12,8 +14,183 @@ Logical subcomponents within the image:
1214
- [model-mesh-ovms-adapter](model-mesh-ovms-adapter)
1315
- [model-mesh-torchserve-adapter](model-mesh-torchserve-adapter)
1416

15-
### Build Image
17+
## Generate sources
1618

17-
```bash
19+
The gRPC code stubs, interfaces and data access classes have to be generated by the
20+
[`protoc` compiler](https://protobuf.dev/getting-started/gotutorial/#compiling-protocol-buffers)
21+
from the `.proto` source files under `internal/proto/*`.
22+
23+
If any of the `.proto` files were modified, run the `protoc` compiler to regenerate
24+
the respective Go source code. It's recommended to use the developer image, which
25+
has all the required libraries pre-installed, by running `make run proto.compile`
26+
instead of `make proto.compile`.
27+
28+
```shell
29+
make run proto.compile
30+
```
31+
32+
## Test the code changes
33+
34+
After making code changes, ensure all existing and new functionality still works
35+
properly by running the unit tests.
36+
37+
```shell
38+
make test
39+
```
40+
41+
## Format code
42+
43+
Run the linter to make sure all code style rules are adhered to. The code will
44+
automatically be formatted if any code style violations are found.
45+
46+
It's recommended to use the developer image, which has all the required libraries
47+
pre-installed, by running `make run fmt` instead of `make fmt`.
48+
49+
```shell
50+
make run fmt
51+
```
52+
53+
## Build the Docker image
54+
55+
Once the code changes have been tested and linted, build a new `modelmesh-runtime-adapter`
56+
Docker image.
57+
58+
```shell
1859
make build
1960
```
61+
62+
## Push the image to a container registry
63+
64+
Push the newly built `modelmesh-runtime-adapter` image to a container registry.
65+
Replace the value of the `DOCKER_USER` environment variable to your docker user ID
66+
and change the `IMAGE_TAG` to something meaningful.
67+
68+
```bash
69+
export DOCKER_USER="<your-docker-userid>"
70+
export IMAGE_TAG="dev"
71+
72+
docker tag kserve/modelmesh-runtime-adapter:latest \
73+
${DOCKER_USER}/modelmesh-runtime-adapter:${IMAGE_TAG}
74+
75+
docker push ${DOCKER_USER}/modelmesh-runtime-adapter:${IMAGE_TAG}
76+
```
77+
78+
## Update the ModelMesh Serving deployment
79+
80+
In order to test the code changes in an existing [ModelMesh Serving](https://github.com/kserve/modelmesh-serving)
81+
deployment, the newly built container image needs to be added to the
82+
`model-serving-config` ConfigMap.
83+
84+
### Check existing model serving configuration
85+
86+
First, check if your ModelMesh Serving deployment already has an existing
87+
`model-serving-config` ConfigMap:
88+
89+
```Shell
90+
kubectl get configmap
91+
92+
NAME DATA AGE
93+
kube-root-ca.crt 1 12d
94+
model-serving-config 1 12d
95+
model-serving-config-defaults 1 12d
96+
tc-config 2 12d
97+
```
98+
99+
### Create a new model serving config
100+
101+
If you did not already have a `model-serving-config` ConfigMap on your cluster,
102+
you can create one. Replace the `<your-docker-userid>` placeholder with your
103+
Docker username. Make sure the value of the `IMAGE_TAG` variable matches
104+
the one that was pushed to the container registry.
105+
106+
```shell
107+
export DOCKER_USER="<your-docker-userid>"
108+
export IMAGE_NAME="${DOCKER_USER}/modelmesh-runtime-adapter"
109+
export IMAGE_TAG="dev"
110+
111+
kubectl apply -f - <<EOF
112+
---
113+
apiVersion: v1
114+
kind: ConfigMap
115+
metadata:
116+
name: model-serving-config
117+
data:
118+
config.yaml: |
119+
storageHelperImage:
120+
name: ${IMAGE_NAME}
121+
tag: ${IMAGE_TAG}
122+
EOF
123+
```
124+
125+
### Update an existing model serving config
126+
127+
If the ConfigMap list contains `model-serving-config`, save the contents of your
128+
existing configuration in a local temp file:
129+
130+
```Bash
131+
mkdir -p temp
132+
kubectl get configmap model-serving-config -o yaml > temp/model-serving-config.yaml
133+
```
134+
135+
Add the `storageHelperImage` property to the `config.yaml` string property.
136+
137+
```YAML
138+
storageHelperImage:
139+
name: your-docker-userid/modelmesh-runtime-adapter
140+
tag: latest
141+
```
142+
143+
Replace the `your-docker-userid` placeholder with _your_ Docker username and make
144+
sure the `tag` matches the one that was pushed to the container registry earlier.
145+
146+
The complete ConfigMap YAML file _may_ look like this:
147+
148+
```YAML
149+
apiVersion: v1
150+
kind: ConfigMap
151+
metadata:
152+
name: model-serving-config
153+
namespace: modelmesh-serving
154+
data:
155+
config.yaml: |
156+
podsPerRuntime: 1
157+
restProxy:
158+
enabled: true
159+
scaleToZero:
160+
enabled: false
161+
gracePeriodSeconds: 5
162+
storageHelperImage:
163+
name: your-docker-userid/modelmesh-runtime-adapter
164+
tag: dev
165+
```
166+
167+
Apply the ConfigMap to your cluster:
168+
169+
```Bash
170+
kubectl apply -f temp/model-serving-config.yaml
171+
```
172+
173+
If you are comfortable using `vi`, you can forgo creating a temp file and edit
174+
the ConfigMap directly in the terminal:
175+
176+
```Shell
177+
kubectl edit configmap model-serving-config
178+
```
179+
180+
### Verify the container images used by the model serving runtime
181+
182+
The `modelmesh-controller` watches the ConfigMap and responds to updates by
183+
automatically restarting the serving runtime pods using the newly built
184+
runtime adapter image.
185+
186+
You can check which container images are used by running the following command:
187+
188+
```Shell
189+
kubectl get pods -o jsonpath='{range .items[*]}{"\n"}{.metadata.name}{"\t"}{range .spec.containers[*]}{.image}{", "}{end}{end}' | sort | column -ts $'\t' | sed 's/, *$//g'
190+
191+
etcd-78ff7867d5-45svw quay.io/coreos/etcd:v3.5.4
192+
minio-6ddbfc9665-gtf7x kserve/modelmesh-minio-examples:latest
193+
modelmesh-controller-64f5c8d6d6-k6rzc kserve/modelmesh-controller:latest
194+
modelmesh-serving-mlserver-1.x-84884c6849-s8dw6 kserve/rest-proxy:latest, seldonio/mlserver:1.3.2, your-docker-userid/modelmesh-runtime-adapter:dev, kserve/modelmesh:latest
195+
modelmesh-serving-mlserver-1.x-84884c6849-xpdw4 kserve/rest-proxy:latest, seldonio/mlserver:1.3.2, your-docker-userid/modelmesh-runtime-adapter:dev, kserve/modelmesh:latest
196+
```

‎model-mesh-torchserve-adapter/.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# files generated by tests
2+
server/testdata/_torchserve_models/mmconfig.properties
3+
torchserve/mocktorchserve
4+
main

‎scripts/build_docker.sh

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ usage() {
2929
exit 1
3030
}
3131

32+
DOCKER_USER=${DOCKER_USER:-"kserve"}
33+
IMAGE_TAG=${IMAGE_TAG:-"latest"}
3234
DOCKER_TARGET="runtime"
3335
DOCKER_TAG="$(git rev-parse --abbrev-ref HEAD)-$(date +"%Y%m%dT%H%M%S%Z")"
3436

@@ -75,8 +77,8 @@ fi
7577

7678
declare -a docker_args=(
7779
--target "${DOCKER_TARGET}"
78-
-t "kserve/modelmesh-runtime-adapter${IMAGE_SUFFIX}:${DOCKER_TAG}"
79-
-t "kserve/modelmesh-runtime-adapter${IMAGE_SUFFIX}:latest"
80+
-t "${DOCKER_USER}/modelmesh-runtime-adapter${IMAGE_SUFFIX}:${DOCKER_TAG}"
81+
-t "${DOCKER_USER}/modelmesh-runtime-adapter${IMAGE_SUFFIX}:${IMAGE_TAG}"
8082
)
8183

8284
if [[ $DOCKER_TARGET == 'runtime' ]]; then

‎scripts/run_tests.sh

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,15 @@
1515

1616
set -e
1717

18-
## declare an array variable
19-
declare -a subpackages=("model-mesh-mlserver-adapter"
20-
"model-mesh-ovms-adapter"
21-
"model-mesh-torchserve-adapter"
22-
"model-mesh-triton-adapter"
23-
"model-serving-puller"
24-
"pullman")
18+
# declare an array variable with all sub-packages
19+
declare -a subpackages=(
20+
"model-mesh-mlserver-adapter"
21+
"model-mesh-ovms-adapter"
22+
"model-mesh-torchserve-adapter"
23+
"model-mesh-triton-adapter"
24+
"model-serving-puller"
25+
"pullman"
26+
)
2527

2628
failed=false
2729
summary=""

0 commit comments

Comments
 (0)