Skip to content
This repository was archived by the owner on Nov 16, 2023. It is now read-only.

Commit 2eef782

Browse files
author
Aleksandr Panchul
committed
adding inference prototype demo
1 parent 94fbf62 commit 2eef782

File tree

5 files changed

+142
-6
lines changed

5 files changed

+142
-6
lines changed

‎Research/kubeflow-on-azure-stack/Readme.md‎

Lines changed: 85 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -512,7 +512,7 @@ it into a json if needed, and pass to the model in a container:
512512
container:
513513
image: docker/whalesay:latest
514514
command: [sh, -c]
515-
args: ["echo {\"x\":3.0} | tee /tmp/input_request.json"]
515+
args: ["echo \"{\\\"x\\\":3.0}\" | tee /tmp/input_request.json"]
516516
outputs:
517517
artifacts:
518518
# generate hello-art artifact from /tmp/input_request.json
@@ -535,6 +535,90 @@ it into a json if needed, and pass to the model in a container:
535535
Depending on the application, you may want to communicate with the pipeline via requests to the first
536536
step. Or you can compress it all and have your logic inside of the single container.
537537

538+
apiVersion: argoproj.io/v1alpha1
539+
kind: Workflow
540+
metadata:
541+
generateName: inferencing-demo-curl-
542+
spec:
543+
entrypoint: inferencing-example
544+
templates:
545+
- name: inferencing-example
546+
steps:
547+
- - name: run-inference-server
548+
template: run-model
549+
550+
- name: run-model
551+
container:
552+
image: nginx:alpine
553+
ports:
554+
- containerPort: 80
555+
556+
Now you can see the pod with this service(a place-holder for your model) running if you want:
557+
558+
![pics/pipeline_server.png](pics/pipeline_server.png)
559+
560+
It is, in the case above, `inferencing-demo-curl-kj6z5-2763558554`.
561+
562+
Another option is to deploy your model as Kubernetes deployment, here is an nginx server you could try:
563+
564+
apiVersion: apps/v1
565+
kind: Deployment
566+
metadata:
567+
name: my-nginx
568+
labels:
569+
app: my-nginx
570+
spec:
571+
replicas: 2
572+
selector:
573+
matchLabels:
574+
app: my-nginx
575+
template:
576+
metadata:
577+
labels:
578+
app: my-nginx
579+
spec:
580+
containers:
581+
- name: my-nginx
582+
image: nginx:alpine
583+
ports:
584+
- containerPort: 80
585+
resources:
586+
limits:
587+
memory: "128Mi" #128 MB
588+
cpu: "200m" #200 millicpu (.2 cpu or 20% of the cpu)
589+
590+
Now you can increment number of replicas and your service will be automatically scaled.
591+
592+
This is how you can forward the port for this deployment:
593+
594+
$ kubectl port-forward deployment/my-nginx 7000:80
595+
Forwarding from 127.0.0.1:7000 -> 80
596+
Forwarding from [::1]:7000 -> 80
597+
Handling connection for 7000
598+
599+
And access your server(model containers) using, for example, `curl`:
600+
601+
$ curl localhost:7000
602+
<!DOCTYPE html>
603+
<html>
604+
<head>
605+
<title>Welcome to nginx!</title>
606+
...
607+
</body>
608+
</html>
609+
610+
Here is an example of how you can connect to the model you trained:
611+
612+
$ curl --header "Content-Type: application/json" \
613+
--request POST \
614+
--data '{"X":123,"Y":568}' \
615+
http://localhost:5050/api/infere
616+
Elasticnet model (alpha=0.050000, l1_ratio=0.050000):
617+
618+
RMSE: 82.16359959591213
619+
MAE: 69.52472687854122
620+
R2: -0.02072575078015859
621+
538622
## Next Steps
539623

540624
Proceed to [TensorFlow on Kubeflow Tutorial](tensorflow-on-kubeflow/Readme.md#tensorflow-on-kubeflow-on-azure-stack)

‎Research/kubeflow-on-azure-stack/pics/pipeline_server.png‎

52.3 KB
Loading

‎Research/kubeflow-on-azure-stack/sbin/deployment-demo.yaml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: my-nginx
5+
labels:
6+
app: my-nginx
7+
spec:
8+
replicas: 2
9+
selector:
10+
matchLabels:
11+
app: my-nginx
12+
template:
13+
metadata:
14+
labels:
15+
app: my-nginx
16+
spec:
17+
containers:
18+
- name: my-nginx
19+
image: nginx:alpine
20+
ports:
21+
- containerPort: 80
22+
resources:
23+
limits:
24+
memory: "128Mi" #128 MB
25+
cpu: "200m" #200 millicpu (.2 cpu or 20% of the cpu)
26+
27+

‎Research/kubeflow-on-azure-stack/sbin/inference.yaml‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
#
2-
# the pipeline that
2+
# the pipeline to pass parameters to a model and
3+
# eventually expose the deployment port to the outside.
34
#
45
apiVersion: argoproj.io/v1alpha1
56
kind: Workflow
@@ -23,15 +24,16 @@ spec:
2324

2425
- name: whalesay
2526
container:
26-
image: docker/whalesay:latest
27+
#image: docker/whalesay:latest
28+
image: alpine:latest
2729
command: [sh, -c]
28-
args: ["echo {\"x\":3.0} | tee /tmp/input_request.json"]
30+
args: ["echo -e \"{\\\"x\\\":3.0}\" | tee /tmp/message"]
2931
outputs:
3032
artifacts:
31-
# generate hello-art artifact from /tmp/input_request.json
33+
# generate hello-art artifact from /tmp/message
3234
# artifacts can be directories as well as files
3335
- name: input-art
34-
path: /tmp/input_request.json
36+
path: /tmp/message
3537

3638
- name: run-model
3739
inputs:

‎Research/kubeflow-on-azure-stack/sbin/inference_curl.yaml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
#
2+
# the pipeline to create an inferencing endpoint
3+
#
4+
apiVersion: argoproj.io/v1alpha1
5+
kind: Workflow
6+
metadata:
7+
generateName: inferencing-demo-curl-
8+
spec:
9+
entrypoint: inferencing-example
10+
templates:
11+
- name: inferencing-example
12+
steps:
13+
- - name: run-inference-server
14+
template: run-model
15+
16+
- name: run-model
17+
container:
18+
image: nginx:alpine
19+
ports:
20+
- containerPort: 80
21+
#command: [sh, -c]
22+
#args: ["do stuff"]
23+

0 commit comments

Comments
 (0)