Skip to content

Commit 03383ad

Browse files
MiguelAAeMiguelAAe
authored
feat(e2e-tests): added inference of pipeline (#7029)
* model and pipeline inference * step for pipeline contains "a" * wip inference * inference with tests * feature tests with inference and better wording * pipeline inference * do not clean resources of scenarios that fail * inference request delay * pipeline assertion test * format pipeline feature files * don't clean up resources of scenarios that fail based on config flag * resource name * raw output contents todo * pipeline setup
1 parent c9bba8a commit 03383ad

16 files changed

+866
-60
lines changed

‎tests/integration/godog/features/model/custom_model_deployment.feature‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ Feature: Explicit Model deployment
1919
storageUri: gs://seldon-models/scv2/samples/mlserver_1.3.5/iris-sklearn
2020
"""
2121
When the model "alpha-1" should eventually become Ready with timeout "20s"
22-
Then send HTTP inference request with timeout "20s" to model "alpha-1" with payload:
22+
Then I send HTTP inference request with timeout "20s" to model "alpha-1" with payload:
2323
"""
2424
{
2525
"inputs": [
@@ -52,7 +52,7 @@ Feature: Explicit Model deployment
5252
}
5353
] }
5454
"""
55-
Then send gRPC inference request with timeout "20s" to model "alpha-1" with payload:
55+
Then I send gRPC inference request with timeout "20s" to model "alpha-1" with payload:
5656
"""
5757
{
5858
"inputs": [
@@ -85,7 +85,7 @@ Feature: Explicit Model deployment
8585
] }
8686
"""
8787
Then delete the model "alpha-1" with timeout "10s"
88-
Then send HTTP inference request with timeout "20s" to model "alpha-1" with payload:
88+
Then I send HTTP inference request with timeout "20s" to model "alpha-1" with payload:
8989
"""
9090
{
9191
"inputs": [
@@ -99,7 +99,7 @@ Feature: Explicit Model deployment
9999
}
100100
"""
101101
And expect http response status code "404"
102-
Then send gRPC inference request with timeout "20s" to model "alpha-1" with payload:
102+
Then I send gRPC inference request with timeout "20s" to model "alpha-1" with payload:
103103
"""
104104
{
105105
"inputs": [

‎tests/integration/godog/features/model/over_commit.feature‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ Feature: Explicit Model deployment
5151
storageUri: gs://seldon-models/scv2/samples/mlserver_1.3.5/iris-sklearn
5252
"""
5353
When the model "overcommit-3" should eventually become Ready with timeout "20s"
54-
Then send HTTP inference request with timeout "20s" to model "overcommit-1" with payload:
54+
Then I send HTTP inference request with timeout "20s" to model "overcommit-1" with payload:
5555
"""
5656
{
5757
"inputs": [
@@ -65,7 +65,7 @@ Feature: Explicit Model deployment
6565
}
6666
"""
6767
And expect http response status code "200"
68-
Then send HTTP inference request with timeout "20s" to model "overcommit-2" with payload:
68+
Then I send HTTP inference request with timeout "20s" to model "overcommit-2" with payload:
6969
"""
7070
{
7171
"inputs": [
@@ -79,7 +79,7 @@ Feature: Explicit Model deployment
7979
}
8080
"""
8181
And expect http response status code "200"
82-
Then send HTTP inference request with timeout "20s" to model "overcommit-3" with payload:
82+
Then I send HTTP inference request with timeout "20s" to model "overcommit-3" with payload:
8383
"""
8484
{
8585
"inputs": [

‎tests/integration/godog/features/pipeline/conditional.feature‎

Lines changed: 73 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
1-
@PipelineDeployment @Functional @Pipelines @Conditional
1+
@PipelineConditional @Functional @Pipelines @Conditional
22
Feature: Conditional pipeline with branching models
3-
This pipeline uses a conditional model to route data to either add10 or mul10.
3+
In order to support decision-based inference
4+
As a model user
5+
I need a conditional pipeline that directs inputs to one of multiple models based on a condition
46

5-
Scenario: Deploy tfsimple-conditional pipeline and wait for readiness
7+
Scenario: Deploy a conditional pipeline, run inference, and verify the output
68
Given I deploy model spec with timeout "30s":
79
"""
810
apiVersion: mlops.seldon.io/v1alpha1
@@ -43,7 +45,7 @@ Feature: Conditional pipeline with branching models
4345
And the model "add10-nbsl" should eventually become Ready with timeout "20s"
4446
And the model "mul10-nbsl" should eventually become Ready with timeout "20s"
4547

46-
And I deploy pipeline spec with timeout "30s":
48+
When I deploy a pipeline spec with timeout "30s":
4749
"""
4850
apiVersion: mlops.seldon.io/v1alpha1
4951
kind: Pipeline
@@ -69,3 +71,70 @@ Feature: Conditional pipeline with branching models
6971
stepsJoin: any
7072
"""
7173
Then the pipeline "tfsimple-conditional-nbsl" should eventually become Ready with timeout "40s"
74+
Then I send gRPC inference request with timeout "20s" to pipeline "tfsimple-conditional-nbsl" with payload:
75+
"""
76+
{
77+
"model_name": "conditional-nbsl",
78+
"inputs": [
79+
{
80+
"name": "CHOICE",
81+
"contents": {
82+
"int_contents": [
83+
0
84+
]
85+
},
86+
"datatype": "INT32",
87+
"shape": [
88+
1
89+
]
90+
},
91+
{
92+
"name": "INPUT0",
93+
"contents": {
94+
"fp32_contents": [
95+
1,
96+
2,
97+
3,
98+
4
99+
]
100+
},
101+
"datatype": "FP32",
102+
"shape": [
103+
4
104+
]
105+
},
106+
{
107+
"name": "INPUT1",
108+
"contents": {
109+
"fp32_contents": [
110+
1,
111+
2,
112+
3,
113+
4
114+
]
115+
},
116+
"datatype": "FP32",
117+
"shape": [
118+
4
119+
]
120+
}
121+
]
122+
}
123+
"""
124+
And expect gRPC response body to contain JSON:
125+
"""
126+
{
127+
"outputs": [
128+
{
129+
"name": "OUTPUT",
130+
"datatype": "FP32",
131+
"shape": [
132+
4
133+
]
134+
}
135+
],
136+
"raw_output_contents": [
137+
"AAAgQQAAoEEAAPBBAAAgQg=="
138+
]
139+
}
140+
"""

0 commit comments

Comments
 (0)