Skip to content

Commit d8625d8

Browse files
szedan-rhszedan-rh
committed
Add comprehensive keyword routing tests to e2e/testcases
Consolidate all keyword routing tests into e2e/testcases for CI integration. This adds comprehensive testing for keyword-based routing with both E2E integration tests and unit tests: E2E Integration Test (keyword_routing.go): - Registers as 'keyword-routing' testcase for Kind-based CI - Makes HTTP requests to router service via port forwarding - Validates classification via X-VSR-Category header - Tests 26 scenarios with 80% accuracy threshold Unit Tests (keyword_routing_test.go): - 35 comprehensive unit tests using Ginkgo/Gomega - Tests OR, AND, NOR operators - Validates case sensitivity and word boundaries - Tests regex special characters and edge cases - Tests error handling and multiple rule matching Test Data: - keyword_routing_cases.json - 26 E2E test scenarios - embedding_routing_cases.json - Embedding test data - hybrid_routing_cases.json - Hybrid routing test data Helper Functions (helpers.go): - CreateKeywordTestRules() - Standard test rule definitions - CreateTestKeywordClassifier() - Test classifier factory - Test suite configuration (suite_test.go) Changes: - Move tests from e2e-tests/testcases to e2e/testcases - Remove old e2e-tests/testcases directory - Update .gitignore to remove e2e-tests/testcases references This addresses reviewer feedback to consolidate tests in e2e/testcases for proper CI integration. Signed-off-by: Senan Zedan <[email protected]>
1 parent 22d8d3a commit d8625d8

File tree

13 files changed

+699
-248
lines changed

13 files changed

+699
-248
lines changed

‎.github/workflows/unit-test-e2e-testcases.yml‎

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ name: E2E Testcases Unit Tests
33
on:
44
pull_request:
55
paths:
6-
- "e2e-tests/testcases/**"
6+
- "e2e/testcases/**"
77
- "src/semantic-router/pkg/classification/**"
88
- "src/semantic-router/pkg/config/**"
99
- "candle-binding/**"
@@ -31,7 +31,7 @@ jobs:
3131
with:
3232
go-version: ${{ env.GO_VERSION }}
3333
cache: true
34-
cache-dependency-path: e2e-tests/testcases/go.sum
34+
cache-dependency-path: e2e/testcases/go.sum
3535

3636
- name: Set up Rust
3737
uses: actions-rust-lang/setup-rust-toolchain@v1
@@ -71,14 +71,14 @@ jobs:
7171
env:
7272
LD_LIBRARY_PATH: ${{ github.workspace }}/candle-binding/target/release
7373
run: |
74-
cd e2e-tests/testcases
74+
cd e2e/testcases
7575
echo "Running keyword routing tests..."
7676
go test -v -coverprofile=coverage-keyword.out -covermode=atomic -coverpkg=github.com/vllm-project/semantic-router/src/semantic-router/pkg/classification
7777
7878
- name: Generate coverage report
7979
if: always()
8080
run: |
81-
cd e2e-tests/testcases
81+
cd e2e/testcases
8282
go tool cover -func=coverage-keyword.out > coverage-summary.txt
8383
echo "=== Full Coverage Summary ==="
8484
cat coverage-summary.txt
@@ -96,7 +96,7 @@ jobs:
9696
- name: Check coverage threshold
9797
if: always()
9898
run: |
99-
cd e2e-tests/testcases
99+
cd e2e/testcases
100100
COVERAGE_PERCENT=$(echo $COVERAGE | sed 's/%//')
101101
THRESHOLD=80
102102
@@ -111,7 +111,7 @@ jobs:
111111
uses: codecov/codecov-action@v4
112112
if: always()
113113
with:
114-
files: ./e2e-tests/testcases/coverage-keyword.out
114+
files: ./e2e/testcases/coverage-keyword.out
115115
flags: e2e-testcases-keyword
116116
name: keyword-routing-coverage
117117
fail_ci_if_error: false
@@ -141,7 +141,7 @@ jobs:
141141
runs-on: ubuntu-latest
142142
# Only run if embedding tests exist (for future PRs)
143143
if: |
144-
contains(github.event.pull_request.changed_files, 'e2e-tests/testcases/embedding_routing_test.go') ||
144+
contains(github.event.pull_request.changed_files, 'e2e/testcases/embedding_routing_test.go') ||
145145
github.event_name == 'workflow_dispatch'
146146
147147
steps:
@@ -168,7 +168,7 @@ jobs:
168168
env:
169169
LD_LIBRARY_PATH: ${{ github.workspace }}/candle-binding/target/release
170170
run: |
171-
cd e2e-tests/testcases
171+
cd e2e/testcases
172172
if [ -f "embedding_routing_test.go" ] && ! [[ "$(basename embedding_routing_test.go)" =~ \.skip$ ]]; then
173173
echo "Running embedding routing tests..."
174174
go test -v -run "Embedding Routing" -coverprofile=coverage-embedding.out -covermode=atomic
@@ -181,7 +181,7 @@ jobs:
181181
runs-on: ubuntu-latest
182182
# Only run if hybrid tests exist (for future PRs)
183183
if: |
184-
contains(github.event.pull_request.changed_files, 'e2e-tests/testcases/hybrid_routing_test.go') ||
184+
contains(github.event.pull_request.changed_files, 'e2e/testcases/hybrid_routing_test.go') ||
185185
github.event_name == 'workflow_dispatch'
186186
187187
steps:
@@ -208,7 +208,7 @@ jobs:
208208
env:
209209
LD_LIBRARY_PATH: ${{ github.workspace }}/candle-binding/target/release
210210
run: |
211-
cd e2e-tests/testcases
211+
cd e2e/testcases
212212
if [ -f "hybrid_routing_test.go" ] && ! [[ "$(basename hybrid_routing_test.go)" =~ \.skip$ ]]; then
213213
echo "Running hybrid routing tests..."
214214
go test -v -run "Hybrid Routing" -coverprofile=coverage-hybrid.out -covermode=atomic
@@ -244,7 +244,7 @@ jobs:
244244
env:
245245
LD_LIBRARY_PATH: ${{ github.workspace }}/candle-binding/target/release
246246
run: |
247-
cd e2e-tests/testcases
247+
cd e2e/testcases
248248
echo "Running tests with race detector..."
249249
go test -race -v || {
250250
echo "❌ Race conditions detected!"
@@ -270,7 +270,7 @@ jobs:
270270
uses: golangci/golangci-lint-action@v6
271271
with:
272272
version: latest
273-
working-directory: e2e-tests/testcases
273+
working-directory: e2e/testcases
274274
args: --timeout=5m
275275

276276
summary:

‎.gitignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -158,3 +158,9 @@ dashboard/frontend/index.html.old
158158

159159
# Kind cluster configuration
160160
tools/kind/kind-config.yaml
161+
162+
# Local test files
163+
test-report.md
164+
test-report.json
165+
e2e/e2e
166+
e2e-tests/llm_katan_pids.txt

‎e2e-tests/testcases/go.mod‎

Lines changed: 0 additions & 48 deletions
This file was deleted.

‎e2e-tests/testcases/go.sum‎

Lines changed: 0 additions & 88 deletions
This file was deleted.

‎e2e/go.mod‎

Lines changed: 60 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,53 +1,83 @@
11
module github.com/vllm-project/semantic-router/e2e
22

3-
go 1.24
3+
go 1.24.1
4+
5+
replace (
6+
github.com/vllm-project/semantic-router/candle-binding => ../candle-binding
7+
github.com/vllm-project/semantic-router/src/semantic-router => ../src/semantic-router
8+
)
49

510
require (
6-
k8s.io/api v0.31.0
7-
k8s.io/apimachinery v0.31.0
8-
k8s.io/client-go v0.31.0
11+
github.com/onsi/ginkgo/v2 v2.23.4
12+
github.com/onsi/gomega v1.38.0
13+
github.com/vllm-project/semantic-router/src/semantic-router v0.0.0-00010101000000-000000000000
14+
k8s.io/api v0.34.2
15+
k8s.io/apimachinery v0.34.2
16+
k8s.io/client-go v0.34.2
917
)
1018

1119
require (
20+
github.com/bahlo/generic-list-go v0.2.0 // indirect
21+
github.com/beorn7/perks v1.0.1 // indirect
22+
github.com/buger/jsonparser v1.1.1 // indirect
23+
github.com/cespare/xxhash/v2 v2.3.0 // indirect
1224
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
13-
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
14-
github.com/fxamacker/cbor/v2 v2.7.0 // indirect
15-
github.com/go-logr/logr v1.4.2 // indirect
16-
github.com/go-openapi/jsonpointer v0.19.6 // indirect
25+
github.com/emicklei/go-restful/v3 v3.12.2 // indirect
26+
github.com/fxamacker/cbor/v2 v2.9.0 // indirect
27+
github.com/go-logr/logr v1.4.3 // indirect
28+
github.com/go-openapi/jsonpointer v0.21.0 // indirect
1729
github.com/go-openapi/jsonreference v0.20.2 // indirect
18-
github.com/go-openapi/swag v0.22.4 // indirect
30+
github.com/go-openapi/swag v0.23.0 // indirect
31+
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
1932
github.com/gogo/protobuf v1.3.2 // indirect
20-
github.com/golang/protobuf v1.5.4 // indirect
21-
github.com/google/gnostic-models v0.6.8 // indirect
22-
github.com/google/go-cmp v0.6.0 // indirect
23-
github.com/google/gofuzz v1.2.0 // indirect
33+
github.com/google/gnostic-models v0.7.0 // indirect
34+
github.com/google/go-cmp v0.7.0 // indirect
35+
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6 // indirect
2436
github.com/google/uuid v1.6.0 // indirect
25-
github.com/gorilla/websocket v1.5.0 // indirect
26-
github.com/imdario/mergo v0.3.16 // indirect
37+
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 // indirect
38+
github.com/invopop/jsonschema v0.13.0 // indirect
2739
github.com/josharian/intern v1.0.0 // indirect
2840
github.com/json-iterator/go v1.1.12 // indirect
2941
github.com/mailru/easyjson v0.7.7 // indirect
30-
github.com/moby/spdystream v0.4.0 // indirect
42+
github.com/mark3labs/mcp-go v0.42.0-beta.1 // indirect
43+
github.com/moby/spdystream v0.5.0 // indirect
3144
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
32-
github.com/modern-go/reflect2 v1.0.2 // indirect
45+
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
3346
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
3447
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect
35-
github.com/spf13/pflag v1.0.5 // indirect
48+
github.com/pkg/errors v0.9.1 // indirect
49+
github.com/prometheus/client_golang v1.23.0 // indirect
50+
github.com/prometheus/client_model v0.6.2 // indirect
51+
github.com/prometheus/common v0.65.0 // indirect
52+
github.com/prometheus/procfs v0.16.1 // indirect
53+
github.com/spf13/cast v1.7.1 // indirect
54+
github.com/spf13/pflag v1.0.6 // indirect
55+
github.com/vllm-project/semantic-router/candle-binding v0.0.0-00010101000000-000000000000 // indirect
56+
github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect
3657
github.com/x448/float16 v0.8.4 // indirect
37-
golang.org/x/net v0.26.0 // indirect
38-
golang.org/x/oauth2 v0.21.0 // indirect
39-
golang.org/x/sys v0.21.0 // indirect
40-
golang.org/x/term v0.21.0 // indirect
41-
golang.org/x/text v0.16.0 // indirect
42-
golang.org/x/time v0.5.0 // indirect
43-
google.golang.org/protobuf v1.34.2 // indirect
58+
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
59+
go.uber.org/automaxprocs v1.6.0 // indirect
60+
go.uber.org/multierr v1.11.0 // indirect
61+
go.uber.org/zap v1.27.0 // indirect
62+
go.yaml.in/yaml/v2 v2.4.2 // indirect
63+
go.yaml.in/yaml/v3 v3.0.4 // indirect
64+
golang.org/x/net v0.43.0 // indirect
65+
golang.org/x/oauth2 v0.30.0 // indirect
66+
golang.org/x/sys v0.37.0 // indirect
67+
golang.org/x/term v0.34.0 // indirect
68+
golang.org/x/text v0.28.0 // indirect
69+
golang.org/x/time v0.9.0 // indirect
70+
golang.org/x/tools v0.35.0 // indirect
71+
google.golang.org/protobuf v1.36.9 // indirect
72+
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
4473
gopkg.in/inf.v0 v0.9.1 // indirect
4574
gopkg.in/yaml.v2 v2.4.0 // indirect
4675
gopkg.in/yaml.v3 v3.0.1 // indirect
4776
k8s.io/klog/v2 v2.130.1 // indirect
48-
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 // indirect
49-
k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 // indirect
50-
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
51-
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
52-
sigs.k8s.io/yaml v1.4.0 // indirect
77+
k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b // indirect
78+
k8s.io/utils v0.0.0-20250604170112-4c0f3b243397 // indirect
79+
sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 // indirect
80+
sigs.k8s.io/randfill v1.0.0 // indirect
81+
sigs.k8s.io/structured-merge-diff/v6 v6.3.0 // indirect
82+
sigs.k8s.io/yaml v1.6.0 // indirect
5383
)

0 commit comments

Comments
 (0)