Add comprehensive keyword routing tests #691

szedan-rh · 2025-11-19T07:25:26Z

Summary

This PR implements comprehensive end-to-end testing for keyword routing functionality to address Issue #667.

Test Coverage

Unit Tests (e2e-tests/testcases/)

35 test cases covering all keyword routing operators and edge cases
OR operator: Match when any keyword is present
AND operator: Match when all keywords are present
NOR operator: Match when no keywords are present
Case-sensitive and case-insensitive matching
Word boundary detection
Special character and regex pattern handling
Unicode and emoji support
Error handling and validation

E2E Integration Test (e2e/testcases/)

HTTP-based integration test using the E2E test framework
Makes requests to deployed router service
Validates classification via X-VSR-Category response header
Tests 26 real-world scenarios from JSON test data
Requires 80% accuracy threshold

Files Changed

Test Implementation

e2e-tests/testcases/keyword_routing_test.go - Unit tests using Ginkgo/Gomega (35 tests)
e2e-tests/testcases/helpers.go - Test helper functions
e2e-tests/testcases/suite_test.go - Test suite setup
e2e/testcases/keyword_routing.go - E2E integration test

Test Data

e2e-tests/testcases/testdata/keyword_routing_cases.json - 26 test scenarios
e2e/testcases/testdata/keyword_routing_cases.json - Copy for E2E framework

Go Modules

e2e-tests/testcases/go.mod - Module definition
e2e-tests/testcases/go.sum - Dependencies

CI/CD

.github/workflows/unit-test-e2e-testcases.yml - GitHub Actions workflow
- Builds Rust candle bindings (CPU-only for CI)
- Runs keyword routing tests
- Generates coverage reports (87.1% of keyword_classifier.go)
- Enforces 80% coverage threshold
- Race condition detection

Test Results

All tests passing:

Ran 35 of 35 Specs in 0.013 seconds
SUCCESS! -- 35 Passed | 0 Failed | 0 Pending | 0 Skipped

Coverage:

keyword_classifier.go: 87.1% (exceeds 80% threshold)
Race detector: PASS

Implementation Highlights

Isolated NOR operator tests - Prevents false matches by using dedicated classifier
Table-driven tests - Efficient testing of multiple scenarios
JSON test data - Maintainable external test cases
CI integration - Automated testing on every PR
Dual test approach - Both unit tests and E2E integration tests

Testing

Run Unit Tests

cd e2e-tests/testcases
LD_LIBRARY_PATH=../../candle-binding/target/release go test -v

Run E2E Test (requires Kind cluster)

make e2e-test-specific E2E_TESTS="keyword-routing"

Run with Coverage

cd e2e-tests/testcases
LD_LIBRARY_PATH=../../candle-binding/target/release \
  go test -coverprofile=coverage.out -covermode=atomic \
  -coverpkg=github.com/vllm-project/semantic-router/src/semantic-router/pkg/classification

Checklist

netlify · 2025-11-19T07:25:33Z

✅ Deploy Preview for vllm-semantic-router ready!

Name	Link
🔨 Latest commit	`49e0e70`
🔍 Latest deploy log	https://app.netlify.com/projects/vllm-semantic-router/deploys/691da3cf1dfd2a00081935f2
😎 Deploy Preview	https://deploy-preview-691--vllm-semantic-router.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Consolidate all keyword routing tests into e2e/testcases for CI integration. This adds comprehensive testing for keyword-based routing with both E2E integration tests and unit tests: E2E Integration Test (keyword_routing.go): - Registers as 'keyword-routing' testcase for Kind-based CI - Makes HTTP requests to router service via port forwarding - Validates classification via X-VSR-Category header - Tests 26 scenarios with 80% accuracy threshold Unit Tests (keyword_routing_test.go): - 35 comprehensive unit tests using Ginkgo/Gomega - Tests OR, AND, NOR operators - Validates case sensitivity and word boundaries - Tests regex special characters and edge cases - Tests error handling and multiple rule matching Test Data: - keyword_routing_cases.json - 26 E2E test scenarios - embedding_routing_cases.json - Embedding test data - hybrid_routing_cases.json - Hybrid routing test data Helper Functions (helpers.go): - CreateKeywordTestRules() - Standard test rule definitions - CreateTestKeywordClassifier() - Test classifier factory - Test suite configuration (suite_test.go) Changes: - Move tests from e2e-tests/testcases to e2e/testcases - Remove old e2e-tests/testcases directory - Update .gitignore to remove e2e-tests/testcases references This addresses reviewer feedback to consolidate tests in e2e/testcases for proper CI integration. Signed-off-by: Senan Zedan <[email protected]>

github-actions · 2025-11-19T11:02:52Z

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 `e2e`

Owners: @Xunzhuo
Files changed:

e2e/testcases/keyword_routing.go
e2e/testcases/testdata/embedding_routing_cases.json
e2e/testcases/testdata/hybrid_routing_cases.json
e2e/go.mod
e2e/go.sum
e2e/testcases/helpers.go
e2e/testcases/keyword_routing_test.go
e2e/testcases/suite_test.go
e2e/testcases/testdata/keyword_routing_cases.json

📁 `Root Directory`

Owners: @rootfs, @Xunzhuo
Files changed:

.github/workflows/unit-test-e2e-testcases.yml
.gitignore

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

szedan-rh requested review from Xunzhuo and rootfs as code owners November 19, 2025 07:25

szedan-rh force-pushed the test_keyword_routing branch from 6360ffb to d9345b2 Compare November 19, 2025 07:28