Skip to content

test(e2e): add embedding signal E2E tests for CRDs #749

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Dec 2, 2025
Merged

test(e2e): add embedding signal E2E tests for CRDs #749

merged 1 commit into from
Dec 2, 2025

Conversation

@yehudit1987
Copy link
Contributor

@yehudit1987 yehudit1987 commented Dec 1, 2025

Implement comprehensive E2E tests for IntelligentRoute CRD with
EmbeddingSignal configurations to validate semantic similarity-based
routing functionality.

Changes:

  • Add embedding_signal_routing.go test with test cases for:
    • PII detection via semantic patterns
    • Security threat detection
    • Technical domain routing (Kubernetes)
    • Domain classification (healthcare, finance)
    • Threshold, aggregation, and edge case testing
  • Create embedding_signal_cases.json test data
  • Add EmbeddingSignal definitions to intelligentroute.yaml:
    • pii_detected signal with candidate patterns
    • security_threat signal for malicious intent
    • kubernetes_topic signal for technical routing
  • Add corresponding decisions: block_pii, block_security,
    kubernetes_expert
  • Configure dynamic-config profile for CRD-based testing
  • Update profile.go to enable embedding-signal-routing test
  • Document test in e2e/README.md

Fixes and improvements:

  • Make init_embedding_models idempotent in Rust FFI
  • Add EMBEDDING_MODEL_OVERRIDE env var for test consistency
  • Fix CreatePIIViolationResponse to set decision header
  • Update test to check decision header on blocked responses
  • Increase memory limits in values.yaml (10Gi) for embedding
    models

Resolves #715

@netlify
Copy link

netlify bot commented Dec 1, 2025 •
edited
Loading

✅ Deploy Preview for vllm-semantic-router ready!

Name Link
🔨 Latest commit 68bfd59
🔍 Latest deploy log https://app.netlify.com/projects/vllm-semantic-router/deploys/692ec0333aefef0008494b93
😎 Deploy Preview https://deploy-preview-749--vllm-semantic-router.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions
Copy link

github-actions bot commented Dec 1, 2025 •
edited
Loading

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 Root Directory

Owners: @rootfs, @Xunzhuo
Files changed:

  • .github/workflows/integration-test-dynamic-config.yml

📁 candle-binding

Owners: @rootfs
Files changed:

  • candle-binding/src/ffi/embedding.rs

📁 e2e

Owners: @Xunzhuo
Files changed:

  • e2e/README.md
  • e2e/profiles/dynamic-config/crds/intelligentroute.yaml
  • e2e/profiles/dynamic-config/profile.go
  • e2e/profiles/dynamic-config/values.yaml
  • e2e/testcases/embedding_signal_routing.go
  • e2e/testcases/testdata/embedding_signal_cases.json

📁 src

Owners: @rootfs, @Xunzhuo, @wangchen615
Files changed:

  • src/semantic-router/pkg/classification/embedding_classifier.go
  • src/semantic-router/pkg/extproc/req_filter_pii.go
  • src/semantic-router/pkg/utils/http/response.go
vLLM

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

@yehudit1987 yehudit1987 force-pushed the signal_embeddings_tests branch 6 times, most recently from 16e3791 to b71e7fb Compare December 2, 2025 07:50
@yehudit1987 yehudit1987 force-pushed the signal_embeddings_tests branch from b71e7fb to 68bfd59 Compare December 2, 2025 10:32
@yehudit1987 yehudit1987 marked this pull request as ready for review December 2, 2025 10:32
Xunzhuo
Xunzhuo approved these changes Dec 2, 2025
View reviewed changes
Copy link
Member

@Xunzhuo Xunzhuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Thank you

@Xunzhuo Xunzhuo merged commit cfc7657 into vllm-project:main Dec 2, 2025
25 of 26 checks passed
@yehudit1987 yehudit1987 deleted the signal_embeddings_tests branch December 2, 2025 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Xunzhuo Xunzhuo Xunzhuo approved these changes

@rootfs rootfs Awaiting requested review from rootfs rootfs is a code owner

@wangchen615 wangchen615 Awaiting requested review from wangchen615 wangchen615 is a code owner

Assignees

@rootfs rootfs

@wangchen615 wangchen615

@Xunzhuo Xunzhuo

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Test] Add Comprehensive Test Coverage for Embedding-Based Signals

4 participants

@yehudit1987 @Xunzhuo @rootfs @wangchen615