-
Notifications
You must be signed in to change notification settings - Fork 275
fix: correct candle-binding replace path in go.mod files #65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Alex Wang <[email protected]>
👥 vLLM Semantic Team NotificationThe following members have been identified for the changed files in this PR and have been automatically assigned: 📁
|
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
@aeft thanks for spotting this! Would please also help add create an integration test suite with these these verifiers? |
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
@rootfs Sure! How about placing all tests with heavy dependencies (e.g., real model calls) into I also noticed that files like |
|
@rootfs I noticed that these verifiers don't have assertions. How about I refactor them into tests with assertions? |
Sounds good!
Please go ahead. Once you are done, please add new Makefile targets for these two cases and ensure |
The only concern is that it relies on a probabilistic model. However, we can exclude those examples that are ambiguous (or relax the evaluation criteria). The main purpose of this integration test is to ensure the lower bound of performance and to verify that the main logic runs correctly. If you have no objections, I'll add the corresponding assertions. |
That's the purpose :D we want to ensure the models are reliable through CI tests |
|
@rootfs So, would you like me to add some assertions? We can exclude those examples that are ambiguous (or relax the evaluation criteria). This ensures the lower bound of performance and verifies that the main logic runs correctly. We can observe it first, and consider removing it if it turns out to be unstable. |
I prefer assert. We can do it in two ways: for functional unit testing, relaxed or lower bound are acceptable. For integration test, we need to be more rigorous on the classification. |
Great. For integration tests, we call the real model and test (assert) classification performance. For functional unit tests, we mock the model call and test the function logic. I will gradually submit PRs over the next few days. Thanks for the discussion! |
|
Great! looking forward to your next PR |
What type of PR is this?
fix: correct candle-binding replace path in go.mod files
What this PR does / why we need it:
candle-binding replace path in go.mod files is incorrect.
make test-pii-classifierwill fail.Which issue(s) this PR fixes:
Fixes #
Release Notes: No