Skip to content

Commit b63f368

Browse files
authored
perf(e2e): reduce test case count to optimize CI execution time (#797)
Reduce test cases for the three slowest tests in dynamic-config profile: - domain-classify: 280 → 56 cases (4 per category across 14 categories) - jailbreak-detection: 100 → 25 cases (every 4th case) - pii-detection: 100 → 25 cases (every 4th case) Total reduction: 480 → 106 test cases (78% reduction) Rationale: Testing revealed the bottleneck is LLM inference time (~5-24s per request), not test framework overhead. Concurrent requests cause HTTP 500 errors as the backend cannot handle parallel load. Representative sampling maintains test coverage while significantly reducing CI execution time. Expected CI time savings: - domain-classify: 38m 38s → ~8m - jailbreak-detection: 39m 52s → ~10m - pii-detection: 14m 40s → ~3m 40s - Total: ~93 minutes → ~22 minutes (~71 minute savings) - Overall CI: 114 minutes → ~43 minutes (62% faster) Issue: #796 Signed-off-by: Yossi Ovadia <[email protected]>
1 parent d7d6a1c commit b63f368

File tree

3 files changed

+90
-1811
lines changed

3 files changed

+90
-1811
lines changed

0 commit comments

Comments
 (0)