You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**Isolation:** Tests are stored outside the plugin directory so they're not auto-loaded with plugin content. The test agent runs in a separate session from the judge agent. Note: Full filesystem isolation isn't possible with CC—this is behavioral testing, not adversarial testing.
321
+
322
+
**Format:**
323
+
```
324
+
# Comment lines start with #
325
+
prompt text here|expected behavior description
326
+
another prompt|what the response should demonstrate
327
+
```
328
+
329
+
Each test runs the prompt against the plugin, then uses a separate CC instance as judge to evaluate if the response matches expected behavior.
330
+
331
+
**Model Strategy:**
332
+
- Test agent: Haiku first (fast/cheap) → Sonnet fallback on failure
333
+
- Judge agent: Always Sonnet (reliable judgment)
334
+
- Bug reports generated at `tests/reports/` for failed tests
335
+
308
336
### Pre-Publish Checklist
309
337
310
338
Before committing a new plugin:
@@ -317,6 +345,7 @@ Before committing a new plugin:
0 commit comments