Skip to content

Commit fb1724f

Browse files
CopilotCopilot
andcommitted
squad: update Lucius history with performance baseline notes
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 7b73ed7 commit fb1724f

File tree

1 file changed

+19
-1
lines changed

1 file changed

+19
-1
lines changed

.squad/agents/lucius/history.md

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,25 @@ Tests under `tests/` mirror the src structure.
2424

2525
## Learnings
2626

27-
<!-- Append new learnings below. Each entry is something lasting about the project. -->
27+
<!-- Append new learnings below. Each entry is something lasting about the project. -->
28+
29+
### Performance Baseline: Multi-Scenario Capture (2026-03-23)
30+
31+
**Task:** Capture and commit NBomber performance baselines for all load test scenarios.
32+
33+
**Root Cause:** Previous baseline.json only had get_transactions from an earlier stress test run. LoadTests.cs has 4 scenarios (accounts, budgets, calendar, transactions). The other three showed as "New" in every CI comparison — no regression detection.
34+
35+
**CI Bug Discovered:** performance.yml used ail -1 to pick the alphabetically-last CSV file. With multiple load test CSVs (load_accounts.csv, load_budgets.csv, load_calendar.csv, load_transactions.csv), only load_transactions.csv was ever compared. Fixed by looping over all CSVs.
36+
37+
**NBomber Behavior:** Each NBomber runner instance cleans/overwrites the reports folder on init. Running 4 sequential load tests = only the last test's CSV survives. Metrics must be extracted from console output for previous runs. The aselines/README.md instructions are correct — use the BaselineComparer --generate mode with the CSV artifact.
38+
39+
**Metrics Captured (dev hardware, in-memory DB, LoadProfile 10 req/s 35s):**
40+
- get_accounts: p95=0.66ms, p99=0.84ms, RPS=9.14
41+
- get_budgets: p95=0.76ms, p99=1.54ms, RPS=9.14
42+
- get_calendar: p95=12.02ms, p99=15.53ms, RPS=9.14
43+
- get_transactions: p95=11.61ms, p99=30.43ms, RPS=9.14
44+
45+
**Decision:** Thresholds already correct (maxLatencyRegressionPercent=15%). No change needed to ThresholdConfig defaults.
2846

2947
## Core Context
3048

0 commit comments

Comments
 (0)