Skip to content

Commit e131b1f

Browse files
Fix sketch tests - wait for dynamic canvas creation
- Added waitForDesignerReady() helper function - Canvas is created dynamically by designer.js, not in Razor markup - Tests now wait for #designer-stage div and canvas initialization - Use correct selector: '#designer-stage canvas' not 'canvas' - Added 3-second wait for JavaScript module to load - Updated canvas drawing tests (line, rect, circle) - Updated toolbar and canvas visibility tests This addresses the 44 failing sketch tests that were timing out because they expected immediate canvas availability.
1 parent df976a2 commit e131b1f

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

48 files changed

+951
-65
lines changed

TEST_FAILURE_ANALYSIS.md

Lines changed: 265 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
1+
# Test Failure Analysis - October 30, 2025
2+
3+
## Summary
4+
5+
**Total Tests Run:** 342 tests
6+
**Status:** 190 passed, 152 failed/skipped
7+
8+
### Pass Rate: **55.6%** (was 92.8% before)
9+
10+
---
11+
12+
## Critical Issues by Category
13+
14+
### 1. **Sketch/Designer Tests** - MAJOR REGRESSION ❌
15+
**Status:** 44/49 tests FAILING (89.8% failure rate)
16+
17+
All newly implemented sketch tests are timing out at 60 seconds, suggesting the Designer page is not functioning:
18+
19+
**Affected Tests:**
20+
- `navigate to sketches from job` - Timeout
21+
- `create new sketch from job` - Timeout
22+
- `designer toolbar loads correctly` - Timeout
23+
- `change sketch units` - Timeout
24+
- `set sketch name` - Timeout
25+
- `canvas loads and is interactive` - Timeout
26+
- All canvas drawing tests (line, rect, circle) - Timeout
27+
- All keyboard shortcut tests - Timeout
28+
- All export tests (PNG, JSON, SVG, DXF) - Timeout
29+
- All advanced feature tests - Timeout
30+
31+
**Only Passing:**
32+
-`discard draft` (2.3s)
33+
- A few basic tests that don't interact with designer
34+
35+
**Root Cause Hypothesis:**
36+
1. Designer page (`/designer/{jobId}`) may not load properly
37+
2. Canvas element not rendering
38+
3. Toolbar buttons not appearing with expected names
39+
4. JavaScript initialization failing
40+
5. WebAssembly not loading correctly for designer
41+
42+
**Action Required:** Verify Designer.razor exists and inspect actual page structure
43+
44+
---
45+
46+
### 2. **Accounting Tests** - ALL SKIPPED ⚠️
47+
**Status:** 24/24 tests SKIPPED
48+
49+
All accounting-ui.spec.ts tests are skipped (marked with `-`):
50+
- accounting tab visibility
51+
- dashboard navigation
52+
- statistics cards
53+
- companies page
54+
- invoices page
55+
- reports page
56+
- chart of accounts
57+
58+
**Root Cause:** Tests may be conditionally skipped or accounting feature not enabled
59+
60+
---
61+
62+
### 3. **Ledger Integration Tests** - TIMEOUTS ❌
63+
**Status:** 9/15 tests FAILING (60% failure rate)
64+
65+
Multiple ledger tests timing out at 22-23 seconds:
66+
67+
**Failing:**
68+
- ✘ journal entries page loads with company selector (22.8s timeout)
69+
- ✘ trial balance page shows company selector (22.5s timeout)
70+
- ✘ profit and loss page has company selector (22.5s timeout)
71+
- ✘ VAT summary page shows MTD boxes (22.5s timeout)
72+
- ✘ ledger invoices page has create modal (22.5s timeout)
73+
- ✘ all ledger pages handle empty company list (22.5s timeout)
74+
75+
**Passing:**
76+
- ✅ trial balance shows balanced/unbalanced status (2.3s)
77+
- ✅ profit and loss shows revenue and expenses (2.3s)
78+
- ✅ VAT summary shows MTD information box (2.3s)
79+
- ✅ invoice create modal has line item functionality (2.2s)
80+
- ✅ invoice modal can be closed (2.2s)
81+
- ✅ export CSV buttons present (4.4s)
82+
83+
**Pattern:** Tests that check for specific UI elements timeout, but tests that verify data/content pass quickly
84+
85+
**Root Cause Hypothesis:** Company selector dropdown not appearing or taking too long to load
86+
87+
---
88+
89+
### 4. **Accounting Blazor Tests** - TIMEOUTS ❌
90+
**Status:** 3/3 tests FAILING (100% failure rate)
91+
92+
- ✘ accounting dashboard loads and shows stats (60s timeout)
93+
- ✘ can navigate to companies page (60s timeout)
94+
- ✘ companies page shows create modal (60s+ timeout)
95+
96+
**Root Cause:** Similar to ledger tests - accounting pages not loading within timeout
97+
98+
---
99+
100+
### 5. **Onboarding Tests** - SKIPPED ⚠️
101+
**Status:** 6 tests SKIPPED
102+
103+
Tests marked with `-`:
104+
- complete onboarding flow - save credentials
105+
- validation - prevents invalid VAT format
106+
- credentials persist across sessions
107+
- success message appears on save
108+
109+
**Root Cause:** Likely flaky or environment-dependent tests that are conditionally skipped
110+
111+
---
112+
113+
### 6. **API Tests** - 2 SKIPPED
114+
**Status:** Most passing, 2 skipped
115+
116+
- `-` dev reseed endpoint (chromium)
117+
- `-` dev reseed endpoint (webkit)
118+
119+
**Root Cause:** Dev reseed likely disabled in test environment
120+
121+
---
122+
123+
## Passing Test Categories ✅
124+
125+
### Strong Performance:
126+
-**API Tests:** 95%+ pass rate
127+
-**CORS Tests:** 100% pass rate
128+
-**UI Smoke Tests:** 100% pass rate
129+
-**Onboarding UI Tests:** 90%+ pass rate
130+
-**Dashboard Tests:** 100% pass rate
131+
-**Invoice Tests:** 100% pass rate
132+
-**Workflow Tests:** 100% pass rate
133+
134+
---
135+
136+
## Immediate Action Items
137+
138+
### Priority 1: Sketch/Designer Tests (CRITICAL)
139+
140+
1. **Verify Designer page exists:**
141+
```bash
142+
# Check if file exists
143+
ls -la ai_mate_blazor/Pages/Designer.razor
144+
```
145+
146+
2. **Inspect actual page structure:**
147+
- Visit `/designer/{valid-job-id}` in browser
148+
- Check browser console for errors
149+
- Verify canvas element exists
150+
- Check button names match test expectations
151+
152+
3. **Common Issues to Check:**
153+
- Is WebAssembly loading?
154+
- Are JavaScript files loaded?
155+
- Canvas rendering properly?
156+
- Button labels exact matches ("Line" vs "line")?
157+
- Tool buttons using `<button>` vs `<a>` tags?
158+
159+
4. **Quick Fix Test:**
160+
Run single test in headed mode to see what's happening:
161+
```bash
162+
npm run test:e2e:headed -- tests/ui.sketches.spec.ts -g "designer toolbar"
163+
```
164+
165+
### Priority 2: Ledger Tests
166+
167+
1. **Check company selector:**
168+
- Visit ledger pages in browser
169+
- Verify company dropdown appears
170+
- Check load time
171+
172+
2. **Increase timeout if needed:**
173+
```typescript
174+
await page.waitForLoadState('networkidle', { timeout: 30000 });
175+
```
176+
177+
### Priority 3: Accounting Tests
178+
179+
1. **Determine why tests are skipped:**
180+
- Check test file for `.skip()` or conditional logic
181+
- Verify accounting feature is enabled
182+
183+
2. **Enable and run:**
184+
```bash
185+
npm run test:e2e -- tests/accounting-ui.spec.ts
186+
```
187+
188+
---
189+
190+
## Test Stability Recommendations
191+
192+
### 1. Add Debug Logging
193+
```typescript
194+
test('failing test', async ({ page }) => {
195+
console.log('Navigating to designer...');
196+
await page.goto('/designer/123');
197+
198+
console.log('Waiting for page load...');
199+
await page.waitForLoadState('networkidle');
200+
201+
console.log('Looking for canvas...');
202+
const canvas = page.locator('canvas').first();
203+
console.log('Canvas found:', await canvas.isVisible());
204+
});
205+
```
206+
207+
### 2. Take Screenshots on Failure
208+
Add to `playwright.config.ts`:
209+
```typescript
210+
use: {
211+
screenshot: 'only-on-failure',
212+
video: 'retain-on-failure',
213+
}
214+
```
215+
216+
### 3. Increase Timeouts for Slow Tests
217+
```typescript
218+
test.setTimeout(90000); // 90 seconds for slow tests
219+
```
220+
221+
### 4. Add Retry Logic for Flaky Selectors
222+
```typescript
223+
await expect(page.locator('button', { name: 'Save' }))
224+
.toBeVisible({ timeout: 30000 });
225+
```
226+
227+
---
228+
229+
## Next Steps
230+
231+
1. **Start backend and frontend:**
232+
```bash
233+
./scripts/start.sh
234+
```
235+
236+
2. **Manually test designer page:**
237+
- Create a job
238+
- Navigate to sketches
239+
- Try to create new sketch
240+
- Verify page loads and tools work
241+
242+
3. **Run single failing test in headed mode:**
243+
```bash
244+
npm run test:e2e:headed -- tests/ui.sketches.spec.ts -g "navigate to sketches"
245+
```
246+
247+
4. **Check browser console for errors**
248+
249+
5. **Fix designer page issues first** (biggest impact - 44 tests)
250+
251+
6. **Then tackle ledger timeouts** (9 tests)
252+
253+
7. **Finally investigate skipped tests** (30+ tests)
254+
255+
---
256+
257+
## Expected Improvement
258+
259+
If we fix the Designer page:
260+
- **Current:** 190/342 passed (55.6%)
261+
- **After Designer fix:** 234/342 passed (68.4%)
262+
- **After Ledger fix:** 243/342 passed (71.1%)
263+
- **After enabling skipped:** ~273/342 passed (79.8%)
264+
265+
**Target:** 90%+ pass rate (308+ tests passing)
0 Bytes
Binary file not shown.
0 Bytes
Binary file not shown.

ai_mate_blazor/bin/Debug/net9.0/ai_mate_blazor.staticwebassets.endpoints.json

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

ai_mate_blazor/bin/Debug/net9.0/ai_mate_blazor.staticwebassets.runtime.json

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

ai_mate_blazor/bin/Debug/net9.0/wwwroot/_framework/blazor.boot.json

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"mainAssemblyName": "ai_mate_blazor",
33
"resources": {
4-
"hash": "sha256-NAMeWOKjbnDRjxzVe0NO8ppEzvl+qlIkqFlHCMyB32c=",
4+
"hash": "sha256-95dWszHMwY0H55OJ6ElwfAo1s8tbFp3QEcITyi57RGo=",
55
"fingerprinting": {
66
"Microsoft.AspNetCore.Authorization.tnyc7xpluy.wasm": "Microsoft.AspNetCore.Authorization.wasm",
77
"Microsoft.AspNetCore.Components.t32damlu4o.wasm": "Microsoft.AspNetCore.Components.wasm",
@@ -201,8 +201,8 @@
201201
"icudt_CJK.tjcz0u77k5.dat": "icudt_CJK.dat",
202202
"icudt_EFIGS.tptq2av103.dat": "icudt_EFIGS.dat",
203203
"icudt_no_CJK.lfu7j35m59.dat": "icudt_no_CJK.dat",
204-
"ai_mate_blazor.pkr4c7ahfg.wasm": "ai_mate_blazor.wasm",
205-
"ai_mate_blazor.z5mhoqt34z.pdb": "ai_mate_blazor.pdb"
204+
"ai_mate_blazor.ojnxkfvvf5.wasm": "ai_mate_blazor.wasm",
205+
"ai_mate_blazor.dwsl6trg2v.pdb": "ai_mate_blazor.pdb"
206206
},
207207
"jsModuleNative": {
208208
"dotnet.native.50iqa8w3ys.js": "sha256-pQdXuxfVeqYHOwd5HUM2ERd7OnXirj9JTF37b6OdTAc="
@@ -412,10 +412,10 @@
412412
"WindowsBase.361km0836k.wasm": "sha256-FacUCEIJYTnbOWgX/R9QP2QnurqRZwRIkDAH0N9z69s=",
413413
"mscorlib.qiu3j3msqk.wasm": "sha256-QvztpxTOEw4aMIl7zP3QtsPLNv/IWaK5wq22J4ReGrA=",
414414
"netstandard.lp03u0xxi4.wasm": "sha256-WVQsxULiAiEbyXLKolS7txnWVT82SsvsuMe4jJ+SqqI=",
415-
"ai_mate_blazor.pkr4c7ahfg.wasm": "sha256-Z7Omq0WXR9rJluHYZJZInFE6qT1A2+XZgLNj5Of1vzM="
415+
"ai_mate_blazor.ojnxkfvvf5.wasm": "sha256-dNXnPkKtgd8+DVJtGYquFwpMkhOkymFDav/LqU7AVUc="
416416
},
417417
"pdb": {
418-
"ai_mate_blazor.z5mhoqt34z.pdb": "sha256-d9on+aMeY1UI1YPNMPsBxsK8w72T68qKQElT7lRhpQc="
418+
"ai_mate_blazor.dwsl6trg2v.pdb": "sha256-jY8W+QhBxnluh/gByNbk3Wp2Soe05gtMILeXbJyUt5U="
419419
}
420420
},
421421
"cacheBootResources": true,
Binary file not shown.

ai_mate_blazor/bin/Debug/net9.0/wwwroot/service-worker-assets.js

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
self.assetsManifest = {
2-
"version": "2Fpl+6jV",
2+
"version": "v/2cbzmu",
33
"assets": [
44
{
55
"hash": "sha256-HPC6AdTYY+y+nGoZaYsDtUdttnjFfcfmkkn/91QRzF8=",
@@ -766,15 +766,15 @@ self.assetsManifest = {
766766
"url": "_framework/WindowsBase.361km0836k.wasm"
767767
},
768768
{
769-
"hash": "sha256-Z7Omq0WXR9rJluHYZJZInFE6qT1A2+XZgLNj5Of1vzM=",
770-
"url": "_framework/ai_mate_blazor.pkr4c7ahfg.wasm"
769+
"hash": "sha256-jY8W+QhBxnluh/gByNbk3Wp2Soe05gtMILeXbJyUt5U=",
770+
"url": "_framework/ai_mate_blazor.dwsl6trg2v.pdb"
771771
},
772772
{
773-
"hash": "sha256-d9on+aMeY1UI1YPNMPsBxsK8w72T68qKQElT7lRhpQc=",
774-
"url": "_framework/ai_mate_blazor.z5mhoqt34z.pdb"
773+
"hash": "sha256-dNXnPkKtgd8+DVJtGYquFwpMkhOkymFDav/LqU7AVUc=",
774+
"url": "_framework/ai_mate_blazor.ojnxkfvvf5.wasm"
775775
},
776776
{
777-
"hash": "sha256-qmpHp4fHw0wHuXNW3NnWJirU8XkR4FklOkipnkjwqJw=",
777+
"hash": "sha256-ShZyo3WLi9HvqmaBmwJayBQzBz4znOtuacddnOG0UUw=",
778778
"url": "_framework/blazor.boot.json"
779779
},
780780
{
@@ -834,7 +834,7 @@ self.assetsManifest = {
834834
"url": "appsettings.json"
835835
},
836836
{
837-
"hash": "sha256-/7y+ahLqo3DUPX99bHc0eqlXYVdXN5eAiAdOlDPVjBM=",
837+
"hash": "sha256-uyurN+CEp5SP05csE+9X7Aev2zaEv7XQ7eCQNNyniCg=",
838838
"url": "css/app.css"
839839
},
840840
{
@@ -1049,6 +1049,10 @@ self.assetsManifest = {
10491049
"hash": "sha256-+rSbNw5rFeqxB4zdmkY3wTDGRxdGAqauZpAzSIFyoXk=",
10501050
"url": "lib/bootstrap/dist/js/bootstrap.min.js.map"
10511051
},
1052+
{
1053+
"hash": "sha256-ko+OTN5utoyP+rLW35qp2IOt0qXrLMYr8U3mePttdZE=",
1054+
"url": "manifest.json"
1055+
},
10521056
{
10531057
"hash": "sha256-+3v+gcIsSzK1xCyi/FW1mEQEf6AuqgWQHhnB5fF3dwE=",
10541058
"url": "manifest.webmanifest"
Binary file not shown.

ai_mate_blazor/bin/Debug/net9.0/wwwroot/service-worker.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
/* Manifest version: 2Fpl+6jV */
1+
/* Manifest version: v/2cbzmu */
22
// In development, always fetch from the network and do not enable offline support.
33
// This is because caching would make development more difficult (changes would not
44
// be reflected on the first load after each change).

0 commit comments

Comments
 (0)