Skip to content

Commit 38ffdc9

Browse files
Copilotkobenguyent
andcommitted
Fix test statistics reporting issue in pool mode - consolidate results properly to prevent duplicate counting
Co-authored-by: kobenguyent <[email protected]>
1 parent 3c55c3e commit 38ffdc9

File tree

5 files changed

+755
-11
lines changed

5 files changed

+755
-11
lines changed

debug_output.log

Lines changed: 303 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,303 @@
1+
CodeceptJS v3.7.4 #StandWithUkraine
2+
Running tests in 2 workers...
3+
4+
bootstrap b1+bootstrap b1+b2b2CodeceptJS v3.7.4 #StandWithUkraine
5+
CodeceptJS v3.7.4 #StandWithUkraine
6+
Using test root "/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox"
7+
Helpers: FileSystem, Workers
8+
Plugins: screenshotOnFail
9+
10+
Using test root "/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox"
11+
Helpers: FileSystem, Workers
12+
Plugins: screenshotOnFail
13+
14+
@feature_grep in worker --
15+
/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox/workers/test_grep.workers.js
16+
Warning: Timeout was set to 10000secs.
17+
Global timeout should be specified in seconds.
18+
From worker @2_grep print message 2
19+
@feature_grep in worker --
20+
/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox/workers/test_grep.workers.js
21+
Warning: Timeout was set to 10000secs.
22+
Global timeout should be specified in seconds.
23+
From worker @1_grep print message 1
24+
[01] ✔ From worker @2_grep print message 2 in 7ms
25+
[02] ✔ From worker @1_grep print message 1 in 7ms
26+
› Test Timeout: 10000s
27+
› [Dir] /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox
28+
Scenario()
29+
message 2
30+
I see this is worker
31+
✔ OK in 7ms
32+
33+
› Test Timeout: 10000s
34+
› [Dir] /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox
35+
Scenario()
36+
message 1
37+
I see this is worker
38+
✔ OK in 7ms
39+
40+
41+
42+
OK | 1 passed // 13ms
43+
[DEBUG] Worker 2 test result stats: {
44+
"passes": 1,
45+
"failures": 0,
46+
"tests": 1,
47+
"pending": 0,
48+
"failedHooks": 0,
49+
"start": "2025-08-22T16:52:22.041Z",
50+
"end": "2025-08-22T16:52:22.054Z",
51+
"duration": 13
52+
}
53+
[DEBUG] Worker 2 result tests count: 1
54+
[DEBUG] Worker 2 result failures count: 0
55+
OK | 1 passed // 14ms
56+
[DEBUG] Worker 1 test result stats: {
57+
"passes": 1,
58+
"failures": 0,
59+
"tests": 1,
60+
"pending": 0,
61+
"failedHooks": 0,
62+
"start": "2025-08-22T16:52:22.041Z",
63+
"end": "2025-08-22T16:52:22.055Z",
64+
"duration": 14
65+
}
66+
[DEBUG] Worker 1 result tests count: 1
67+
[DEBUG] Worker 1 result failures count: 0
68+
CodeceptJS v3.7.4 #StandWithUkraine
69+
Using test root "/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox"
70+
Helpers: FileSystem, Workers
71+
Plugins: screenshotOnFail
72+
73+
Retry Workers --
74+
/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox/workers/retry_test.workers.js
75+
Warning: Timeout was set to 10000secs.
76+
Global timeout should be specified in seconds.
77+
retry a test
78+
CodeceptJS v3.7.4 #StandWithUkraine
79+
Using test root "/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox"
80+
Helpers: FileSystem, Workers
81+
Plugins: screenshotOnFail
82+
83+
Workers Failing --
84+
/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox/workers/failing_test.worker.js
85+
Warning: Timeout was set to 10000secs.
86+
Global timeout should be specified in seconds.
87+
should not be executed
88+
› Test Timeout: 10000s
89+
[02] ✖ retry a test in 1ms
90+
› Test Timeout: 10000s
91+
› [Dir] /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox
92+
Scenario()
93+
› Test Timeout: 10000s
94+
[01] ✖ should not be executed in 0ms
95+
[02] ✔ retry a test in 2ms
96+
› [Dir] /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox
97+
Scenario()
98+
✔ OK in 1ms
99+
100+
101+
› [Dir] /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox
102+
Before()
103+
✖ FAILED in 2ms
104+
105+
OK | 2 passed // 5ms
106+
[DEBUG] Worker 2 test result stats: {
107+
"passes": 2,
108+
"failures": 0,
109+
"tests": 2,
110+
"pending": 0,
111+
"failedHooks": 0,
112+
"start": "2025-08-22T16:52:22.041Z",
113+
"end": "2025-08-22T16:52:22.063Z",
114+
"duration": 5
115+
}
116+
[DEBUG] Worker 2 result tests count: 3
117+
[DEBUG] Worker 2 result failures count: 0
118+
119+
-- FAILURES:
120+
121+
1) Workers Failing
122+
"before each" hook: Before for "should not be executed":
123+
124+
worker has failed
125+
at Context.<anonymous> (test/data/sandbox/workers/failing_test.worker.js:4:9)
126+
at promiseRetry.retries.retries (lib/mocha/asyncWrapper.js:165:20)
127+
at /home/runner/work/CodeceptJS/CodeceptJS/node_modules/promise-retry/index.js:29:24
128+
129+
130+
◯ File: file:///home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox/workers/failing_test.worker.js
131+
132+
133+
134+
FAIL | 1 passed, 1 failed, 1 failedHooks // 5ms
135+
Run with --verbose flag to see complete NodeJS stacktrace
136+
[DEBUG] Worker 1 test result stats: {
137+
"passes": 1,
138+
"failures": 1,
139+
"tests": 1,
140+
"pending": 0,
141+
"failedHooks": 1,
142+
"start": "2025-08-22T16:52:22.041Z",
143+
"end": "2025-08-22T16:52:22.064Z",
144+
"duration": 5
145+
}
146+
[DEBUG] Worker 1 result tests count: 2
147+
[DEBUG] Worker 1 result failures count: 1
148+
CodeceptJS v3.7.4 #StandWithUkraine
149+
Using test root "/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox"
150+
Helpers: FileSystem, Workers
151+
Plugins: screenshotOnFail
152+
153+
CodeceptJS v3.7.4 #StandWithUkraine
154+
Using test root "/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox"
155+
Helpers: FileSystem, Workers
156+
Plugins: screenshotOnFail
157+
158+
[02] ✔ say something in 1ms
159+
Workers --
160+
/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox/workers/base_test.workers.js
161+
Warning: Timeout was set to 10000secs.
162+
Global timeout should be specified in seconds.
163+
glob current dir
164+
Workers --
165+
/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox/workers/base_test.workers.js
166+
Warning: Timeout was set to 10000secs.
167+
Global timeout should be specified in seconds.
168+
say something
169+
› Test Timeout: 10000s
170+
› [Dir] /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox
171+
Scenario()
172+
I say "Hello Workers"
173+
Hello Workers
174+
I see this is worker
175+
✔ OK in 1ms
176+
177+
178+
OK | 3 passed // 3ms
179+
[DEBUG] Worker 2 test result stats: {
180+
"passes": 3,
181+
"failures": 0,
182+
"tests": 3,
183+
"pending": 0,
184+
"failedHooks": 0,
185+
"start": "2025-08-22T16:52:22.041Z",
186+
"end": "2025-08-22T16:52:22.068Z",
187+
"duration": 3
188+
}
189+
[DEBUG] Worker 2 result tests count: 4
190+
[DEBUG] Worker 2 result failures count: 0
191+
[01] ✔ glob current dir in 2ms
192+
› Test Timeout: 10000s
193+
› [Dir] /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox
194+
Scenario()
195+
I am in path "."
196+
› [Dir] /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox
197+
I say "hello world"
198+
hello world
199+
I see this is worker
200+
I see file "codecept.glob.js"
201+
› [File] /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox/codecept.glob.js
202+
✔ OK in 2ms
203+
204+
CodeceptJS v3.7.4 #StandWithUkraine
205+
Using test root "/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox"
206+
Helpers: FileSystem, Workers
207+
Plugins: screenshotOnFail
208+
209+
210+
Workers --
211+
/home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox/workers/base_test.workers.js
212+
Warning: Timeout was set to 10000secs.
213+
Global timeout should be specified in seconds.
214+
fail a test
215+
-- FAILURES:
216+
217+
218+
FAIL | 2 passed, 1 failed, 1 failedHooks // 3ms
219+
Run with --verbose flag to see complete NodeJS stacktrace
220+
[DEBUG] Worker 1 test result stats: {
221+
"passes": 2,
222+
"failures": 1,
223+
"tests": 2,
224+
"pending": 0,
225+
"failedHooks": 1,
226+
"start": "2025-08-22T16:52:22.041Z",
227+
"end": "2025-08-22T16:52:22.070Z",
228+
"duration": 3
229+
}
230+
[DEBUG] Worker 1 result tests count: 3
231+
[DEBUG] Worker 1 result failures count: 1
232+
[DEBUG] Worker 1 final consolidated stats: {
233+
"passes": 4,
234+
"failures": 2,
235+
"tests": 4,
236+
"pending": 0,
237+
"failedHooks": 2
238+
}
239+
[DEBUG] Worker 1 final result tests count: 6
240+
[DEBUG] Worker 1 final result failures count: 2
241+
[02] ✖ fail a test in 2ms
242+
› Test Timeout: 10000s
243+
› [Dir] /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox
244+
Scenario()
245+
I am in path "."
246+
› [Dir] /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox
247+
I see this is worker
248+
I see file "notafile"
249+
› [File] /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox/notafile
250+
✖ FAILED in 2ms
251+
252+
253+
-- FAILURES:
254+
255+
1) Workers
256+
fail a test:
257+
258+
259+
File notafile not found in /home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox
260+
+ expected - actual
261+
262+
-false
263+
+true
264+
265+
AssertionError [ERR_ASSERTION]:
266+
at FileSystem.seeFile (lib/helper/FileSystem.js:70:12)
267+
at HelperStep.run (lib/step/helper.js:28:49)
268+
269+
270+
◯ File: file:///home/runner/work/CodeceptJS/CodeceptJS/test/data/sandbox/workers/base_test.workers.js
271+
272+
◯ Scenario Steps:
273+
✖ I.seeFile("notafile") at Test.<anonymous> (./workers/base_test.workers.js:18:5)
274+
✔ I.seeThisIsWorker() at Test.<anonymous> (./workers/base_test.workers.js:17:5)
275+
✔ I.amInPath(".") at Test.<anonymous> (./workers/base_test.workers.js:16:5)
276+
277+
278+
279+
FAIL | 3 passed, 1 failed // 5ms
280+
Run with --verbose flag to see complete NodeJS stacktrace
281+
[DEBUG] Worker 2 test result stats: {
282+
"passes": 3,
283+
"failures": 1,
284+
"tests": 4,
285+
"pending": 0,
286+
"failedHooks": 0,
287+
"start": "2025-08-22T16:52:22.041Z",
288+
"end": "2025-08-22T16:52:22.075Z",
289+
"duration": 5
290+
}
291+
[DEBUG] Worker 2 result tests count: 5
292+
[DEBUG] Worker 2 result failures count: 1
293+
[DEBUG] Worker 2 final consolidated stats: {
294+
"passes": 9,
295+
"failures": 1,
296+
"tests": 10,
297+
"pending": 0,
298+
"failedHooks": 0
299+
}
300+
[DEBUG] Worker 2 final result tests count: 13
301+
[DEBUG] Worker 2 final result failures count: 1
302+
303+
OK | 0 passed // 462ms

docs/plugins.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,10 @@
1+
---
2+
permalink: plugins
3+
sidebarDepth:
4+
sidebar: auto
5+
title: Plugins
6+
---
7+
18
<!-- Generated by documentation.js. Update this documentation by updating the source code. -->
29

310
## analyze

0 commit comments

Comments
 (0)