[Bug]: Worker auto-fixture is nested into the first beforeAll hook

[vitalets]

Version

1.57.0

Steps to reproduce

Full minimal reproduction code.

1. Create a test file with worker-scoped auto fixture and beforeAll hook:

import { test as base } from '@playwright/test';

const test = base.extend<{}, { workerFixture: void }>({
  workerFixture: [async ({}, use) => {
    await new Promise((r) => setTimeout(r, 1000)); // delay 1000 ms
    await use();
  }, { scope: 'worker', auto: true }]
})

test.beforeAll(async () => {
  await new Promise((r) => setTimeout(r, 100)); // delay 100 ms
});

test('test 1', async ({}) => { });

2. Run the test and check the result in the reporter API onTestEnd() method.

Expected behavior

• worker fixture and beforeAll hook are placed on the same level in the result.steps array
• beforeAll hook duration is ~100 ms

hook Before Hooks (1122ms)
  hook beforeAll hook (117ms)
  fixture Fixture "workerFixture" (1001ms)
hook After Hooks (3ms)

Actual behavior

• worker fixture is nested under the beforeAll hook
• beforeAll hook duration contains worker fixture duration, in total ~1100 ms instead of ~100 ms

hook Before Hooks (1139ms)
  hook beforeAll hook (1134ms)
    fixture Fixture "workerFixture" (1002ms)
hook After Hooks (4ms)

Additional context

• If the worker fixture is not auto, the output structure is correct.
• if there is another identical beforeAll hook, it does not contain the worker fixture, and it's duration is correct:

hook Before Hooks (1263ms)
  hook beforeAll hook (1131ms)
    fixture Fixture "workerFixture" (1004ms)
  hook beforeAll hook (121ms)
hook After Hooks (5ms)

Environment

System:
    OS: macOS 26.2
    CPU: (10) arm64 Apple M1 Pro
    Memory: 90.27 MB / 16.00 GB
  Binaries:
    Node: 20.18.3 - /Users/vitalets/.nvm/versions/node/v20.18.3/bin/node
    Yarn: 3.4.1 - /Users/vitalets/.nvm/versions/node/v20.18.3/bin/yarn
    npm: 11.6.0 - /Users/vitalets/.nvm/versions/node/v20.18.3/bin/npm
    pnpm: 9.15.4 - /Users/vitalets/.nvm/versions/node/v20.18.3/bin/pnpm
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: ^1.57.0 => 1.57.0

[dgozman]

@vitalets Thank you for the issue! Somehow, it looks like working as intended to me. Perhaps I'm missing something.

If the worker fixture is not auto, the output structure is correct.

I was not able to repro this behavior. Here's the snippet I tried:

import { test as base } from '@playwright/test';

const test = base.extend<{}, { workerFixture: void }>({
  workerFixture: [async ({}, use) => {
    await new Promise((r) => setTimeout(r, 1000)); // delay 1000 ms
    await use();
  }, { scope: 'worker', auto: false }]
})

test.beforeAll(async ({ workerFixture }) => {
  await new Promise((r) => setTimeout(r, 100)); // delay 100 ms
});

test('test 1', async ({}) => { });

---

I assume you think that fixtures overall should not be a part of any hook? Today, they are placed inside the hook that triggered the fixture. Same goes for test/worker fixtures inside beforeEach, for example.

I guess we can reconsider the nesting, if that's what you are asking for.

[vitalets]

Somehow, it looks like it’s working as intended to me. Perhaps I’m missing something.

@dgozman my main concern is the duration. I’m building a custom reporter that highlights long-running hooks. With the current output, the duration of beforeAll is reported as 1100 ms, although it actually takes only 100 ms.

I was not able to repro this behavior.

Could you try the following: move workerFixture into the test dependency instead of beforeAll:

import { test as base } from '@playwright/test';

const test = base.extend<{}, { workerFixture: void }>({
  workerFixture: [async ({}, use) => {
    await new Promise((r) => setTimeout(r, 1000)); // delay 1000 ms
    await use();
  }, { scope: 'worker', auto: false }]
})

test.beforeAll(async () => {
  await new Promise((r) => setTimeout(r, 100)); // delay 100 ms
});

test('test 1', async ({ workerFixture }) => {});

Output:

hook Before Hooks (1124ms)
  hook beforeAll hook (117ms)
  fixture Fixture "workerFixture" (1002ms)
hook After Hooks (9ms)

I assume you think that fixtures overall should not be a part of any hook?

I think fixtures can be nested inside hooks if they are triggered directly (as you mentioned). Such a structure helps investigate why a fixture was executed. However, keeping the duration as a total of the hook’s own code and its dependencies is less useful. Maybe we could introduce a new field like selfDuration to show the execution time of the function itself. What do you think?

I guess we can reconsider the nesting, if that's what you are asking for.

Even with the current approach (where fixtures are nested inside the caller), I believe the current output is not accurate for auto fixtures. In my example, workerFixture is not referenced in the beforeAll hook, but it is still nested inside this hook in the reporter output.

[vitalets]

However, keeping the duration as a total of the hook’s own code and its dependencies is less useful. Maybe we could introduce a new field like selfDuration to show the execution time of the function itself.

I’ve realized that tracking the total duration is needed for proper handling of timeouts. I can calculate a hook’s self duration by subtracting fixture durations from the total time. For my example above, the beforeAll duration will be 1134 - 1002 = ~100ms, which is correct.

Two things still confuse me a bit:

1. The worker auto fixture is automatically attributed to the first beforeAll hook in a file (as shown in this issue). If there are no beforeAll hooks, the fixture will be attributed to the test timeout. This means the test can start failing when I simply add or remove a beforeAll hook.
   For example, this passes with a timeout of 1000ms:

   const test = base.extend<{}, { workerFixture: void }>({
     workerFixture: [async ({}, use) => {
       await delay(900);
       await use();
     }, { scope: 'worker', auto: true }]
   });
   
   test.beforeAll(() => {});
   
   test('test 1', async () => delay(200));

   but this fails:

   const test = base.extend<{}, { workerFixture: void }>({
     workerFixture: [async ({}, use) => {
       await delay(900);
       await use();
     }, { scope: 'worker', auto: true }]
   });
   
   // test.beforeAll(() => {}); // <-- commented out
   
   test('test 1', async () => delay(200));

   @dgozman, is this also expected behavior?

2. I wasn’t able to reliably calculate the test body self duration. If I simply subtract fixtures and hooks from result.duration, I can get a negative value. For example, given a test:

   test.beforeAll(() => delay(900));
   test.beforeEach(() => delay(300));
   
   test('test 1', () => delay(500));

   Output:

   test duration (805ms)
      hook Before Hooks (1234ms)
        hook beforeAll hook (924ms)
        hook beforeEach hook (307ms)
      hook After Hooks (9ms)
   

   It looks like result.duration does not include the beforeAll duration. I can exclude such hooks by name (e.g. /beforeAll/), but this is not reliable, as users can provide custom hook names. For example:

   test.beforeAll('hook 1', () => delay(900));
   test.beforeEach('hook 2', () => delay(300));
   
   test('test 1', () => delay(500));

   Output:

   test duration (804ms)
      hook Before Hooks (1238ms)
        hook hook 1 (928ms)
        hook hook 2 (308ms)
      hook After Hooks (6ms)
   

   There is no way to distinguish beforeAll from beforeEach here. Some kind of flag would be very useful. @dgozman, could you also comment on this? I can create a separate issue if needed.

[dgozman]

@dgozman, is this also expected behavior?

Yes, this is expected today. That said, we want to reconsider this behavior. Just need to be careful to avoid regressions.

There is no way to distinguish beforeAll from beforeEach here. Some kind of flag would be very useful. @dgozman, could you also comment on this? I can create a separate issue if needed.

Yep, I have also stumbled upon this need once or twice. However, there were no requests or questions until this point. Please file a separate issue so we can keep track of it.

[vitalets]

Yep, I have also stumbled upon this need once or twice. However, there were no requests or questions until this point. Please file a separate issue so we can keep track of it.

Done: #38747