feat(test runner): server side mocking

[Skn0tt]

Implements #30766.

Adds an HTTP proxy that application servers can route their network traffic through to make it subject to Playwright routes.

[vitalets]

Hi @Skn0tt @dgozman
I've recently discovered that Next.js has an experimental flag to mock server-side requests with Playwright:
https://github.com/vercel/next.js/blob/canary/packages/next/src/experimental/testmode/playwright/README.md

Just curios, how does it relates to this PR? Seems like they provide own fixtures for running proxy as well..

[Skn0tt]

Just curios, how does it relates to this PR? Seems like they provide own fixtures for running proxy as well..

Their experiment tries to solve the same problem in a very similar manner to this. It doesn't look like a lot happened to that experiment in Next.js, and it's not even in the docs. So I could imagine that they'll recommend our mocking proxy instead.

[Skn0tt]

cc @dvoytenko @ijjk for visibility

[MindaugasMateika]

Hi @Skn0tt @dgozman I've recently discovered that Next.js has an experimental flag to mock server-side requests with Playwright: https://github.com/vercel/next.js/blob/canary/packages/next/src/experimental/testmode/playwright/README.md

Just curios, how does it relates to this PR? Seems like they provide own fixtures for running proxy as well..

Third-party implementations, such as msw, playwright-msw and others, have always been available for those who wanted server-side mocking. However, it doesn't compare with having server-side mocking built directly into Playwright which will offer parallel support, bug fixes, and most imporantly peace of mind knowing it will remain working with future releases. My 2 cents.

[Skn0tt]

From my understanding of playwright-msw, it's about mocking browser requests. This work is about mocking server requests, which is different.

[github-actions]

Test results for "tests 1"

1 failed
❌ [playwright-test] › tests/reporter-html.spec.ts:2680:5 › merged › execSync doesnt produce a second stdout attachment @macos-latest-node18-2

13 flaky

⚠️ [firefox-page] › tests/page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @firefox-ubuntu-22.04-node18
⚠️ [webkit-library] › tests/library/browsercontext-proxy.spec.ts:27:3 › should work when passing the proxy only on the context level @webkit-ubuntu-22.04-node18
⚠️ [webkit-library] › tests/library/browsertype-connect.spec.ts:554:5 › launchServer › should error when saving download after deletion @webkit-ubuntu-22.04-node18
⚠️ [webkit-library] › tests/library/inspector/cli-codegen-2.spec.ts:24:7 › cli codegen › should contain open page @webkit-ubuntu-22.04-node18
⚠️ [webkit-library] › tests/library/proxy.spec.ts:93:11 › should proxy local network requests › with other bypasses › link-local @webkit-ubuntu-22.04-node18
⚠️ [webkit-library] › tests/library/proxy.spec.ts:228:3 › should use socks proxy @webkit-ubuntu-22.04-node18
⚠️ [webkit-library] › tests/library/proxy.spec.ts:238:3 › should use socks proxy in second page @webkit-ubuntu-22.04-node18
⚠️ [webkit-library] › tests/library/selector-generator.spec.ts:129:5 › selector generator › should use ordinal for identical nodes @webkit-ubuntu-22.04-node18
⚠️ [webkit-library] › tests/library/selector-generator.spec.ts:437:5 › selector generator › should ignore empty aria-label for candidate consideration @webkit-ubuntu-22.04-node18
⚠️ [webkit-library] › tests/library/trace-viewer.spec.ts:109:1 › should show tracing.group in the action list with location @webkit-ubuntu-22.04-node18
⚠️ [webkit-page] › tests/page/page-leaks.spec.ts:82:5 › click should not leak @webkit-ubuntu-22.04-node18
⚠️ [webkit-page] › tests/page/page-leaks.spec.ts:107:5 › fill should not leak @webkit-ubuntu-22.04-node18
⚠️ [webkit-page] › tests/page/page-leaks.spec.ts:161:5 › waitFor should not leak @webkit-ubuntu-22.04-node18

37848 passed, 655 skipped
✔️✔️✔️

Merge workflow run.

[schrufygroovy]

@Skn0tt could you please point me to the critical path, where you are ensuring that "test isolation" is still possible?
With "test isolation" I mean:

• 2 different unrelated tests
• each of those tests are setting up for instance different responses for the same URL pattern
• when those tests are executed in parallel
  how are you ensuring that the setup responses of both tests are not interfering? That each test is running "in isolation"?

[Skn0tt]

There's a header added to all outgoing browser requests that contains correlation data on what test the request originates from: https://github.com/microsoft/playwright/pull/34520/files#diff-087773eea292da9db5a3f27de8f1a2940cdb895383ad750c3cd8e01772a35b40R373

[Skn0tt]

Work on this has gone stale b/c i'm prioritising different stuff currently. I'll get back to it.

[s-cochrane]

We're looking forward to this one once you're able to get it completed. We're currently using MSW, but the implementation is less than ideal. It will be really nice when we can handle this within PW instead.

[siggerzz]

Just echoing what others have said about a native solution. I'm in the process of evaluating MSW for our NextJS application, but I'm somewhat apprehensive given the NextJS implementation doesn't seem to be documented anywhere and has been in an experimental state for quite some time!