feat(e2e): simplify E2E config and add weekly sanity workflow by amitjoshi438 · Pull Request #1533 · microsoft/powerplatform-vscode

amitjoshi438 · 2026-04-02T14:02:34Z

What this PR does

This PR sets up Playwright E2E sanity tests for the VS Code Web extension — tests that open a real Power Pages site in VS Code Web (insiders.vscode.dev), authenticate via Microsoft login, and verify core functionality: extension activation, file tree loading, navigation, file operations, and file save.

E2E Test Coverage (5 specs)

Extension activation — VS Code Web loads and the Power Pages extension activates
Site tree — file explorer renders with site pages/content
Navigation — clicking tree items opens correct editors
File save — edit and save a file, verify persistence

How it works

Playwright opens insiders.vscode.dev with the test site URL
Handles Microsoft login automatically (username/password from secrets)
Caches auth state in .auth/storageState.json — subsequent runs skip login
Runs 5 sanity specs covering the core editing workflow

Secrets needed (repo Settings → Secrets → Actions)

Secret	Value
`PP_TEST_VSCODE_URL`	Full VS Code Web URL from browser
`PP_TEST_USERNAME`	Test account email
`PP_TEST_PASSWORD`	Test account password (MFA must be disabled)

How to run

# Locally (one-time setup)
cp src/e2e/.env.example src/e2e/.env
# Fill in the 3 values, then:
npm run test-e2e

# Via GitHub Actions
# Actions → E2E Sanity Tests → Run workflow

Test plan

Verify npm run test-e2e works locally with .env containing the 3 vars
Trigger E2E Sanity workflow manually via Actions → Run workflow
Confirm PullRequest.yml CI passes (no E2E job on non-release PRs)

🤖 Generated with GitHub Copilot

Automates manual web sanity test suite with Playwright tests that run against a real Power Platform environment. Includes tests for extension activation, site tree loading, file operations, file save/revert, and navigation. Adds a nightly GitHub Actions workflow (E2ESanity.yml). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Orchestration log: e2e test infrastructure review findings - Session log: Critical (13 hardcoded waits), major (POM, URL mismatch), minor issues - Decision inbox merged into decisions.md - Recommendations for test maintainability and CI integration Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Orchestration log: Keaton approved Hockney's e2e review with adjustments - Session log: Summary of verdict and P0/P1/P2 priority plan - Decision merge: Inbox entry consolidated into decisions.md with full action plan - Hockney history: Append Keaton lead decision and severity adjustments - Adjusted POM severity to Minor (premature for 5 specs) - Dismissed serial marker finding (config already enforces serial execution) - Elevated URL casing issue: test sends camelCase, enum is lowercase, production query failures likely Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Orchestration log: 2026-04-02T07:35:00Z Fenster P1 leaf-node detection + storageState caching - Session log: Brief summary of both fixes - Merged decision inbox: fenster-e2e-auth-caching.md → decisions.md - Updated hockney/history.md: P1 completion note - Updated mcmanus/history.md: storageState caching details Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Replace all 13 waitForTimeout() calls with condition-based waits - Fix URL builder param casing to match queryParameters enum (case-sensitive) - Fix vacuous assertion (toBeGreaterThanOrEqual(0) -> toBeGreaterThan(0)) - Improve auth error handling: swallow timeouts, propagate real errors - Replace brittle aria-expanded leaf-node detection with .monaco-tl-twistie.collapsible - Add storageState caching for auth reuse across test runs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Keep squad team state local-only — not committed to repo. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Moved squad exclusions to .git/info/exclude (machine-local) to keep .gitignore clean of non-project entries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Replace individual env vars (org URL, site IDs, tenant) with single PP_TEST_VSCODE_URL containing the full VS Code Web URL - Remove TestSiteConfig interface and buildVSCodeWebUrl helper - Simplify .env.example to 3 vars: URL, username, password - Trim E2ESanity.yml to 3 secrets, bump actions to v4 - Add --with-deps to Playwright install for CI runners - Add 'node' to e2e tsconfig types to fix process reference Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Change E2ESanity.yml schedule from nightly to weekly (Monday 2AM UTC) - Remove e2e-sanity job from PullRequest.yml - E2E tests run via E2ESanity.yml (weekly + manual dispatch only) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Adds a Playwright-based E2E “sanity” suite for the VS Code Web extension (insiders.vscode.dev) that authenticates via Microsoft login, opens a Power Pages site, and validates key UI flows; plus a scheduled/manual GitHub Actions workflow to run it regularly.

Changes:

Introduces Playwright E2E harness (fixtures, selectors, URL helper) and 5 sanity specs under src/e2e/.
Adds test-e2e npm script and Playwright dependency.
Adds a weekly + manual E2E Sanity Tests GitHub Actions workflow and ignores local E2E artifacts (.env, auth state, reports).

Reviewed changes

Copilot reviewed 13 out of 15 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`.github/workflows/E2ESanity.yml`	New scheduled/manual workflow to run Playwright E2E sanity tests and upload artifacts.
`.gitignore`	Ignores E2E local config/auth cache and Playwright outputs.
`package.json`	Adds `test-e2e` script and Playwright test dependency.
`package-lock.json`	Locks Playwright test dependency.
`src/e2e/.env.example`	Documents required env vars for local E2E runs.
`src/e2e/playwright.config.ts`	Playwright config, reporters, retries, and storageState reuse; loads `src/e2e/.env`.
`src/e2e/fixtures/vscode-web.fixture.ts`	Provides authenticated VS Code Web page fixture with login + optional auth caching.
`src/e2e/helpers/selectors.ts`	Centralized DOM selectors for VS Code Web + Microsoft login.
`src/e2e/helpers/url-builder.ts`	Reads `PP_TEST_VSCODE_URL` and errors if missing.
`src/e2e/tests/extension-activation.spec.ts`	Basic workbench/activation sanity assertions.
`src/e2e/tests/site-tree.spec.ts`	Verifies explorer renders and key entity folders appear.
`src/e2e/tests/navigation.spec.ts`	Confirms explorer presence and checks for error notifications.
`src/e2e/tests/file-operations.spec.ts`	Opens a file from the tree and verifies an editor tab appears.
`src/e2e/tests/file-save.spec.ts`	Edits, saves, undoes, and re-saves a file; checks for dialogs/errors.
`src/e2e/tsconfig.json`	TypeScript config for the E2E suite output/build.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.github/workflows/E2ESanity.yml

src/e2e/tests/file-save.spec.ts

src/e2e/tests/file-operations.spec.ts

src/e2e/fixtures/vscode-web.fixture.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

priyanshu92

Looks great overall. Let's keep pages specific tests in power-pages folder

priyanshu92 · 2026-04-02T15:02:53Z

.github/workflows/E2ESanity.yml

We can mention Web in the file name

Amit Joshi and others added 10 commits March 31, 2026 18:37

docs(.squad): log McManus P0 e2e fixes session

bc7927e

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

chore: remove .squad from git tracking and update .gitignore

2c1114a

Keep squad team state local-only — not committed to repo. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

chore: remove squad-related entries from .gitignore

d05d0b3

Moved squad exclusions to .git/info/exclude (machine-local) to keep .gitignore clean of non-project entries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

amitjoshi438 requested review from a team as code owners April 2, 2026 14:02

Amit Joshi and others added 2 commits April 2, 2026 19:35

chore: remove .omc from .gitignore

9463b57

Merge branch 'main' into users/amitjoshi/e2e-web-sanity

9b97398

amitjoshi438 requested a review from Copilot April 2, 2026 14:09

Copilot started reviewing on behalf of amitjoshi438 April 2, 2026 14:10 View session

Copilot AI reviewed Apr 2, 2026

View reviewed changes

Copilot stopped work on behalf of amitjoshi438 due to an error April 2, 2026 14:52
Before you can use Copilot coding agent, you need to pick a "Usage billed to" option in your Copilot settings. Update your settings at https://github.com/settings/copilot/features, then try again.

Update src/e2e/fixtures/vscode-web.fixture.ts

c9f59c7

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

priyanshu92 approved these changes Apr 2, 2026

View reviewed changes

priyanshu92 reviewed Apr 2, 2026

View reviewed changes

.github/workflows/E2ESanity.yml

Copy link
Copy Markdown

Contributor

priyanshu92 Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can mention Web in the file name

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(e2e): simplify E2E config and add weekly sanity workflow#1533

feat(e2e): simplify E2E config and add weekly sanity workflow#1533
amitjoshi438 wants to merge 13 commits intomainfrom
users/amitjoshi/e2e-web-sanity

amitjoshi438 commented Apr 2, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

priyanshu92 left a comment

Uh oh!

priyanshu92 Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

amitjoshi438 commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does

E2E Test Coverage (5 specs)

How it works

Secrets needed (repo Settings → Secrets → Actions)

How to run

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

priyanshu92 left a comment

Choose a reason for hiding this comment

Uh oh!

priyanshu92 Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

amitjoshi438 commented Apr 2, 2026 •

edited

Loading