Skip to content

Commit 747e212

Browse files
BYKcursoragent
BYK
and
cursoragent
authored
doc: E2E testing documentation updates (#18649)
Before submitting a pull request, please take a look at our [Contributing](https://github.com/getsentry/sentry-javascript/blob/master/CONTRIBUTING.md) guidelines and verify: - [x] If you've added code that should be tested, please add tests. - [x] Ensure your code lints and the test suite passes (`yarn lint`) & (`yarn test`). - [ ] Link an issue if there is one related to your pull request. If no issue is linked, one will be auto-generated and linked. Closes #issue_link_here This PR documents key learnings and best practices for E2E testing within the Sentry JavaScript SDKs, focusing on common pitfalls and setup requirements. **Why these changes?** To improve the developer experience for E2E testing by providing clear guidance on Verdaccio setup, the critical `.npmrc` file, bundler-specific behaviors, and debugging strategies, thereby reducing common issues and debugging time. **What was changed?** - **`CONTRIBUTING.md`**: Added a new section "Running E2E Tests Locally" with prerequisites, step-by-step instructions, and troubleshooting. - **`CLAUDE.md` & `.cursor/rules/sdk_development.mdc`**: Added an "E2E Testing" section under "Development Guidelines," explaining Verdaccio, `.npmrc` requirements, how to run single tests, and common pitfalls. - **`dev-packages/e2e-tests/README.md`**: Expanded existing documentation with detailed `.npmrc` explanations, a comprehensive troubleshooting guide, and notes on bundler-specific environment variable handling (Webpack, Vite, Next.js) and `import.meta.env` behavior. --- <a href="https://cursor.com/background-agent?bcId=bc-d15ff251-4280-4c73-a3a1-074dc448417c"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/open-in-cursor-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/open-in-cursor-light.svg"><img alt="Open in Cursor" src="https://cursor.com/open-in-cursor.svg"></picture></a>&nbsp;<a href="https://cursor.com/agents?id=bc-d15ff251-4280-4c73-a3a1-074dc448417c"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/open-in-web-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://cursor.com/open-in-web-light.svg"><img alt="Open in Web" src="https://cursor.com/open-in-web.svg"></picture></a> Closes #18651 (added automatically) --------- Co-authored-by: Cursor Agent <[email protected]>
1 parent 4fd5021 commit 747e212

File tree

4 files changed

+254
-0
lines changed

4 files changed

+254
-0
lines changed

‎.cursor/rules/sdk_development.mdc‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,54 @@ Each package typically contains:
121121
- Integration tests use Playwright extensively
122122
- Never change the volta, yarn, or package manager setup in general unless explicitly asked for
123123

124+
### E2E Testing
125+
126+
E2E tests are located in `dev-packages/e2e-tests/` and verify SDK behavior in real-world framework scenarios.
127+
128+
#### How Verdaccio Registry Works
129+
130+
E2E tests use [Verdaccio](https://verdaccio.org/), a lightweight npm registry running in Docker. Before tests run:
131+
132+
1. SDK packages are built and packed into tarballs (`yarn build && yarn build:tarball`)
133+
2. Tarballs are published to Verdaccio at `http://127.0.0.1:4873`
134+
3. Test applications install packages from Verdaccio instead of public npm
135+
136+
#### The `.npmrc` Requirement
137+
138+
Every E2E test application needs an `.npmrc` file with:
139+
140+
```
141+
@sentry:registry=http://127.0.0.1:4873
142+
@sentry-internal:registry=http://127.0.0.1:4873
143+
```
144+
145+
Without this file, pnpm installs from the public npm registry instead of Verdaccio, so your local changes won't be tested. This is a common cause of "tests pass in CI but fail locally" or vice versa.
146+
147+
#### Running a Single E2E Test
148+
149+
```bash
150+
# Build packages first
151+
yarn build && yarn build:tarball
152+
153+
# Run a specific test app
154+
cd dev-packages/e2e-tests
155+
yarn test:run <app-name>
156+
157+
# Run with a specific variant (e.g., Next.js 15)
158+
yarn test:run <app-name> --variant <variant-name>
159+
```
160+
161+
#### Common Pitfalls and Debugging
162+
163+
1. **Missing `.npmrc`**: Most common issue. Always verify the test app has the correct `.npmrc` file.
164+
165+
2. **Stale tarballs**: After SDK changes, must re-run `yarn build:tarball`.
166+
167+
3. **Debugging tips**:
168+
- Check browser console logs for SDK initialization errors
169+
- Use `debug: true` in Sentry config
170+
- Verify installed package version: check `node_modules/@sentry/*/package.json`
171+
124172
### Notes for Background Tasks
125173

126174
- Make sure to use [volta](https://volta.sh/) for development. Volta is used to manage the node, yarn and pnpm version used.

‎CLAUDE.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,54 @@ Each package typically contains:
120120
- Integration tests use Playwright extensively
121121
- Never change the volta, yarn, or package manager setup in general unless explicitly asked for
122122

123+
### E2E Testing
124+
125+
E2E tests are located in `dev-packages/e2e-tests/` and verify SDK behavior in real-world framework scenarios.
126+
127+
#### How Verdaccio Registry Works
128+
129+
E2E tests use [Verdaccio](https://verdaccio.org/), a lightweight npm registry running in Docker. Before tests run:
130+
131+
1. SDK packages are built and packed into tarballs (`yarn build && yarn build:tarball`)
132+
2. Tarballs are published to Verdaccio at `http://127.0.0.1:4873`
133+
3. Test applications install packages from Verdaccio instead of public npm
134+
135+
#### The `.npmrc` Requirement
136+
137+
Every E2E test application needs an `.npmrc` file with:
138+
139+
```
140+
@sentry:registry=http://127.0.0.1:4873
141+
@sentry-internal:registry=http://127.0.0.1:4873
142+
```
143+
144+
Without this file, pnpm installs from the public npm registry instead of Verdaccio, so your local changes won't be tested. This is a common cause of "tests pass in CI but fail locally" or vice versa.
145+
146+
#### Running a Single E2E Test
147+
148+
```bash
149+
# Build packages first
150+
yarn build && yarn build:tarball
151+
152+
# Run a specific test app
153+
cd dev-packages/e2e-tests
154+
yarn test:run <app-name>
155+
156+
# Run with a specific variant (e.g., Next.js 15)
157+
yarn test:run <app-name> --variant <variant-name>
158+
```
159+
160+
#### Common Pitfalls and Debugging
161+
162+
1. **Missing `.npmrc`**: Most common issue. Always verify the test app has the correct `.npmrc` file.
163+
164+
2. **Stale tarballs**: After SDK changes, must re-run `yarn build:tarball`.
165+
166+
3. **Debugging tips**:
167+
- Check browser console logs for SDK initialization errors
168+
- Use `debug: true` in Sentry config
169+
- Verify installed package version: check `node_modules/@sentry/*/package.json`
170+
123171
### Notes for Background Tasks
124172

125173
- Make sure to use [volta](https://volta.sh/) for development. Volta is used to manage the node, yarn and pnpm version used.

‎CONTRIBUTING.md‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,89 @@ the tests in each location. Check out the `scripts` entry of the corresponding `
7373

7474
Note: you must run `yarn build` before `yarn test` will work.
7575

76+
## Running E2E Tests Locally
77+
78+
E2E tests verify SDK behavior in real-world framework scenarios using a local npm registry (Verdaccio).
79+
80+
### Prerequisites
81+
82+
1. **Docker**: Required to run the Verdaccio registry container
83+
2. **Volta with pnpm support**: Enable pnpm in Volta by setting `VOLTA_FEATURE_PNPM=1` in your environment. See [Volta pnpm docs](https://docs.volta.sh/advanced/pnpm).
84+
85+
### Step-by-Step Instructions
86+
87+
1. **Build the SDK packages and create tarballs:**
88+
89+
```bash
90+
yarn build
91+
yarn build:tarball
92+
```
93+
94+
Note: You must re-run `yarn build:tarball` after any changes to packages.
95+
96+
2. **Set up environment (optional):**
97+
98+
```bash
99+
cd dev-packages/e2e-tests
100+
cp .env.example .env
101+
# Fill in Sentry project auth info if running tests that send data to Sentry
102+
```
103+
104+
3. **Run all E2E tests:**
105+
106+
```bash
107+
yarn test:e2e
108+
```
109+
110+
4. **Or run a specific test application:**
111+
112+
```bash
113+
yarn test:run <app-name>
114+
# Example: yarn test:run nextjs-app-dir
115+
```
116+
117+
5. **Run with a specific variant:**
118+
```bash
119+
yarn test:run <app-name> --variant <variant-name>
120+
# Example: yarn test:run nextjs-pages-dir --variant 15
121+
```
122+
123+
### Common Issues and Troubleshooting
124+
125+
#### Packages install from public npm instead of Verdaccio
126+
127+
Every E2E test application **must** have an `.npmrc` file with:
128+
129+
```
130+
@sentry:registry=http://127.0.0.1:4873
131+
@sentry-internal:registry=http://127.0.0.1:4873
132+
```
133+
134+
Without this, pnpm will fetch packages from the public npm registry instead of the local Verdaccio instance, causing tests to use outdated/published versions instead of your local changes.
135+
136+
#### Tests fail after making SDK changes
137+
138+
Make sure to rebuild tarballs:
139+
140+
```bash
141+
yarn build
142+
yarn build:tarball
143+
```
144+
145+
#### Docker-related issues
146+
147+
- Ensure Docker daemon is running
148+
- Check that port 4873 is not in use by another process
149+
- Try stopping and removing existing Verdaccio containers
150+
151+
#### Debugging test failures
152+
153+
1. Check browser console logs for SDK initialization errors
154+
2. Enable debug mode in the test app's Sentry config: `debug: true`
155+
3. Verify packages are installed from Verdaccio by checking the version in `node_modules/@sentry/*/package.json`
156+
157+
For more details, see [dev-packages/e2e-tests/README.md](dev-packages/e2e-tests/README.md).
158+
76159
## Debug Build Flags
77160

78161
Throughout the codebase, you will find a `__DEBUG_BUILD__` constant. This flag serves two purposes:

‎dev-packages/e2e-tests/README.md‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,81 @@ EOF
8888

8989
Make sure to add a `test:build` and `test:assert` command to the new app's `package.json` file.
9090

91+
### The `.npmrc` File
92+
93+
Every test application needs an `.npmrc` file (as shown above) to tell pnpm to fetch `@sentry/*` and `@sentry-internal/*` packages from the local Verdaccio registry. Without it, pnpm will install from the public npm registry and your local changes won't be tested - this is one of the most common causes of confusing test failures.
94+
95+
To verify packages are being installed from Verdaccio, check the version in `node_modules/@sentry/*/package.json`. If it shows something like `0.0.0-pr.12345`, Verdaccio is working. If it shows a released version (e.g., `8.0.0`), the `.npmrc` is missing or incorrect.
96+
97+
## Troubleshooting
98+
99+
### Common Issues
100+
101+
#### Tests fail with "Cannot find module '@sentry/...'" or use wrong package version
102+
103+
1. Verify the test application has an `.npmrc` file (see above)
104+
2. Rebuild tarballs: `yarn build && yarn build:tarball`
105+
3. Delete `node_modules` in the test application and re-run the test
106+
107+
#### Docker/Verdaccio issues
108+
109+
- Ensure Docker daemon is running
110+
- Check that port 4873 is not already in use: `lsof -i :4873`
111+
- Stop any existing Verdaccio containers: `docker ps` and `docker stop <container-id>`
112+
- Check Verdaccio logs for errors
113+
114+
#### Tests pass locally but fail in CI (or vice versa)
115+
116+
- Most likely cause: missing `.npmrc` file
117+
- Verify all `@sentry/*` dependencies use `latest || *` version specifier
118+
- Check if the test relies on environment-specific behavior
119+
120+
### Debugging Tips
121+
122+
1. **Enable Sentry debug mode**: Add `debug: true` to the Sentry init config to see detailed SDK logs
123+
2. **Check browser console**: Look for SDK initialization errors or warnings
124+
3. **Inspect network requests**: Verify events are being sent to the expected endpoint
125+
4. **Check installed versions**: `cat node_modules/@sentry/browser/package.json | grep version`
126+
127+
## Bundler-Specific Behavior
128+
129+
Different bundlers handle environment variables and code replacement differently. This is important when writing tests or SDK code that relies on build-time constants.
130+
131+
### Webpack
132+
133+
- `DefinePlugin` replaces variables in your application code
134+
- **Does NOT replace values inside `node_modules`**
135+
- Environment variables must be explicitly defined
136+
137+
### Vite
138+
139+
- `define` option replaces variables in your application code
140+
- **Does NOT replace values inside `node_modules`**
141+
- `import.meta.env.VITE_*` variables are replaced at build time
142+
- For replacing values in dependencies, use `@rollup/plugin-replace`
143+
144+
### Next.js
145+
146+
- Automatically injects `process.env` via webpack/turbopack
147+
- Handles environment variables more seamlessly than raw webpack/Vite
148+
- Server and client bundles may have different environment variable access
149+
150+
### `import.meta.env` Considerations
151+
152+
- Only available in Vite and ES modules
153+
- Webpack and Turbopack do not have `import.meta.env`
154+
- SDK code accessing `import.meta.env` must use try-catch to handle environments where it doesn't exist
155+
156+
```typescript
157+
// Safe pattern for SDK code
158+
let envValue: string | undefined;
159+
try {
160+
envValue = import.meta.env.VITE_SOME_VAR;
161+
} catch {
162+
// import.meta.env not available in this bundler
163+
}
164+
```
165+
91166
Test apps in the folder `test-applications` will be automatically picked up by CI in the job `job_e2e_tests` (in `.github/workflows/build.yml`).
92167
The test matrix for CI is generated in `dev-packages/e2e-tests/lib/getTestMatrix.ts`.
93168

0 commit comments

Comments
 (0)