Skip to content

Commit 7ded2fc

Browse files
jonathanKingstonjonathanKingston
authored
Document test pages (#1796)
* Document test pages * Link into cursorrules * Simplify and update docs * Add index page advice * Add best practices about test state * Lint fix
1 parent 88ce679 commit 7ded2fc

File tree

5 files changed

+354
-2
lines changed

5 files changed

+354
-2
lines changed

‎.cursorrules

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,5 +11,6 @@ When asked about Content Scope Scripts topics, refer to these documentation file
1111
- **Testing**: `injected/docs/testing-guide.md`
1212
- **Favicon**: `injected/docs/favicon.md`
1313
- **Message Bridge**: `injected/docs/message-bridge.md`
14+
- **Test Pages**: `injected/docs/test-pages-guide.md`
1415
- **Documentation Index**: `injected/docs/README.md`
1516
- **High-level Overview**: `injected/README.md`

‎injected/docs/test-pages-guide.md

Lines changed: 322 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,322 @@
1+
# Test Pages Guide
2+
3+
## Overview
4+
5+
The test-pages system is a comprehensive testing framework for Content Scope Scripts that validates feature functionality across different platforms (Android, Apple, Windows, and browser extensions). These test pages are shared by clients and can be run both in browsers and in CI environments.
6+
7+
## Architecture
8+
9+
### Directory Structure
10+
11+
```
12+
injected/integration-test/test-pages/
13+
├── index.html # Main entry point
14+
├── blank.html # Minimal page for extension testing
15+
├── shared/ # Shared utilities and styles
16+
│ ├── utils.js # Test framework utilities
17+
│ ├── style.css # Common styling
18+
│ └── ...
19+
├── {feature-name}/ # Feature-specific test directories
20+
│ ├── index.html # Feature test index
21+
│ ├── pages/ # Individual test pages
22+
│ │ └── {test-name}.html # Test page implementations
23+
│ ├── config/ # Feature configurations
24+
│ │ └── {config-name}.json # JSON configuration files
25+
│ └── scripts/ # Additional test scripts (optional)
26+
└── ...
27+
```
28+
29+
### Key Components
30+
31+
#### 1. Test Pages (`pages/*.html`)
32+
33+
Individual HTML pages that implement specific test scenarios. Each page:
34+
35+
- Loads the Content Scope Scripts
36+
- Defines test cases using the `test()` function
37+
- Validates expected outcomes against actual results
38+
- Renders results in a standardized format
39+
40+
#### 2. Configuration Files (`config/*.json`)
41+
42+
JSON files that define feature configurations for testing:
43+
44+
- Feature states (enabled/disabled)
45+
- Settings and parameters
46+
- Conditional logic and exceptions
47+
- Platform-specific configurations
48+
49+
#### 3. Shared Utilities (`shared/utils.js`)
50+
51+
Provides the testing framework:
52+
53+
- `test(name, testFunction)` - Define test cases
54+
- `renderResults()` - Execute tests and display results
55+
- Result collection and validation
56+
- Automation support for CI environments
57+
58+
## How It Works
59+
60+
### Test Execution Flow
61+
62+
1. **Page Loading**: Test pages are loaded with Content Scope Scripts injected
63+
2. **Configuration Application**: Feature configurations are applied based on the test scenario
64+
3. **Test Execution**: Individual test cases run and validate expected outcomes
65+
4. **Result Collection**: Results are collected and displayed in a standardized format
66+
5. **Validation**: CI systems validate that all tests pass across different platforms
67+
68+
### Example Test Page
69+
70+
```html
71+
<!DOCTYPE html>
72+
<html>
73+
<head>
74+
<title>Conditional Matching Test</title>
75+
<link rel="stylesheet" href="../../shared/style.css" />
76+
</head>
77+
<body>
78+
<script src="../../shared/utils.js"></script>
79+
80+
<script>
81+
test('Conditional matching', async () => {
82+
const results = [
83+
{
84+
name: 'APIs changing, expecting to always match',
85+
result: navigator.hardwareConcurrency,
86+
expected: 222,
87+
},
88+
];
89+
90+
// Test logic here...
91+
92+
return results;
93+
});
94+
95+
renderResults();
96+
</script>
97+
</body>
98+
</html>
99+
```
100+
101+
### Example Configuration
102+
103+
```json
104+
{
105+
"readme": "This config tests conditional matching of experiments",
106+
"version": 1,
107+
"features": {
108+
"apiManipulation": {
109+
"state": "enabled",
110+
"settings": {
111+
"apiChanges": {
112+
"Navigator.prototype.hardwareConcurrency": {
113+
"type": "descriptor",
114+
"getterValue": {
115+
"type": "number",
116+
"value": 222
117+
}
118+
}
119+
},
120+
"conditionalChanges": [
121+
{
122+
"condition": {
123+
"urlPattern": "/test/*"
124+
},
125+
"patchSettings": [
126+
{
127+
"op": "replace",
128+
"path": "/apiChanges/Navigator.prototype.hardwareConcurrency/getterValue/value",
129+
"value": 333
130+
}
131+
]
132+
}
133+
]
134+
}
135+
}
136+
}
137+
}
138+
```
139+
140+
**Tip**: The `apiManipulation` feature is particularly useful for testing config conditions because it modifies browser APIs in predictable ways. You can use it to validate that conditional logic, URL patterns, and other config conditions are being applied correctly by checking if the expected API values are returned.
141+
142+
## Platform Integration
143+
144+
### Cross-Platform Testing
145+
146+
The test pages are designed to work across multiple platforms:
147+
148+
- **Browser Extensions**: Tests run in extension context with injected content scripts
149+
- **Android**: Tests run in WebView with platform-specific messaging
150+
- **Apple**: Tests run in WKWebView with WebKit-specific implementations
151+
- **Windows**: Tests run with Windows-specific global polyfills
152+
153+
### Platform-Specific Handling
154+
155+
The test framework automatically handles platform differences through the `ResultsCollector` class, which applies appropriate setup and polyfills for each platform during test execution.
156+
157+
## Running Tests
158+
159+
### Local Development
160+
161+
1. **Start the test server**:
162+
163+
```bash
164+
npm run serve
165+
```
166+
167+
2. **Access test pages**:
168+
- Navigate to `http://localhost:3220/` for the main index
169+
- Browse to specific test categories and pages
170+
171+
### CI Integration
172+
173+
Tests are ran in CI environments:
174+
175+
```javascript
176+
// Example CI test
177+
test('Test infra', async ({ page }, testInfo) => {
178+
await testPage(
179+
page,
180+
testInfo,
181+
'/infra/pages/conditional-matching.html',
182+
'./integration-test/test-pages/infra/config/conditional-matching.json',
183+
);
184+
});
185+
```
186+
187+
See [pages.spec.js](../integration-test/pages.spec.js) for complete CI test examples.
188+
189+
## Testing Best Practices
190+
191+
When writing integration tests, follow these important guidelines:
192+
193+
### 1. Avoid Custom State in Spec Files
194+
195+
It's unadvisable to add custom state for tests directly in `.spec.js` files as it makes validation difficult and reduces test reliability. If custom state is absolutely required, ensure this is clearly explained in the corresponding test HTML file with detailed comments about what state is being set and why it's necessary.
196+
197+
### 2. Platform Configuration
198+
199+
The `Platform` parameter can be passed to test functions to simulate different platform environments. This is demonstrated in the min-supported-version tests in [pages.spec.js](../integration-test/pages.spec.js):
200+
201+
- `minSupportedVersion (string)`: Uses `{ version: '1.5.0' }`
202+
- `minSupportedVersion (int)`: Uses `{ version: 99 }`
203+
204+
This is needed when testing features that have platform-specific behavior or version requirements. The platform object allows testing how features behave under different version constraints without modifying the core test infrastructure.
205+
206+
### 3. Config-Driven Testing
207+
208+
Where possible, prefer purely config-driven testing to validate features. This approach:
209+
210+
- Makes tests more maintainable and readable
211+
- Reduces coupling between test logic and implementation details
212+
- Allows for easier test data management and updates
213+
- Provides better separation of concerns between test setup and validation
214+
215+
For detailed testing guidelines and examples, see the [IMPORTANT TESTING GUIDELINES section](../integration-test/pages.spec.js#L7) in the pages.spec.js file.
216+
217+
## Interactive and Automation Modes
218+
219+
### Interactive Mode
220+
221+
- When the test page is loaded **without** `?automation=true` in the URL, a **"Run Tests" button** appears at the top of the page.
222+
- This allows a human tester or a platform test harness to decide when to start the tests, rather than running them immediately on page load.
223+
- Clicking the button will execute all defined tests and display the results.
224+
225+
### Automation Mode
226+
227+
- When the test page is loaded **with** `?automation=true` in the URL, tests will run automatically as soon as the Content Scope Scripts are initialized.
228+
- This is used for CI and automated testing environments.
229+
- Tests wait for Content Scope Scripts initialization
230+
- Results are collected programmatically
231+
- Standardized result format for validation
232+
233+
## Result Reporting and Visual Validation
234+
235+
- After the tests finish, the results are assigned to `window.results` as a standardized object.
236+
- The results are also rendered as an HTML table on the page, with pass/fail indicators for each test case.
237+
- This visual output can be used by visual testing tools (such as Maestro) to validate that the test run is complete and that all tests have passed.
238+
- The test suite status is also displayed at the top of the page, showing "pass" or "fail" for the overall run.
239+
240+
## Result Format
241+
242+
Tests return results in a standardized format:
243+
244+
```javascript
245+
{
246+
"Test Name": [
247+
{
248+
"name": "Specific test case",
249+
"result": "actual value",
250+
"expected": "expected value"
251+
}
252+
]
253+
}
254+
```
255+
256+
Results are displayed in HTML tables with pass/fail indicators and can be collected programmatically for CI validation.
257+
258+
## Best Practices
259+
260+
### Creating Index Pages
261+
262+
When creating a new feature directory in the test pages system, it's best practice to include an `index.html` file that serves as a navigation hub for that feature's tests. This provides several benefits:
263+
264+
1. **Easy Navigation**: Developers can quickly browse available tests without digging through subdirectories
265+
2. **Clear Organization**: Each feature directory has a consistent entry point
266+
3. **Documentation**: The index page can include descriptions of what the feature tests cover
267+
4. **Maintainability**: Makes it easier for new team members to understand the test structure
268+
269+
#### Index Page Template
270+
271+
```html
272+
<!DOCTYPE html>
273+
<html>
274+
<head>
275+
<meta charset="utf-8" />
276+
<meta name="viewport" content="width=device-width" />
277+
<title>Feature Name</title>
278+
</head>
279+
<body>
280+
<p><a href="../index.html">[Home]</a></p>
281+
282+
<p>Feature Name</p>
283+
<ul>
284+
<li><a href="./pages/test-page-1.html">Test Page 1</a></li>
285+
<li><a href="./pages/test-page-2.html">Test Page 2</a></li>
286+
</ul>
287+
</body>
288+
</html>
289+
```
290+
291+
#### Key Elements
292+
293+
- **Navigation Link**: Always include a link back to the parent index page
294+
- **Feature Title**: Clear heading indicating what feature is being tested
295+
- **Test Links**: Organized list of all test pages in the `pages/` subdirectory
296+
- **Consistent Structure**: Follow the same pattern as existing feature directories
297+
298+
### Writing Test Pages
299+
300+
1. **Use descriptive test names** that clearly indicate what is being tested
301+
2. **Include multiple test cases** to cover different scenarios and edge cases
302+
3. **Use realistic test data** that mimics real-world usage
303+
4. **Include proper cleanup** for tests that modify state
304+
305+
### Writing Configurations
306+
307+
1. **Document the purpose** in the `readme` field
308+
2. **Use semantic feature names** that match the codebase
309+
3. **Include platform-specific configurations** when needed
310+
4. **Keep configurations focused** on specific test scenarios
311+
312+
### Platform Compatibility
313+
314+
1. **Test across all platforms** to ensure consistency
315+
2. **Handle platform-specific APIs** appropriately
316+
3. **Use platform-agnostic test logic** when possible
317+
4. **Validate messaging interfaces** work correctly
318+
5. **Test fallback behaviors** for unsupported features
319+
320+
## Integration with Privacy Test Pages
321+
322+
The test pages are hosted at [https://privacy-test-pages.site/](https://privacy-test-pages.site/) and used by DuckDuckGo clients, platform teams, CI systems, and external developers to ensure consistent functionality across all platforms.

‎injected/docs/testing-guide.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,10 @@ Everything within `integration-test` is integration tests controlled by Playwrig
3939
npm run test-int
4040
```
4141

42+
**Important**: When writing integration tests, follow the [testing best practices](../docs/test-pages-guide.md#testing-best-practices) outlined in the Test Pages Guide. These guidelines cover avoiding custom state in spec files, using platform configuration, and preferring config-driven testing approaches.
43+
44+
**Preferred Testing Approach**: The [Test Pages Guide](../docs/test-pages-guide.md) describes the most preferred type of testing for the `/injected` directory. Test pages are the preferred approach where possible because they are **sharable with platforms** - the same test pages can be used by Android, Apple, Windows, and browser extension teams, ensuring consistent functionality validation across all platforms.
45+
4246
### Feature Build Process
4347

4448
To produce all artefacts that are used by platforms, just run the `npm run build` command. This will create platform specific code within the `build` folder (that is not checked in).

‎injected/integration-test/pages.spec.js

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,31 @@
11
/**
22
* Tests for shared pages.
33
* Note: these only run in the extension setup for now.
4+
*
5+
* IMPORTANT TESTING GUIDELINES:
6+
*
7+
* 1. AVOID CUSTOM STATE IN SPEC FILES:
8+
* It's unadvisable to add custom state for tests directly in .spec.js files as it makes
9+
* validation difficult and reduces test reliability. If custom state is absolutely required,
10+
* ensure this is clearly explained in the corresponding test HTML file with detailed
11+
* comments about what state is being set and why it's necessary.
12+
*
13+
* 2. PLATFORM CONFIGURATION:
14+
* The 'Platform' parameter can be passed as an argument to testPage() to simulate different
15+
* platform environments. This is demonstrated in the min-supported-version tests:
16+
* - minSupportedVersion (string): Uses { version: '1.5.0' }
17+
* - minSupportedVersion (int): Uses { version: 99 }
18+
*
19+
* This is needed when testing features that have platform-specific behavior or version
20+
* requirements. The platform object allows testing how features behave under different
21+
* version constraints without modifying the core test infrastructure.
22+
*
23+
* 3. CONFIG-DRIVEN TESTING:
24+
* Where possible, prefer purely config-driven testing to validate features. This approach:
25+
* - Makes tests more maintainable and readable
26+
* - Reduces coupling between test logic and implementation details
27+
* - Allows for easier test data management and updates
28+
* - Provides better separation of concerns between test setup and validation
429
*/
530
import { test as base, expect } from '@playwright/test';
631
import { testContextForExtension } from './helpers/harness.js';

‎package.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,8 @@
2727
"lint-fix": "eslint . --fix && npx prettier . --write && npm run tsc",
2828
"stylelint": "npx stylelint \"**/*.css\"",
2929
"stylelint-fix": "npx stylelint \"**/*.css\" --fix",
30-
"serve": "http-server -c-1 --port 3220 integration-test/test-pages",
31-
"serve-special-pages": "http-server -c-1 --port 3221 build/integration/pages"
30+
"serve": "npm run serve --workspace=injected",
31+
"serve-special-pages": "npm run serve --workspace=special-pages"
3232
},
3333
"type": "module",
3434
"workspaces": [

0 commit comments

Comments
 (0)