Skip to content

Commit 9c5f854

Browse files
authored
docs(amazonq): Documentation for setting up and running new UI E2E Tests w/ vscode-extension-tester (#7750)
## Problem The AWS Toolkit repository lacks documentation for the UI testing commands (test:ui:prepare, test:ui:install, test:ui:run, test:ui) and the new e2e_new testing suite. Contributors need clear guidance on how to run UI tests and understand the testing framework structure. ## Solution Added comprehensive UI testing documentation covering: - Command usage and workflow for the complete UI test suite - Directory structure and organization of the e2e_new testing framework - Guidelines for writing new UI tests with example code - Troubleshooting common issues and debug techniques - Prerequisites and setup requirements --- - Treat all work as PUBLIC. Private `feature/x` branches will not be squash-merged at release time. - Your code changes must meet the guidelines in [CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines). - License: I confirm that my contribution is made under the terms of the Apache 2.0 license.
1 parent dd2145c commit 9c5f854

File tree

2 files changed

+129
-0
lines changed

2 files changed

+129
-0
lines changed

docs/TESTPLAN.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,10 @@ The test suite has the following categories of tests:
4242
- Live in `src/testE2E`
4343
- These tests are heavier than Integration tests.
4444
- Some E2E tests have a more complicated architecture, described in [TEST_E2E](./TEST_E2E.md)
45+
- UI E2E Tests: **slow** tests
46+
- Live in `packages/amazonq/test/e2e_new`
47+
- These tests target the UI testing aspect in an E2E sense.
48+
- Some E2E tests should be set up using pre-defined helpers and architecture, described in [UI_E2E_Test](./TUI_E2E_Test.md)
4549
- Performance Tests: **slow** tests
4650
- Live in `src/testInteg/perf`.
4751
- A subset of integration tests focused on catching performance regressions.

docs/UI_E2E_Test.md

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
## UI Testing
2+
3+
UI tests use [vscode-extension-tester](https://github.com/redhat-developer/vscode-extension-tester) to test the Amazon Q extension in a real VS Code environment.
4+
5+
### Quick Start
6+
7+
```bash
8+
# Run complete UI test suite
9+
npm run test:ui
10+
```
11+
12+
Note: All of these commands must be run at the root level aws-toolkit-vscode directory.
13+
14+
### Individual Commands
15+
16+
#### `test:ui:prepare`
17+
18+
Downloads VS Code and ChromeDriver to `~/.vscode-test-resources` in order to properly hook Selenium/VET to our new VSCode instance we launch. Without this, test cannot be run at all.
19+
20+
```bash
21+
npm run test:ui:prepare
22+
```
23+
24+
#### `test:ui:install`
25+
26+
Packages the Amazon Q extension VSIX and installs it for testing. This reflects local changes within your immediate aws-toolkit-vscode directory.
27+
28+
```bash
29+
npm run test:ui:install
30+
```
31+
32+
- Runs `npm run package` in amazonq directory
33+
- Extracts version from build output
34+
- Installs VSIX using `extest install-vsix`
35+
- Sets up extension in test environment
36+
37+
#### `test:ui:run`
38+
39+
Compiles TypeScript and runs UI tests within the dist directory. You must run this everytime you make a new change to your tests in order to recompile the tests.
40+
41+
```bash
42+
npm run test:ui:run
43+
```
44+
45+
- Compiles test files with `npm run testCompile`
46+
- Runs tests matching `packages/amazonq/dist/test/e2e_new/amazonq/tests/*.js`
47+
48+
#### Authentication
49+
50+
Currently, authentication is not configured to be automatically logged into AmazonQ. To bypass this for now (as of July 24th, 2025), you must click the approve/open button that redirects you to a browser in order for tests to be run in an authenticated environment at the start of a new.
51+
52+
```bash
53+
npm run test:ui:run
54+
```
55+
56+
#### Test Categories
57+
58+
- **Chat** - Amazon Q chat functionality
59+
- **Pin Context** - Context pinning features
60+
- **Quick Actions** - Quick action commands
61+
- **Switch Model** - Model switching functionality
62+
63+
### Writing New Tests
64+
65+
1. Create test files in `packages/amazonq/test/e2e_new/amazonq/tests/`
66+
2. Import utilities from `../utils/`
67+
3. Use helpers from `../helpers/`
68+
4. Follow existing patterns for setup/cleanup
69+
70+
#### Writing New Helpers
71+
72+
1. Plan out abstractions/helpers after confirming current helpers are not viable
73+
2. Create helper file in `../helpers/`
74+
3. Following existing pattens for helper conventions and parameters
75+
76+
#### Example Test Structure
77+
78+
```typescript
79+
import { describe, it, before, after } from 'mocha'
80+
import { setupTestContext, cleanupTestContext } from '../utils/setup'
81+
82+
describe('Feature Tests', () => {
83+
before(async () => {
84+
await setupTestContext()
85+
})
86+
87+
after(async () => {
88+
await cleanupTestContext()
89+
})
90+
91+
it('should test functionality', async () => {
92+
// Test implementation
93+
})
94+
})
95+
```
96+
97+
### Prerequisites
98+
99+
- Node.js and npm
100+
- VS Code extension development environment
101+
- Chrome/Chromium browser
102+
103+
### Troubleshooting
104+
105+
#### Common Issues
106+
107+
- **VS Code download fails**: Check internet connection, retry `test:ui:prepare`
108+
- **Extension install fails**: Ensure packaging succeeds
109+
- **Tests won't start**: Verify ChromeDriver/Chrome compatibility
110+
- **Permission errors**: Check `~/.vscode-test-resources` permissions
111+
112+
#### Reset
113+
114+
```bash
115+
# Reset test environment
116+
rm -rf ~/.vscode-test-resources
117+
npm run test:ui:prepare
118+
```
119+
120+
### Test Development Tips
121+
122+
- Tests should be independent and run in any order
123+
- Use existing utilities for common operations
124+
- Clean up resources in `after` hooks
125+
- Follow naming conventions from existing tests

0 commit comments

Comments
 (0)