feat(amazonq): Initializing E2E UI Testing Framework #7685

laura-codess · 2025-07-16T18:17:33Z

Change

This PR sets up the structure for the new E2E UI Testing Framework. The directory is currently structured as follows:

amazonq
└── test
└── e2e_new/amazonq
├── framework
│ ├── chatHelper.ts
│ ├── cleanupHelper.ts
│ ├── generalHelper.ts
│ └── loginHelper.ts
└── chat.test.ts

I took my old "VET.test.ts" files and split them up into the files above.

Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
Your code changes must meet the guidelines in CONTRIBUTING.md.
License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

…as webpack.NormalModuleReplacementPlugin that needed to be there in order for the amazonq to compile

…he checks. have to commit to check since the build compiles with no errors

…est case

…onse

…e tests are done running

…lkit-vscode into feature/ui-e2e-tests

github-actions · 2025-07-16T18:17:46Z

This pull request implements a feat or fix, so it must include a changelog entry (unless the fix is for an unreleased feature). Review the changelog guidelines.
- Note: beta or "experiment" features that have active users should announce fixes in the changelog.
- If this is not a feature or fix, use an appropriate type from the title guidelines. For example, telemetry-only changes should use the telemetry type.

laura-codess · 2025-07-16T20:05:32Z

hmm, it looks like 10 checks didn't fail but were not run at all?

Hweinstock

Nice start on the abstractions. I like how you organized everything.

Hweinstock · 2025-07-17T19:21:38Z

packages/amazonq/test/e2e_new/amazonq/chat.test.ts

+    let webviewView: WebviewView
+
+    before(async function () {
+        webviewView = testContext.webviewView!


why do we need to assert defined here?

sorry, Im a bit confused on what you mean by assert defined?

the testContext is used to keep track of the state that we're in. So we need to grab the webview from testContext because that testContext.webview can be updated depending on what webview we're looking at.

he's asking why did you use this https://www.typescriptlang.org/docs/handbook/release-notes/typescript-2-0.html#non-null-assertion-operator

yes, thanks for the link!

packages/amazonq/test/e2e_new/amazonq/chat.test.ts

packages/amazonq/test/e2e_new/amazonq/framework/chatHelper.ts

Hweinstock · 2025-07-17T19:27:35Z

packages/amazonq/test/e2e_new/amazonq/framework/chatHelper.ts

+    return true
+}
+
+/* Waits for a chat response and outputs whether the response is "correct"


what is 'correct'? Based on the implementation it looks like its checking for any response.

yes, we are just checking for any response. Since I cant actually read the content of the response, i just check if a response exists. is there a different way you recommend doing this?

packages/amazonq/test/e2e_new/amazonq/framework/cleanupHelper.ts

Hweinstock · 2025-07-17T19:31:07Z

packages/amazonq/test/e2e_new/amazonq/framework/generalHelper.ts

+): Promise<WebElement | WebElement[]> {
+    const driver = webview.getDriver()
+    await driver.wait(until.elementsLocated(locator), timeout)
+    return multiple ? await webview.findWebElements(locator) : await webview.findWebElement(locator)


if multiple is false, which element does it pick? What gives one element higher priority over another?

if multiple is false, it will just pick the first element it finds. my goal for the waitForElement function when multiple is false when the writer knows that there is a single element (ie. a button) but the function is flexible enough that it can also be used to find multiple elements (ie. a menu list)

not blocking, but I am also curious what might make something the 'first' element it finds. For example, does it mean it appears first the in the raw html? If so, do we know what determines that?

packages/amazonq/test/e2e_new/amazonq/framework/generalHelper.ts

packages/amazonq/test/e2e_new/amazonq/framework/loginHelper.ts

packages/amazonq/test/e2e_new/amazonq/utils/testContext.ts

Hweinstock · 2025-07-17T19:42:45Z

hmm, it looks like 10 checks didn't fail but were not run at all?

Ah yeah I believe this because of the merge conflicts. You'll have to rebase onto the target branch. When Github can't merge in the changes its not able to run some of the GitHub actions so it stays pending forever. (bad UI if you ask me).

Hweinstock · 2025-07-21T20:17:23Z

P261194666.md

    ```

 5. **Ensure Auto-login Happens Early** (`packages/amazonq/src/lsp/activation.ts`):
+


I don't think you meant to modify this file? Might have to explicitly exclude it from PR.

If its happening on pre-commit hook, you can use -n to bypass that.

Hweinstock · 2025-07-21T20:17:59Z

packages/amazonq/test/e2e_new/amazonq/chat.test.ts

+    let webviewView: WebviewView
+
+    before(async function () {
+        webviewView = testContext.webviewView!


yes, thanks for the link!

Hweinstock · 2025-07-21T20:19:11Z

packages/amazonq/test/e2e_new/amazonq/framework/generalHelper.ts

+): Promise<WebElement | WebElement[]> {
+    const driver = webview.getDriver()
+    await driver.wait(until.elementsLocated(locator), timeout)
+    return multiple ? await webview.findWebElements(locator) : await webview.findWebElement(locator)


not blocking, but I am also curious what might make something the 'first' element it finds. For example, does it mean it appears first the in the raw html? If so, do we know what determines that?

laura-codess added 16 commits July 2, 2025 14:38

initial VET commit with baseline setup + dependencies

98688bf

deleting unnecessary dependencies and refactoring some of VET.test.ts

5959a38

added back the buffer dependency because it failed the tests without it

4540f16

it was not any of the dependencies that was causing it to break, it w…

f1afe95

…as webpack.NormalModuleReplacementPlugin that needed to be there in order for the amazonq to compile

added back some more node: protocol imports to see if it passes all t…

4fa90ec

…he checks. have to commit to check since the build compiles with no errors

forgot to save a webpack change. hoping it will fix the one failing t…

e1c33d8

…est case

added back basically all the dependencies because 1 test keeps failing

ea46464

Adding a simple test to send a chat prompt and check if theres a resp…

66f198d

…onse

Adding the wait to improve the robustness

dd62e27

added logic to the after function to close all the chat tabs after th…

866478a

…e tests are done running

fixed the merge conflicts

35c656f

tested the timeouts, implemented a general wait function

ed3c220

Merge branch 'feature/ui-e2e-tests' of https://github.com/aws/aws-too…

824635e

…lkit-vscode into feature/ui-e2e-tests

checked out master version

b30f7f4

fix: ignore scripts change

5da6336

implementing the initial setup for the framework

c8ccbae

laura-codess requested a review from a team as a code owner July 16, 2025 18:17

laura-codess changed the base branch from master to feature/ui-e2e-tests July 16, 2025 18:17

added setup.ts and testContext so that auth can be shared across tests

74011d2

Hweinstock mentioned this pull request Jul 17, 2025

feat(amazonq): Implement backslash abstraction #7694

Closed

Hweinstock reviewed Jul 17, 2025

View reviewed changes

laura-codess mentioned this pull request Jul 17, 2025

feat(amazonq): Implement Pin Context Test Suite #7699

Closed

laura-codess added 3 commits July 17, 2025 16:51

Removed VET.test.ts

a4d6b32

small . fix

d2bc9dd

fixed the PR based on the comments

693404d

laura-codess mentioned this pull request Jul 21, 2025

feat(amazonq): UI Tests Baseline Framework Complete Set Up #7727

Merged

Hweinstock reviewed Jul 21, 2025

View reviewed changes

laura-codess closed this Jul 22, 2025

laura-codess deleted the initializing_framework branch July 23, 2025 14:43

		```

		5. Ensure Auto-login Happens Early (`packages/amazonq/src/lsp/activation.ts`):

feat(amazonq): Initializing E2E UI Testing Framework #7685

feat(amazonq): Initializing E2E UI Testing Framework #7685

Uh oh!

Conversation

laura-codess commented Jul 16, 2025

Change

Uh oh!

github-actions bot commented Jul 16, 2025

Uh oh!

laura-codess commented Jul 16, 2025

Uh oh!

Hweinstock left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Hweinstock commented Jul 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants