feat(amazonq): add shortcut test for UI E2E Tests #7863

surajrdy-aws · 2025-08-11T19:26:09Z

Problem

We have not implemented the shortcut tests for our UI E2E Test suite with VSCode-Extension-Tester. Additionally, we face a small problem in our pressShortcut abstraction that throws a type error when trying to reference a Key command like Key.COMMAND since it is a IKey String.

Solution

We have implemented the shortcut tests:

Optimize Code Shortcut
Generate Test Shortcut
Keybind Check
Explain Code Shortcut

We have also fixed the pressShortcut abstraction with a small edit in the logic of accepted parameters.

NOTE: We have chosen to implement the Inline Shortcut within the Inline Tests even though it could be classified under Shorcuts.

Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
Your code changes must meet the guidelines in CONTRIBUTING.md.
License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

github-actions · 2025-08-11T19:26:22Z

This pull request implements a feat or fix, so it must include a changelog entry (unless the fix is for an unreleased feature). Review the changelog guidelines.
- Note: beta or "experiment" features that have active users should announce fixes in the changelog.
- If this is not a feature or fix, use an appropriate type from the title guidelines. For example, telemetry-only changes should use the telemetry type.

Hweinstock · 2025-08-12T15:05:20Z

Looks like CI is blocked:

Invalid pull request title: `feature(amazonq): add shortcut test for UI E2E Tests`

* Problem: invalid type "feature"
* Expected format: `type(scope): subject...`
    * type: one of (build, ci, config, deps, docs, feat, fix, perf, refactor, revert, style, telemetry, test, types)
    * scope: lowercase, <30 chars
    * subject: must be <100 chars
    * documentation: https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#pull-request-title
* Hint: *close and re-open the PR* to re-trigger CI (after fixing the PR title).

eslint

Hweinstock · 2025-08-12T16:43:10Z

packages/amazonq/test/e2e_new/amazonq/tests/shortcut.test.ts

+
+describe('Amazon Q Shortcut Functionality', function () {
+    // this timeout is the general timeout for the entire test suite
+    this.timeout(150000)


can we centralize this to one place?

TODO: once all PRs are in, go into all files and create a constant.ts file to manage repeated code. @surajrdy-aws

Hweinstock · 2025-08-12T16:43:35Z

packages/amazonq/test/e2e_new/amazonq/tests/shortcut.test.ts

+        await closeAllTabs(webviewView)
+        await clearChat(webviewView)
+    })
+    it('Keybind Check', async () => {


same comment as other PR for the test titles.

Hweinstock · 2025-08-12T16:44:14Z

packages/amazonq/test/e2e_new/amazonq/tests/shortcut.test.ts

+    it('Keybind Check', async () => {
+        const driver = webviewView.getDriver()
+        // Open Command Palette
+        await pressShortcut(driver, Key.COMMAND, Key.SHIFT, 'p')


does VET not have native support for opening the command palette? Somewhat surprised by that.

They do, but for these tests, they specifically wanted to test Keybinds on the computer as short cuts. I wonder if just using the native VET support is any different in the testing sense?

interesting. Does VSC allow for users to overwrite key binds? If so would this test always fail on their machine?

Yes I believe they do allow for users to overwrite key binds. The tests would then fail on the machine. I think generally this test is not extremely robust, but not sure how else to tackle it since some of the test descriptions explicitly asked for keybinds to be used. What are your thoughts on it?

Yea, I guess this is fine for now. Could be worth adding a comment, but don't see it as a blocker.

Hweinstock · 2025-08-12T16:44:53Z

packages/amazonq/test/e2e_new/amazonq/tests/shortcut.test.ts

+
+        const driver = webviewView.getDriver()
+        await pressShortcut(driver, Key.COMMAND, Key.ALT, 't')
+        // Clean Up Text


nit: idk if these comments are needed.

Hweinstock · 2025-08-12T16:52:27Z

packages/amazonq/test/e2e_new/amazonq/tests/shortcut.test.ts

+        if (!responseReceived) {
+            throw new Error('Chat response not received  within timeout')
+        }
+        console.log('Chat response detected successfully')


you shouldn't need to log successful tests. This should be done automatically by mocha.

Let me fix that!

Hweinstock · 2025-08-12T19:42:20Z

packages/amazonq/test/e2e_new/amazonq/utils/generalUtils.ts

 * Ctrl + Shift + T | await pressShortcut(driver, Key.CONTROL, Key.SHIFT, 't')
 */
-export async function pressShortcut(driver: WebDriver, ...keys: (keyof typeof Key)[]): Promise<void> {
+export async function pressShortcut(driver: WebDriver, ...keys: (string | keyof typeof Key)[]): Promise<void> {


Is there a way to describe the 'a' key as the specified type rather than the string? Or is this change necessary?

Yeah, so to my understanding, the problem isn't actually the 'a' string but also the IKey variables which are declared as strings. For example:

COMMAND: string; // Apple command key

surajrdy-aws · 2025-08-15T16:40:11Z

/retryBuilds

feature: add shortcut test

6dddd03

surajrdy-aws requested a review from a team as a code owner August 11, 2025 19:26

Merge branch 'feature/ui-e2e-tests' into shortcuts

97a6ad7

surajrdy-aws changed the title ~~feature(amazonq): add shortcut test for UI E2E Tests~~ feat(amazonq): add shortcut test for UI E2E Tests Aug 12, 2025

surajrdy-aws closed this Aug 12, 2025

surajrdy-aws reopened this Aug 12, 2025

Update shortcut.test.ts

c8bc42c

eslint

Hweinstock reviewed Aug 12, 2025

View reviewed changes

fix: update code from PR comments

4761d6e

Hweinstock reviewed Aug 12, 2025

View reviewed changes

Hweinstock approved these changes Aug 13, 2025

View reviewed changes

laura-codess self-requested a review August 13, 2025 17:47

laura-codess approved these changes Aug 13, 2025

View reviewed changes

surajrdy-aws added 4 commits August 14, 2025 11:56

fix: multiple OS test

87a1c2e

Merge branch 'feature/ui-e2e-tests' into shortcuts

8549a9b

Merge branch 'feature/ui-e2e-tests' into shortcuts

3cca3f3

fix: rebase

5be15af

bywang56 merged commit a9a8b27 into aws:feature/ui-e2e-tests Aug 15, 2025
26 of 37 checks passed

feat(amazonq): add shortcut test for UI E2E Tests #7863

feat(amazonq): add shortcut test for UI E2E Tests #7863

Uh oh!

Conversation

surajrdy-aws commented Aug 11, 2025

Problem

Solution

Uh oh!

github-actions bot commented Aug 11, 2025

Uh oh!

Hweinstock commented Aug 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

surajrdy-aws Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

surajrdy-aws commented Aug 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

surajrdy-aws Aug 12, 2025 •

edited

Loading