Skip to content

fix(amazonq): Inline test after function addition, writeToTextEditor fix, and package.json pathing fix #7800

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 4, 2025
Merged

fix(amazonq): Inline test after function addition, writeToTextEditor fix, and package.json pathing fix #7800

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0d506f1
fix: add inline test
surajrdy-aws Aug 1, 2025
79b1b3a
fix: PR comments
surajrdy-aws Aug 2, 2025
92c3f61
fix: PR comments from Harrison
surajrdy-aws Aug 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
"testE2E": "npm run testE2E -w packages/ --if-present",
"test:ui:prepare": "./node_modules/.bin/extest get-vscode -s ~/.vscode-test-resources -n && extest get-chromedriver -s ~/.vscode-test-resources -n",
"test:ui:install": "cd packages/amazonq && npm run package 2>&1 | grep -o 'VSIX Version: [^ ]*' | cut -d' ' -f3 | xargs -I{} bash -c 'cd ../../ && ./node_modules/.bin/extest install-vsix -f amazon-q-vscode-{}.vsix -e packages/amazonq/test/e2e_new/amazonq/resources -s ~/.vscode-test-resources'",
"test:ui:run": "npm run testCompile && ./node_modules/.bin/extest run-tests -s ~/.vscode-test-resources -e packages/amazonq/test/e2e_new/amazonq/resources packages/amazonq/dist/test/e2e_new/amazonq/tests/*.js",
"test:ui:run": "npm run testCompile && ./node_modules/.bin/extest run-tests -s ~/.vscode-test-resources -e packages/amazonq/test/e2e_new/amazonq/resources packages/amazonq/dist/test/e2e_new/amazonq/tests/*.test.js",
"test:ui": "npm run test:ui:prepare && npm run test:ui:install && npm run test:ui:run",
"testInteg": "npm run testInteg -w packages/ --if-present",
"package": "npm run package -w packages/toolkit -w packages/amazonq",
Expand Down
28 changes: 17 additions & 11 deletions packages/amazonq/test/e2e_new/amazonq/tests/inline.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import '../utils/setup'
import { Workbench, EditorView, InputBox, TextEditor, WebviewView, Key } from 'vscode-extension-tester'
import { testContext } from '../utils/testContext'
import { pressKey, createNewTextFile, writeToTextEditor } from '../utils/generalUtils'
import { createNewTextFile, writeToTextEditor, sleep } from '../utils/generalUtils'
import assert from 'assert'

describe('Amazon Q Inline Completion / Chat Functionality', function () {
Expand All @@ -20,25 +20,31 @@ describe('Amazon Q Inline Completion / Chat Functionality', function () {
webviewView = testContext.webviewView
await webviewView.switchBack()
workbench = testContext.workbench

editorView = new EditorView()
testContext.editorView = editorView

textEditor = await createNewTextFile(workbench, editorView)
})

it('Inline Test', async () => {
await writeToTextEditor(textEditor, 'Select Me')
after(async function () {
// Switch back to Webview Iframe when dealing with external webviews from Amazon Q.
await editorView.closeAllEditors()
await webviewView.switchToFrame()
})
it('Inline Test Shortcut', async () => {
await writeToTextEditor(textEditor, 'def factorial(n):')
const text = await textEditor.getText()
assert.equal(text, 'Select Me')
assert.equal(text, 'def factorial(n): ')
await textEditor.clearText()

const textBefore = await textEditor.getText()
await workbench.executeCommand('Amazon Q: Inline Chat')
const input = new InputBox()
await input.sendKeys('Write a simple sentece')
await input.sendKeys('Generate the fibonacci sequence through iteration')
await input.sendKeys(Key.ENTER)
Copy link

@laura-codess laura-codess Aug 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this enter now accept the generation?

Copy link
Author

@surajrdy-aws surajrdy-aws Aug 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately it does not, but it is required for us to get out of the suggestion decision messing up any following clean ups

const driver = textEditor.getDriver()
await pressKey(driver, 'ENTER')
await pressKey(driver, 'TAB')
await sleep(8000)
Copy link
Contributor

@Hweinstock Hweinstock Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this sleep for? 8 seconds sounds like a lot

Copy link
Author

@surajrdy-aws surajrdy-aws Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, the sleep is for waiting for the AmazonQ response to be generated. I realized its never always the same and can take sometime to either reason or just start coding right away. 8 seconds seems like the sweet spot for leeway for waiting for a response to be generated fully on other devices as well.

Copy link
Contributor

@Hweinstock Hweinstock Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Is there a way we can wait dynamically with a cap? Like basically poll every 1 second until the response shows up the in the DOM. I think this general util will be helpful for other areas as well.

Copy link
Author

@surajrdy-aws surajrdy-aws Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, will look into it!


const textAfter = await textEditor.getText()
assert(textAfter.length > textBefore.length, 'Amazon Q should have generated code')
assert(textAfter.includes('fibonacci'), 'Generated code should contain fibonacci logic')
Copy link
Contributor

@Hweinstock Hweinstock Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be potentially flaky? Wondering if the model ever generates a solution where they use fib or hallucinates something.

Copy link
Author

@surajrdy-aws surajrdy-aws Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, fixed!

await textEditor.clearText()
})
})
6 changes: 4 additions & 2 deletions packages/amazonq/test/e2e_new/amazonq/utils/generalUtils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,9 +157,11 @@ export async function createNewTextFile(workbench: Workbench, editorView: Editor
* @returns Promise<void>
*/
export async function writeToTextEditor(textEditor: TextEditor, text: string): Promise<void> {
// We require a "dummy" space to be written such that we can properly index the
// number of lines to register the textEditor.
await textEditor.typeTextAt(1, 1, ' ')
Copy link

@laura-codess laura-codess Aug 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know the reason why we need a space, but should we add a comment since its not obvious?

Copy link
Author

@surajrdy-aws surajrdy-aws Aug 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

const currentLines = await textEditor.getNumberOfLines()
const nextLine = currentLines + 1
await textEditor.typeTextAt(nextLine, 1, text)
await textEditor.typeTextAt(currentLines, 1, text)
}

/**
Expand Down
Loading