Skip to content

fix(amazonq): Addition of wait function for Amazon Q inline chat UI E2E Tests and Inline Keybind Shortcut Test #7840

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: feature/ui-e2e-tests
Choose a base branch
from
Open
28 changes: 17 additions & 11 deletions packages/amazonq/test/e2e_new/amazonq/tests/inline.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import '../utils/setup'
import { Workbench, EditorView, InputBox, TextEditor, WebviewView, Key } from 'vscode-extension-tester'
import { testContext } from '../utils/testContext'
import { pressKey, createNewTextFile, writeToTextEditor } from '../utils/generalUtils'
import { createNewTextFile, writeToTextEditor, waitForInlineGeneration } from '../utils/generalUtils'
import assert from 'assert'

describe('Amazon Q Inline Completion / Chat Functionality', function () {
Expand All @@ -20,25 +20,31 @@ describe('Amazon Q Inline Completion / Chat Functionality', function () {
webviewView = testContext.webviewView
await webviewView.switchBack()
workbench = testContext.workbench

editorView = new EditorView()
testContext.editorView = editorView

textEditor = await createNewTextFile(workbench, editorView)
})

it('Inline Test', async () => {
await writeToTextEditor(textEditor, 'Select Me')
after(async function () {
// Switch back to Webview Iframe when dealing with external webviews from Amazon Q.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should be corrected to iframe

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha!

await editorView.closeAllEditors()
await webviewView.switchToFrame()
})
it('Inline Test Shortcut', async () => {
await writeToTextEditor(textEditor, 'def factorial(n):')
const text = await textEditor.getText()
assert.equal(text, 'Select Me')
assert.equal(text, 'def factorial(n): ')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe a silly question, but why does this pass with the space at the end, if the writeToTextEditor above doesn't add a space at the end? Or does it automatically?

Is this related to the dummy space below?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! The dummy space actually creates a space at the end of the word, and then it types from before that space so the space ends up at the end of whatever text we generate. I actually removed this logic since it is no longer needed in the test (the assert equal text)

await textEditor.clearText()

const textBefore = await textEditor.getText()
await workbench.executeCommand('Amazon Q: Inline Chat')
const input = new InputBox()
await input.sendKeys('Write a simple sentece')
await input.sendKeys('Generate the fibonacci sequence through iteration')
await input.sendKeys(Key.ENTER)
const driver = textEditor.getDriver()
await pressKey(driver, 'ENTER')
await pressKey(driver, 'TAB')
// Wait for Amazon Q to finish generating code
await waitForInlineGeneration(textEditor)

const textAfter = await textEditor.getText()
assert(textAfter.length > textBefore.length, 'Amazon Q generated code')
await textEditor.clearText()
})
})
37 changes: 35 additions & 2 deletions packages/amazonq/test/e2e_new/amazonq/utils/generalUtils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -157,9 +157,42 @@ export async function createNewTextFile(workbench: Workbench, editorView: Editor
* @returns Promise<void>
*/
export async function writeToTextEditor(textEditor: TextEditor, text: string): Promise<void> {
// We require a "dummy" space to be written such that we can properly index the
// number of lines to register the textEditor.
await textEditor.typeTextAt(1, 1, ' ')
const currentLines = await textEditor.getNumberOfLines()
const nextLine = currentLines + 1
await textEditor.typeTextAt(nextLine, 1, text)
await textEditor.typeTextAt(currentLines, 1, text)
}

/**
* Waits for Inline Generation by Amazon Q by checking if line count stops changing
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I like the logic you decided on here. Might be worth summarizing this logic in the doc-string since it would not be obvious to me without reading your PR description. (Nice PR description btw!)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Will do.

* @param editor The TextEditor instance
* @param timeout Maximum time to wait in milliseconds (default: 15000)
* @returns Promise<void>
* @throws Error if timeout is exceeded
*/
export async function waitForInlineGeneration(editor: TextEditor, timeout = 15000): Promise<void> {
const startTime = Date.now()
let previousLines = await editor.getNumberOfLines()
let stableCount = 0

while (Date.now() - startTime < timeout) {
await sleep(1000)
const currentLines = await editor.getNumberOfLines()

if (currentLines === previousLines) {
stableCount++
if (stableCount >= 3) {
return
}
} else {
stableCount = 0
}

previousLines = currentLines
}

throw new Error(`Editor stabilization timed out after ${timeout}ms`)
}

/**
Expand Down