test(amazonq): Add E2E tests for Q Chat's /review command #6696
Conversation
|
/runIntegrationTests |
| }) | ||
|
|
||
| describe('Quick action availability', () => { | ||
| console.log('running this test') |
| assert.deepStrictEqual(noFileMessage.type, 'answer') | ||
| assert.deepStrictEqual( | ||
| noFileMessage.body, | ||
| 'Sorry, your current active window is not a source code file. Make sure you select a source file as your primary context.' |
There was a problem hiding this comment.
You might want to see if you can extract text like this out and share it between your src and test
| import path from 'path' | ||
|
|
||
| function getWorkspaceFolder(): string { | ||
| return ( |
There was a problem hiding this comment.
I've seen a couple teams do the same thing, do we not have a util? Also, we can expect the first workspace folder to load, otherwise the test setup didn't complete successfully
There was a problem hiding this comment.
I was not able to find the util for this, so I used vscode.workspace.workspaceFolders. You are right that we can expect the workspace to load, I will make a change to reflect that.
| const scanResultBody = await waitForReviewResults(tab) | ||
|
|
||
| const issues = extractAndValidateIssues(scanResultBody) | ||
| assert.deepStrictEqual( |
There was a problem hiding this comment.
Are these in some way slightly deterministic? i.e. will we be introducing accidental flake if the backend changes and doesn't emit the right amount of criticals, high, medium, etc
There was a problem hiding this comment.
yes. Are re-discussing with team, we will change this to only having the critical, rule-based findings that we will not alter. I will make a change for that.
| hasExactlyMatchingSecurityDiagnostic( | ||
| securityDiagnostics, | ||
| 'multilanguage-password', | ||
| 'CWE-798 - Hardcoded credentials', |
There was a problem hiding this comment.
likewise, will these always be deterministic and return the same responses?
There was a problem hiding this comment.
will make a change to reflect that.
|
/runIntegrationTests |
| @@ -0,0 +1,4 @@ | |||
| { | |||
| "type": "Test", | |||
| "description": "add Q Chat /review command test coverage" | |||
There was a problem hiding this comment.
changelog is not necessary for adding tests.
3 participants
Problem
/review command functionality lacks end-to-end test coverage.
Solution
Add basic E2E tests for file review, project review, and ignore lines features.
feature/xbranches will not be squash-merged at release time.