Skip to content

fix(amazonq): Prevent adding same file twice for /doc and /dev #6568

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jpinkney-aws merged 3 commits into from
Feb 18, 2025
Merged

fix(amazonq): Prevent adding same file twice for /doc and /dev #6568

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bf9c855
fix(amazonq): Prevent adding same file twice for /doc and /dev
avi-alpert Feb 12, 2025
f140e1a
fix(test): add unit test for prepareRepoData on multi-root workspace
avi-alpert Feb 13, 2025
c36c84f
fix(amazonq): Update changelog
avi-alpert Feb 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions packages/amazonq/.changes/next-release/Bug Fix-037e19bd-95f5-4cdf-8408-6eb273ff7ebd.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"type": "Bug Fix",
"description": "/dev and /doc: Multi-root workspace with duplicate files causes infinite 'Uploading code...' loop"
}
34 changes: 27 additions & 7 deletions packages/amazonq/test/unit/amazonqFeatureDev/util/files.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ import { fs, AmazonqCreateUpload, ZipStream } from 'aws-core-vscode/shared'
import { MetricName, Span } from 'aws-core-vscode/telemetry'
import sinon from 'sinon'
import { CodeWhispererSettings } from 'aws-core-vscode/codewhisperer'
import { CurrentWsFolders } from 'aws-core-vscode/amazonq'
import path from 'path'

const testDevfilePrepareRepo = async (devfileEnabled: boolean) => {
const files: Record<string, string> = {
Expand Down Expand Up @@ -44,19 +46,24 @@ const testDevfilePrepareRepo = async (devfileEnabled: boolean) => {
.stub(CodeWhispererSettings.instance, 'getAutoBuildSetting')
.returns(devfileEnabled ? { [workspace.uri.fsPath]: true } : {})

await testPrepareRepoData(workspace, expectedFiles)
await testPrepareRepoData([workspace], expectedFiles)
}

const testPrepareRepoData = async (
workspace: vscode.WorkspaceFolder,
workspaces: vscode.WorkspaceFolder[],
expectedFiles: string[],
expectedTelemetryMetrics?: Array<{ metricName: MetricName; value: any }>
) => {
expectedFiles.sort((a, b) => a.localeCompare(b))
const telemetry = new TelemetryHelper()
const result = await prepareRepoData([workspace.uri.fsPath], [workspace], telemetry, {
record: () => {},
} as unknown as Span<AmazonqCreateUpload>)
const result = await prepareRepoData(
workspaces.map((ws) => ws.uri.fsPath),
workspaces as CurrentWsFolders,
telemetry,
{
record: () => {},
} as unknown as Span<AmazonqCreateUpload>
)

assert.strictEqual(Buffer.isBuffer(result.zipFileBuffer), true)
// checksum is not the same across different test executions because some unique random folder names are generated
Expand Down Expand Up @@ -87,7 +94,7 @@ describe('file utils', () => {
await folder.write('file2.md', 'test content')
const workspace = getWorkspaceFolder(folder.path)

await testPrepareRepoData(workspace, ['file1.md', 'file2.md'])
await testPrepareRepoData([workspace], ['file1.md', 'file2.md'])
})

it('prepareRepoData ignores denied file extensions', async function () {
Expand All @@ -96,7 +103,7 @@ describe('file utils', () => {
const workspace = getWorkspaceFolder(folder.path)

await testPrepareRepoData(
workspace,
[workspace],
[],
[{ metricName: 'amazonq_bundleExtensionIgnored', value: { filenameExt: 'mp4', count: 1 } }]
)
Expand Down Expand Up @@ -126,5 +133,18 @@ describe('file utils', () => {
ContentLengthError
)
})

it('prepareRepoData properly handles multi-root workspaces', async function () {
const folder = await TestFolder.create()
const testFilePath = 'innerFolder/file.md'
await folder.write(testFilePath, 'test content')

// Add a folder and its subfolder to the workspace
const workspace1 = getWorkspaceFolder(folder.path)
const workspace2 = getWorkspaceFolder(folder.path + '/innerFolder')
const folderName = path.basename(folder.path)

await testPrepareRepoData([workspace1, workspace2], [`${folderName}_${workspace1.name}/${testFilePath}`])
Copy link
Contributor

@jpinkney-aws jpinkney-aws Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're mostly just testing that this doesn't reject right? Does it make sense to wrap this with:

assert.doesNotReject(() => {
    await testPrepareRepoData([workspace1, workspace2], [`${folderName}_${workspace1.name}/${testFilePath}`])
})

? otherwise the PR looks good to me

Copy link
Contributor Author

@avi-alpert avi-alpert Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the testPrepareRepoData has asserts inside of it, which will fail if the function fails

Copy link
Contributor Author

@avi-alpert avi-alpert Feb 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error thrown by the test would be Error: File already exists at ZipWriter.add which indicates the root cause, the extra assert is not needed

})
})
})
6 changes: 6 additions & 0 deletions packages/core/src/amazonq/util/files.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,14 @@ export async function prepareRepoData(

let totalBytes = 0
const ignoredExtensionMap = new Map<string, number>()
const addedFilePaths = new Set()

for (const file of files) {
if (addedFilePaths.has(file.zipFilePath)) {
Copy link
Contributor

@vikshe vikshe Feb 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does collectFiles returns same file path if sourcePaths/repoRootPaths > 1 ?

Copy link
Contributor Author

@avi-alpert avi-alpert Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have a file that shows up in a multi root workspace more than once, then collectFiles will output it multiple times, all with the same zipFilePath

continue
}
addedFilePaths.add(file.zipFilePath)

let fileSize
try {
fileSize = (await fs.stat(file.fileUri)).size
Expand Down
Loading