merge main into agentic-mcp (#1520)

* fix: handle requestAborted errors silently (#1394)

* fix: Revert stop text align (#1397)

* Revert "fix: remove the unused center stop message and add comment"

This reverts commit 654a25a.

* Revert "fix: when user input a prompt, it wont have stop message before the new prompt"

This reverts commit f6e9904.

* fix: add validation for empty chat history (#1403)

* fix: the new prompt wont stop the process properly (#1404)

* chore: generate a temporary workspaceIdentifier if not given (#1389)

Co-authored-by: Jiatong Li <[email protected]>

* feat(amazonq): add fileUri to FileContext (#1399)

Co-authored-by: Jiatong Li <[email protected]>

* fix: Set `source` parameter chat request context to 'IDE' (#1407)

* fix: set source to 'IDE' for chat client

---------

Co-authored-by: Alice Koreman <[email protected]>

* feat: integrate server side project context into agentic chat (#1405)

* feat: pass remote workspaceId in agentic chat API calls

* feat: removed dev logs

* feat: add unit test

* feat: bumping up dependency version

* chore: updating package-lock.json

* fix: install new streaming client correctly

---------

Co-authored-by: Ege Ozcan <[email protected]>

* fix: add crypto import (#1408)

#1389 this change broke LSP initialization as the file does not add the required import for crypto.
Added the required import

* fix: accidental formatting #1410

* feat(amazonq): integrate server side workspace context with inline completion (#1402)

* feat(amazonq): integrate workspace context with inline completion

* fix: replace got with axios for s3 uploads in workspace context capability

---------

Co-authored-by: Jiatong Li <[email protected]>
Co-authored-by: Ege Ozcan <[email protected]>

* fix: SSPC dependency upload and watcher fixes (#1377)

* fix: resolve symlink of python dependency upload

* fix: bundle events from dependency watcher

* fix: correct watcher.close() with watcher.dispose()

* fix: fix dependency watcher event

* fix: add additional log for error cases of watcher and symlink resolver

---------

Co-authored-by: Jiatong Li <[email protected]>

* fix: change the version to axios to ^1.8.4 (#1421)

Co-authored-by: Paras Miglani <[email protected]>

* chore: bump runtimes dependencies (#1422)

* chore(release): release packages from branch main (#1339)

* chore(release): release packages from branch main

* chore: apply missing package updates

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Natalia Durlik <[email protected]>

* fix: ensure local index server updates with workspaceChangeEvent and bump runtimes (#1424)

This change bumps runtimes to include fix related to didWorkspaceChangeFolders handling in aws/language-server-runtimes#522
In addition this change makes a fix in the localProjectContextServer where onDidChangeWorkpaceFolders event didn't update the workspaceFolders list used for indexing. It now updates to reflect the change.

* fix: fix uncaught exception in workspaceFolderManager (#1428)

* fix: fix uncaught exception in workspaceFolderManager

* fix: fix agenticChatTriggerContexts test

* chore: set a filter to run SSPC on supported extensions

* fix: axios upload presigned url

* fix: update log comments of extension filter

---------

Co-authored-by: Jiatong Li <[email protected]>

* chore: bump runtimes (#1438)

* chore: bump the typescript-eslint group with 2 updates (#1362)

Bumps the typescript-eslint group with 2 updates: [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) and [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser).


Updates `@typescript-eslint/eslint-plugin` from 8.31.0 to 8.32.1
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.32.1/packages/eslint-plugin)

Updates `@typescript-eslint/parser` from 8.31.0 to 8.32.1
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.32.1/packages/parser)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-version: 8.32.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: typescript-eslint
- dependency-name: "@typescript-eslint/parser"
  dependency-version: 8.32.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: typescript-eslint
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(amazonq): fix UTDE suggestion state for pagination cases (#1433)

In pagination cases, in a certain condition when user has made a decision before a
paginated response arrived to flare,  an extra discard UTD event will be sent after
an accept/reject event corresponding to a user action, this is incorrect

* fix(chat-client): fix bug where pair programmer mode option update was not stored properly (#1400)

`OnPromptInputOptionChange` event when the agentic mode button is toggled on and off, the mynahUI store was not updated properly to store the updated value. This resulted in "insert to cursor" option being disabled/hidden in a codeblock shown in an agentic mode OFF response.

Solution
With this change, on prompt option change event, the MynahUi store is now updated to preserve the latest value of "pair-programmer-mode" option. I verified the insert to cursor option now appears on toggle

* fix: enable fuzzySearch tool (#1328)

* fix: enable grepSearch tool (#1396)

* fix: properly tokenize command args using shlex.split() for Windows (#1440)

* fix: properly tokenize command args using shlex.split() for Windows

* fix: handle built-in windows commands in validation check

* fix(amazonq): fix for honouring the index cache dir path value (#1448)

* fix: convert array values to comma-separated strings in telemetry metrics emitAgencticLoop_InvokeLLM (#1458)

* fix: flatten array into string before emitting telemetry metric

* fix: rename the toolName to toolNames for better readability

* fix: update executeBash UI for failures during command existence check (#1462)

* fix(amazonq): wrap sspc lsp handlers in try/catch so failures do not take down server (#1464)

for example, if file is renamed from supported language to unsupported language,
or upload itself fails, the server crashes.

* fix: increase the code start and end line number by 1 (#1470)

* fix: decode UTF-16LE shell output on Windows (#1456)

* fix: remove gradient from create prompt button (#1475)

* feat(amazonq): add abap as supported language #1463

* fix: reorder cancellation operations (#1478)

* fix: use updated version of vecLib and use local context controller to raise context command updates (#1479)

## Problem
`DidWorkspaceFolderChange` events did not update context commands. This issue is particularly pronounced in Visual Studio where this event is used to send information about projects contained in a solution on intial load which should be indexed.
In addition, the updateIndex calls for vecLib was gated by the indexingEnabled flag which meant when a change event came with the indexer off, the context commands were not updated. The context command updates should happen unconditionally even when indexer is disabled.
## Solution
This change addresses the issues above:
* A new version of the vecLib has been added to the local development copy with the tgz file. A corresponding release PR is being tracked in a separate repo.
* The update above adds support for updating indexer and context commands when workspace folders change.
* A corresponding change has been made to the language server to support sending the workspace folders updates to the vecLib update index call.
* The context command provider and controller were very strongly coupled causing issues updating the two independently. With this change, the controller now becomes the sole authority that raises an event to context command provider to get and push latest context commands after indexing finishes.
* IndexingEnabled flag has been removed as check from the updateIndex calls. It is now expected that the vecLib knows when to not index locally(for @workspace) when the buildIndex('default') is invoked with this value.

* chore: bump mynah-ui version to 4.35.1 in chat-client (#1485)

Co-authored-by: Ege Ozcan <[email protected]>

* chore: add workspace context server to the agent-standalone bundle (#1486)

Co-authored-by: Ege Ozcan <[email protected]>

* chore: bump runtimes version to 0.2.89 (#1487)

* chore: bump runtimes version to 0.2.89

* chore: bump chat client types dependency

---------

Co-authored-by: Supraja Venkatesh <[email protected]>

* fix: add tests for workspace change supports (#1484)

* chore(release): release packages from branch main (#1488)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: improve the executeBash tool spec (#1465)

* Improve the executeBash tool spec

* Remove unnecessary instructions

* fix(amazonq): fix line endings before fswrite for windows (#1483)

* feat(amazonq): send relative file path for inline completion (#1481)

Co-authored-by: Jiatong Li <[email protected]>

* chore: bump runtimes (#1490)

* feat: model selection for agentic chat (#1294)

* feat: model selection for agentic chat

* move modelId to userInputMessage

* add modelId to sendTelemetryEvent

* support per-region selection

* update modelId values

* feat: add custom model exception handling

* merge main and adapt to new changes in runtime with new mynah

* change code to 500

* remove unwanted changes

* persist modelId selection in fs

* model change indicator

* keep model selection for other tabs

* fix tests

* update mynahui version

* remove region for now

* remove comment

* fix issues after merge

* only persist selection for the current tab

---------

Co-authored-by: Yaofu Zuo <[email protected]>

* fix: grepSearch on Windows (#1494)

Fix Windows GrepSearch path

* fix: fix paths array issue in fsRead (#1496)

* fix(amazonq): pagination request should also used truncated left/right context (#1497)

* fix(amazonq): pagination request should also used truncated left/right context

also fixes a fallout that the function call should be used for isIndexingEnabled()

* fix: add tests

* fix: extra line when user run the command (#1499)

* fix: add environment variable override to disable indexing library init (#1504)

* chore(release): release packages from branch main (#1489)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: disable grep search (#1514)

* fix: model doesn't update in session for new tabs (#1506)

* feat: added configurable token cache location (#1517)

* feat: added configurable token cache location

- Add support for configuring token cache location through LSP initialization options
- Move default token cache from local .cache directory to $HOMEDIR/.aws/device-sso-lsp/cache
- Update README with configuration documentation
- Pass token cache location through the authentication flow components

* fix: use standard aws/sso/cache location

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: tsmithsz <[email protected]>
Co-authored-by: Randall-Jiang <[email protected]>
Co-authored-by: Jiatong Li <[email protected]>
Co-authored-by: Jiatong Li <[email protected]>
Co-authored-by: Alice Koreman <[email protected]>
Co-authored-by: Alice Koreman <[email protected]>
Co-authored-by: guvishl <[email protected]>
Co-authored-by: Ege Ozcan <[email protected]>
Co-authored-by: Shruti Sinha <[email protected]>
Co-authored-by: Jason Guo <[email protected]>
Co-authored-by: wweitao <[email protected]>
Co-authored-by: pras0131 <[email protected]>
Co-authored-by: Paras Miglani <[email protected]>
Co-authored-by: Natalia Durlik <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Natalia Durlik <[email protected]>
Co-authored-by: Tai Lai <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: andrewyuq <[email protected]>
Co-authored-by: Lokesh <[email protected]>
Co-authored-by: Sherry Lu <[email protected]>
Co-authored-by: Richard Li <[email protected]>
Co-authored-by: Christopher Christou <[email protected]>
Co-authored-by: Aseem sharma <[email protected]>
Co-authored-by: ege0zcan <[email protected]>
Co-authored-by: suprajaven <[email protected]>
Co-authored-by: Supraja Venkatesh <[email protected]>
Co-authored-by: jngyuamz <[email protected]>
Co-authored-by: Yaofu Zuo <[email protected]>
Co-authored-by: Jonathan Breedlove <[email protected]>
Co-authored-by: Lei Gao <[email protected]>
Co-authored-by: Viktor Shcherba <[email protected]>

Author: 32 people

server/aws-lsp-codewhisperer/src/language-server/agenticChat/agenticChatController.test.ts

@@ -338,6 +338,18 @@ describe('AgenticChatController', () => {
         sinon.assert.calledWithExactly(activeTabSpy.set, mockTabId)
     })
 
+    it('onTabAdd updates model ID in chat options and session', () => {
+        const modelId = 'test-model-id'
+        sinon.stub(ChatDatabase.prototype, 'getModelId').returns(modelId)
+
+        chatController.onTabAdd({ tabId: mockTabId })
+
+        sinon.assert.calledWithExactly(testFeatures.chat.chatOptionsUpdate, { modelId, tabId: mockTabId })
+
+        const session = chatSessionManagementService.getSession(mockTabId).data
+        assert.strictEqual(session!.modelId, modelId)
+    })
+
     it('onTabChange sets active tab id in telemetryController and emits metrics', () => {
         chatController.onTabChange({ tabId: mockTabId })
 

server/aws-lsp-codewhisperer/src/language-server/agenticChat/agenticChatController.ts

@@ -2353,14 +2353,19 @@ export class AgenticChatController implements ChatHandlers {
     onTabAdd(params: TabAddParams) {
         this.#telemetryController.activeTabId = params.tabId
 
+        const modelId = this.#chatHistoryDb.getModelId()
+        this.#features.chat.chatOptionsUpdate({ modelId: modelId, tabId: params.tabId })
+
         const sessionResult = this.#chatSessionManagementService.createSession(params.tabId)
-        if (sessionResult.success && sessionResult.data) {
+        const { data: session, success } = sessionResult
+        if (!success) {
+            return new ResponseError<ChatResult>(ErrorCodes.InternalError, sessionResult.error)
+        }
+        session.modelId = modelId
+        if (success && session) {
             // Set the logging object on the session
-            sessionResult.data.setLogging(this.#features.logging)
+            session.setLogging(this.#features.logging)
         }
-
-        const modelId = this.#chatHistoryDb.getModelId()
-        this.#features.chat.chatOptionsUpdate({ modelId: modelId, tabId: params.tabId })
     }
 
     onTabChange(params: TabChangeParams) {

server/aws-lsp-codewhisperer/src/language-server/agenticChat/tools/toolServer.ts

@@ -47,10 +47,11 @@ export const FsToolsServer: Server = ({ workspace, logging, agent, lsp }) => {
         return await fuzzySearchTool.invoke(input, token)
     })
 
-    agent.addTool(grepSearchTool.getSpec(), async (input: GrepSearchParams, token?: CancellationToken) => {
-        await grepSearchTool.validate(input)
-        return await grepSearchTool.invoke(input, token)
-    })
+    // Temporarily disable grep search
+    // agent.addTool(grepSearchTool.getSpec(), async (input: GrepSearchParams, token?: CancellationToken) => {
+    //     await grepSearchTool.validate(input)
+    //     return await grepSearchTool.invoke(input, token)
+    // })
 
     return () => {}
 }

server/device-sso-auth-lsp/README.md

@@ -8,6 +8,20 @@ It is port of [SSO flow implementation in VSCode sample client](../../client/vsc
 
 Supports only [`standalone`](https://github.com/aws/language-server-runtimes/blob/main/runtimes/runtimes/standalone.ts) AWS Server Runtime, as it requires NodeJS `fs` access.
 
+## Configuration
+
+Configure Auth language server by passing `configurationOptions` at LSP Initialize handshake from LSP client. Capability supports next `configurationOptions`:
+
+```typescript
+interface InitializeParams {
+    initializationOptions: {
+        // Path to writable directory to store SSO auth and refresh token cache
+        // Default: $HOMEDIR/.aws/device-sso-lsp/cache
+        tokenCacheLocation?: string
+    }
+}
+```
+
 ## Supported features
 
 ### Custom commands

server/device-sso-auth-lsp/src/language-server/SsoAuthServer.ts

@@ -7,8 +7,8 @@ import {
     Telemetry,
     Workspace,
 } from '@aws/language-server-runtimes/server-interface'
-import { CancellationToken, ExecuteCommandParams } from 'vscode-languageserver/node'
-import { BuilderIdConnectionBuilder, SsoConnection } from './sso/builderId'
+import { CancellationToken, ExecuteCommandParams, InitializeParams } from 'vscode-languageserver/node'
+import { BuilderIdConnectionBuilder, SsoConnection, DEFAULT_TOKEN_CACHE_DIR } from './sso/builderId'
 
 const AUTH_DEVICE_COMMAND = 'ssoAuth/authDevice/getToken'
 
@@ -21,6 +21,7 @@ export const SsoAuthServer: Server = (features: {
 }) => {
     const { lsp, logging } = features
     let activeBuilderIdConnection: SsoConnection | undefined
+    let tokenCacheLocation = DEFAULT_TOKEN_CACHE_DIR
 
     const onInitializedHandler = async () => {}
 
@@ -44,7 +45,8 @@ export const SsoAuthServer: Server = (features: {
                     return true
                 },
             },
-            startUrl
+            startUrl,
+            tokenCacheLocation
         )
 
         const token = await activeBuilderIdConnection.getToken()
@@ -67,9 +69,11 @@ export const SsoAuthServer: Server = (features: {
         return
     }
 
-    lsp.addInitializer(() => {
+    lsp.addInitializer((params: InitializeParams) => {
         logging.log('SSO Auth capability has been initialised')
 
+        tokenCacheLocation = params.initializationOptions?.tokenCacheLocation || DEFAULT_TOKEN_CACHE_DIR
+
         return {
             capabilities: {
                 executeCommandProvider: {

server/device-sso-auth-lsp/src/language-server/sso/builderId.ts

@@ -12,7 +12,9 @@ import { SsoProfile as BaseSsoProfile, ClientRegistration, SsoToken, isExpired }
 
 import * as fs from 'fs'
 import * as path from 'path'
-const TOKEN_CACHE_DIR = path.join(__dirname, '.cache')
+import * as os from 'os'
+
+export const DEFAULT_TOKEN_CACHE_DIR = path.join(os.homedir(), '.aws/sso/cache')
 
 // For the Proof of concept, this file's code was copied (and culled) from the AWS Toolkit for VS Code repo
 // https://github.com/aws/aws-toolkit-vscode/blob/5d621c8405a8b20ffe571ad0ba10ae700178e051/src/auth/auth.ts
@@ -200,14 +202,18 @@ export class OidcClient {
  *         - RefreshToken (optional)
  */
 export class SsoAccessTokenProvider {
-    private readonly TOKEN_CACHE_FILE = path.join(TOKEN_CACHE_DIR, 'token.json')
-    private readonly REGISTRATION_CACHE_FILE = path.join(TOKEN_CACHE_DIR, 'registration.json')
+    private readonly TOKEN_CACHE_FILE
+    private readonly REGISTRATION_CACHE_FILE
 
     public constructor(
         private readonly profile: Pick<SsoProfile, 'startUrl' | 'region' | 'scopes' | 'identifier'>,
         private readonly uiHandler: UiHandler | undefined = undefined,
+        private readonly tokenCacheDir: string = DEFAULT_TOKEN_CACHE_DIR,
         private readonly oidc = OidcClient.create(profile.region)
-    ) {}
+    ) {
+        this.TOKEN_CACHE_FILE = path.join(this.tokenCacheDir, 'device-sso-lsp-token.json')
+        this.REGISTRATION_CACHE_FILE = path.join(this.tokenCacheDir, 'device-sso-lsp-registration.json')
+    }
 
     public async invalidate(): Promise<void> {
         await fs.promises.unlink(this.TOKEN_CACHE_FILE)
@@ -258,12 +264,12 @@ export class SsoAccessTokenProvider {
     }
 
     private async saveCachedToken(data: SsoAccess): Promise<void> {
-        await fs.promises.mkdir(TOKEN_CACHE_DIR, { recursive: true })
+        await fs.promises.mkdir(this.tokenCacheDir, { recursive: true })
         await fs.promises.writeFile(this.TOKEN_CACHE_FILE, JSON.stringify(data))
     }
 
-    private async saveRegisrationData(data: ClientRegistration): Promise<void> {
-        await fs.promises.mkdir(TOKEN_CACHE_DIR, { recursive: true })
+    private async saveRegistrationData(data: ClientRegistration): Promise<void> {
+        await fs.promises.mkdir(this.tokenCacheDir, { recursive: true })
         await fs.promises.writeFile(this.REGISTRATION_CACHE_FILE, JSON.stringify(data))
     }
 
@@ -279,7 +285,7 @@ export class SsoAccessTokenProvider {
         let cachedRegistration = await this.loadCachedRegistrationData()
         if (cachedRegistration === undefined) {
             cachedRegistration = await this.registerClient()
-            await this.saveRegisrationData(cachedRegistration)
+            await this.saveRegistrationData(cachedRegistration)
         }
 
         return await this.authorize(cachedRegistration)
@@ -355,9 +361,15 @@ interface UiHandler {
 export class BuilderIdConnectionBuilder {
     private static readonly getToken = keyedDebounce(BuilderIdConnectionBuilder._getToken.bind(this))
     public static uiHandler: UiHandler
+    private static tokenCacheDir: string
 
-    public static async build(uiHandler: UiHandler, startUrl: string = builderIdStartUrl): Promise<SsoConnection> {
+    public static async build(
+        uiHandler: UiHandler,
+        startUrl: string = builderIdStartUrl,
+        tokenCacheDir: string = DEFAULT_TOKEN_CACHE_DIR
+    ): Promise<SsoConnection> {
         BuilderIdConnectionBuilder.uiHandler = uiHandler
+        BuilderIdConnectionBuilder.tokenCacheDir = tokenCacheDir
 
         const awsBuilderIdSsoProfile = BuilderIdConnectionBuilder.createBuilderIdProfile(defaultScopes, startUrl)
         const connection = await BuilderIdConnectionBuilder.createConnection(awsBuilderIdSsoProfile)
@@ -392,7 +404,8 @@ export class BuilderIdConnectionBuilder {
                 scopes: profile.scopes,
                 region: profile.ssoRegion,
             },
-            BuilderIdConnectionBuilder.uiHandler
+            BuilderIdConnectionBuilder.uiHandler,
+            BuilderIdConnectionBuilder.tokenCacheDir
         )
 
         return provider