Merge remote-tracking branch 'upstream/master' into cwspr-usergroup

Author: Will-ShaoHua

.eslintrc.js

@@ -173,13 +173,15 @@ module.exports = {
                             "Avoid importing from the core lib's dist/ folders; please use directly from the core lib defined exports.",
                     },
                 ],
+                // The following will place an error on the `fs-extra` import since we do not want it to be used for browser compatibility reasons.
+                paths: [
+                    {
+                        name: 'fs-extra',
+                        message:
+                            'Avoid fs-extra, use shared/fs/fs.ts. Notify the Toolkit team if your required functionality is not available.',
+                    },
+                ],
             },
-            // The following will place an error on the `fs-extra` import since we do not want it to be used for browser compatibility reasons.
-            // {
-            //     name: 'fs-extra',
-            //     message:
-            //         'Avoid fs-extra, use shared/fs/fs.ts. Notify the Toolkit team if your required functionality is not available.',
-            // },
         ],
     },
 }

buildspec/lint.yml

@@ -26,7 +26,8 @@ phases:
     build:
         commands:
             - export HOME=/home/codebuild-user
-            - npm run testCompile
+            - npm run compile -w packages/core
+            - npm run testCompile -w packages/ --if-present
             - npm run lint
             - VCS_COMMIT_ID="${CODEBUILD_RESOLVED_SOURCE_VERSION}"
             - CI_BUILD_URL=$(echo $CODEBUILD_BUILD_URL | sed 's/#/%23/g') # Encode `#` in the URL because otherwise the url is clipped in the Codecov.io site

buildspec/windowsTests.yml

@@ -30,7 +30,8 @@ phases:
 
     build:
         commands:
-            - npm run testCompile
+            - npm run compile -w packages/core
+            - npm run testCompile -w packages/ --if-present
             - npm run lint
             - $env:TEST_REPORT_DIR="$env:CODEBUILD_SRC_DIR/.test_reports"; npm run test
             - |

docs/TESTPLAN.md

@@ -45,8 +45,7 @@ The test suite has the following categories of tests:
 
 ## Test files
 
-Currently, most if not all testing code lives in the subproject `packages/core/`.
-For more information, see [arch_develop.md](./arch_develop.md#monorepo-structure)
+Currently, most testing code lives in the subproject `packages/core/` due to the move to monorepo. See [arch_develop.md](./arch_develop.md#monorepo-structure).
 
 -   `src/test/` : unit tests
     -   `src/test/globalSetup.test.ts` :
@@ -60,6 +59,8 @@ For more information, see [arch_develop.md](./arch_develop.md#monorepo-structure
 -   `.vscode/launch.json` : defines VSCode launch configs useful for Toolkit
     developers, e.g. the `Extension Tests` config runs all tests in `src/test/`.
 
+Many tests required running in an activated extension environment, but the core-lib is just a library and does not have any extension to activate itself. Core-lib tests are run from `packages/toolkit`, and web tests are run from `packages/amazonq`.
+
 ## How we test
 
 VSCode documentation describes an [extension testing](https://code.visualstudio.com/api/working-with-extensions/testing-extension)

docs/arch_develop.md

@@ -23,7 +23,7 @@ If you are considering contributing, please consider whether your implementation
 library or in `packages/toolkit`. If your work could be re-used by other packages (e.g. auth mechanisms,
 utilities), then it may belong in the core library. If instead you are adding something toolkit specific
 (eg. an integration to a new AWS service in the Explorer Tree), consider putting it in `packages/toolkit`.
-To import from the core library, please export your desired code using `index.ts` files, and add an appropriate `exports` statement
+To import from the core library, please export your desired code using `index.ts` files and add an appropriate `exports` statement
 in `packages/core/package.json`.
 
 Unless otherwise stated, the documentation throughout this project is referring to the code and
@@ -51,25 +51,9 @@ Current quirks of the current monorepo status that should be resolved/evaluated
 -   Pre-release only publishes packages/toolkit extension directly. It should be extended to other added extensions. See [`release.yml`](../.github/workflows/release.yml)
 -   VSCode does not support inheriting/extending `.vscode/` settings: https://github.com/microsoft/vscode/issues/15909
 
-Additional quirks introduced by creating a core library from the original extension code:
-
--   Tests are ran from `packages/core/`
--   Extension runs from `packages/toolkit`
--   Extension tests run from the core lib. Since some of the tests require an extension context/sandbox, we initiate a "fake" extension to run these tests. This is also why there are vscode extension properties in the package.json
--   Some of original extension code (that now lives in `packages/core`) depends on the package.json, specifically the contributes section. This section is very large AND needs to be present in both the core library and toolkit extension package.jsons. The core library code needs access to this section to create types, set up SAM debuggers, etc. The toolkit needs this section during packaging/debugging so that the extension can run in vscode. The short term solution was to creat a [build script](../packages/toolkit/scripts/build/handlePackageJson.ts) to copy necessary fields over to the toolkit extension during packaging and debugging.
-
 ### Contributes and Settings
 
-Some components of the core library depend on the `package.json`s of the extensions. One example of this is compile time checking of the extension's settings values. However, VSCode also requires a complete local `package.json` for the individual extensions during packaging. As a temporary workaround to this, we are using scripts to auto-populate the `package.json`s for the individual extensions from the core `package.json`.
-
--   [`packages/toolkit/../handlePackageJson.ts`](../packages/toolkit/scripts/build/handlePackageJson.ts)
-    -   Copies the entirety of the `contributes` and `engine` sections, except for `configuration.properties` relating to `packages/amazon`.
-    -   Restores to the original barebones `package.json` after packaging/debugging, to avoid a large amount of duplicate code.
-    -   To develop for the Toolkit extension: add all changes to `packages/core/package.json`
--   [`packages/amazonq/../syncPackageJson.ts`](../packages/amazonq/scripts/build/syncPackageJson.ts)
-    -   Moves all Amazon Q related `configuration.properties` to the local `package.json` only, overwriting anything that exists with the same name locally.
-    -   Does not restore, it is a superset of what exists in `packages/core` for `configuration.properties`.
-    -   To develop for the Amazon Q extension: add all changes to `packages/amazonq/package.json`, EXCEPT for settings that are references by code in the core library, or settings that already exist in the core `package.json`
+`packages/toolkit/` and `packages/amazonq` have independent extension packageJSON files. They do not rely on `packages/core/package.json`. However, to get typed icons in the core-lib we require a place to store the icon entries. This currently happens in `packages/core/package.json`. See [`icons.md`](./icons.md) for more information.
 
 If you are modifying or registering new debuggers in VS Code via the `debuggers` contribution point, you may need to regenerate the [definitions file](../packages/core/src/shared/sam/debugger/awsSamDebugConfiguration.gen.ts). After updating ['toolkit/package.json'](../packages/toolkit/package.json), run `npm run generateConfigurationAttributes -w packages/toolkit`
 

docs/arch_features.md

@@ -39,6 +39,13 @@ For connecting a new VSCode _terminal_, remote connect works like this:
 1. Toolkit [builds a session-manager-plugin command](https://github.com/aws/aws-toolkit-vscode/blob/c77fc076fd0ed837d077bc0318716b711a2854c8/packages/core/src/ecs/util.ts#L92-L104) and [passes it to a new VSCode Terminal](https://github.com/aws/aws-toolkit-vscode/blob/c77fc076fd0ed837d077bc0318716b711a2854c8/packages/core/src/ecs/commands.ts#L141-L147).
 1. VSCode displays the terminal, so the user can enter shell commands on the remote machine.
 
+For EC2 specifically, there are a few additional steps:
+
+1. If connecting to EC2 instance via remote window, the toolkit generates temporary SSH keys (30 second lifetime), with the public key sent to the remote instance.
+    - Key type is ed25519 if supported, or RSA otherwise.
+1. If insufficient permissions are detected on the attached IAM role, toolkit will prompt to add an inline policy with the necessary actions.
+1. If SSM sessions remain open after closing the window/terminal, the toolkit will terminate them on-shutdown, or when starting another session to the same instance.
+
 ### Implementation of remote connect
 
 These modules show how to use and extend the "remote connect" functionality:

docs/telemetry.md

@@ -260,3 +260,71 @@ outerB()
         return telemetry.my_Metric.run(() => b(), { functionId: { name: 'aButMoreUnique' } })
     }
     ```
+
+## Tracing Telemetry Events
+
+All telemetry events include a traceId in addition to other attributes. Traceids allow for improved tracking and correlation of related events across a single operation or user flow.
+
+### What is a traceId?
+
+A traceId is a unique identifier that is generated for the top-level telemetry event in a flow and then propagated to all subsequent related events. This allows us to group and analyze all events associated with a particular operation.
+
+### How it works
+
+1. When a top-level telemetry event is created (e.g., `vscode_executeCommand`), a new traceId is generated.
+2. This traceId is then attached to all subsequent related telemetry events that occur as part of the same operation or flow.
+3. The traceId remains consistent for all events within the same flow
+
+### Example
+
+Consider a flow where `vscode_executeCommand` triggers `amazonq_enterFocusChat` and `amazonq_openChat`. The resulting telemetry events would look like this:
+
+```
+vscode_executeCommand:
+traceId: 'aaaaa-aaaaa-aaaaa-aaaaa-aaaaa'
+
+amazonq_enterFocusChat
+traceId: 'aaaaa-aaaaa-aaaaa-aaaaa-aaaaa'
+
+amazonq_openChat
+traceId: 'aaaaa-aaaaa-aaaaa-aaaaa-aaaaa'
+```
+
+allowing us to look up `traceId=aaaaa-aaaaa-aaaaa-aaaaa-aaaaa` in our telemetry instance and find all the related events.
+
+For more information visit the OpenTelemetry documentation on traces: https://opentelemetry.io/docs/concepts/signals/traces/
+
+### Manual Trace ID Instrumentation
+
+In certain scenarios you may need to manually instrument disjoint flows to track how a `traceId` propagates through them. e.g.
+
+1. Measuring the time it takes for a message to travel from Amazon Q chat, through VS Code, and back to the customer.
+2. Determining the duration for Amazon Q inline to display a message to the user.
+
+In these cases, where there isn't a direct hierarchy of function calls, manual instrumentation of the `traceId` is necessary.
+
+#### Implementation Options
+
+#### 1. When not currently running in a span
+
+If you're not within an active span and you know the `traceId` you want to use:
+
+```javascript
+telemetry.withTraceId(() => {
+    // Code to be executed within this trace
+}, 'myTraceId')
+```
+
+This method wraps the provided function with the specified traceId
+
+#### 2. When currently running in a span
+
+If you're already executing within a span (e.g., vscode_executeCommand) and you know the traceId you want to use:
+
+```javascript
+telemetry.record({
+    traceId: 'myTraceId',
+})
+```
+
+This approach records the traceId for the current span and all future spans within the same execution context.

docs/vscode_behaviors.md

@@ -0,0 +1,31 @@
+# VS Code Behaviors
+
+Many VS Code behavoirs for certain APIs or user interactions with the IDE are not clearly documented,
+or documented at all. Please add any findings to this document.
+
+## `deactivate()` - extension shutdown
+
+This method is defined as part of the VS Code extension API, and is run on a **graceful** shutdown
+for each extension.
+
+-   Our extension and its `deactivate()` function are in the Extension Host process [1]
+-   The Extension Host process has at most 5 seconds to shut down, after which it will exit. [1]
+-   The vscode API will be unreliable at deactivation time. So certain VSC APIs like the filesystem may not work. [1]
+    -   The VSC Filesystem API has been confirmed to not work
+-   In `Run & Debug` mode, closing the Debug IDE instance behaves differently depending on how it is closed
+    -   The regular close button in the Debug IDE instance results in a graceful shutdown
+    -   The red square in the root IDE instance to stop the debugging session results on a non-graceful shutdown, meaning `deactivate()` is not run.
+-   `Reload Window` triggers `deactivate()`
+
+Sources:
+
+-   [[1]](https://github.com/Microsoft/vscode/issues/47881#issuecomment-381910587)
+-   [[2]](https://github.com/microsoft/vscode/issues/122825#issuecomment-814218149)
+
+## State (`globalState`, `Memento`)
+
+TODO:
+
+-   How it behaves between remote (ssh) and local
+-   How it is not completely reliable. Reads/writes have no guarantee to work (though we are fine in most cases)
+-   How it can break as observed with crash monitoring work. At a certain point writes were seemingly succeeding, but did not actually propagate to all IDE instances.