Merge remote-tracking branch 'origin/master' into feature/dynamodb

Author: LokeshDogga13

.github/workflows/node.js.yml

@@ -21,6 +21,8 @@ jobs:
         env:
             VSCODE_TEST_VERSION: ${{ matrix.vscode-version }}
             NODE_OPTIONS: '--max-old-space-size=8192'
+            AWS_TOOLKIT_TEST_CACHE_DIR: '/tmp/.vscode-test/'
+            AWS_TOOLKIT_TEST_USER_DIR: '/tmp/.vscode-test/user-data/'
         steps:
             - uses: actions/checkout@v4
             - name: Use Node.js ${{ matrix.node-version }}
@@ -88,6 +90,8 @@ jobs:
         env:
             VSCODE_TEST_VERSION: ${{ matrix.vscode-version }}
             NODE_OPTIONS: '--max-old-space-size=8192'
+            AWS_TOOLKIT_TEST_CACHE_DIR: '/tmp/.vscode-test/'
+            AWS_TOOLKIT_TEST_USER_DIR: '/tmp/.vscode-test/user-data/'
         steps:
             - uses: actions/checkout@v4
             - name: Use Node.js ${{ matrix.node-version }}

.gitignore

@@ -20,6 +20,8 @@ __pycache__
 
 # Auto generated definitions
 packages/*/src/**/*.gen.ts
+!packages/core/src/**/awsSamDebugConfiguration.gen.ts
+!packages/core/src/shared/settings-*.gen.ts
 src.gen/*
 
 # Test reports

CONTRIBUTING.md

@@ -132,7 +132,7 @@ You can also use these NPM tasks (see `npm run` for the full list):
 
     1. Declare a global unhandledRejection handler.
         ```ts
-        process.on('unhandledRejection', e => {
+        process.on('unhandledRejection', (e) => {
             getLogger('channel').error(
                 localize(
                     'AWS.channel.aws.toolkit.activation.error',
@@ -549,7 +549,7 @@ For extensions to contribute their own codicons, VSCode requires a font file as
 As a simple example, let's say I wanted to add a new icon for CloudWatch log streams. I would do the following:
 
 1. Place the icon in `resources/icons/aws/cloudwatch`. I'l name the icon `log-stream.svg`.
-1. Use `npm run generatePackage` to update `package.json`. Commit this change with the new icon.
+1. Use `npm run generateIcons` to update `package.json`. Commit this change with the new icon.
 1. You can now use the icon in the Toolkit:
 
     ```ts

codecov.yml

@@ -27,6 +27,21 @@ coverage:
                     - packages/core/src/amazonqFeatureDev/*
                 flags:
                     - 'amazonqFeatureDev'
+            amazonqGumby:
+                paths:
+                    - packages/core/src/amazonqGumby/*
+            codewhispererChat:
+                paths:
+                    - packages/core/src/codewhispererChat/*
+            applicationcomposer:
+                paths:
+                    - packages/core/src/applicationcomposer/*
+            stepFunctions:
+                paths:
+                    - packages/core/src/stepFunctions/*
+            threatComposer:
+                paths:
+                    - packages/core/src/threatComposer/*
         patch: no
         changes: no
 

docs/TESTPLAN.md

@@ -104,7 +104,7 @@ Checking the state works well if user interactions are not required by the code
 To handle this, test code can register event handler that listen for when a certain type of UI element is shown. For example, if we wanted to always accept the first item of a quick pick we can do this:
 
 ```ts
-getTestWindow().onDidShowQuickPick(async picker => {
+getTestWindow().onDidShowQuickPick(async (picker) => {
     // Some pickers load items asychronously
     // Wait until the picker is not busy before accepting an item
     await picker.untilReady()
@@ -121,3 +121,9 @@ const secondPicker = await pickers.next()
 ```
 
 Exceptions thrown within one of these handlers will cause the current test to fail. This allows you to make assertions within the callback without worrying about causing the test to hang.
+
+## Common issues
+
+### Stubbing VSCode outside of core
+
+-   Stubbing VSCode imports (like executeCommand) does not work outside of core. For now you will need to put any tests that require spying/stubbing VSCode imports in core until we move more source files into the amazon q package

docs/arch_develop.md

@@ -71,39 +71,37 @@ Some components of the core library depend on the `package.json`s of the extensi
     -   Does not restore, it is a superset of what exists in `packages/core` for `configuration.properties`.
     -   To develop for the Amazon Q extension: add all changes to `packages/amazonq/package.json`, EXCEPT for settings that are references by code in the core library, or settings that already exist in the core `package.json`
 
-## Shared vs Common names
+If you are modifying or registering new debuggers in VS Code via the `debuggers` contribution point, you may need to regenerate the [definitions file](../packages/core/src/shared/sam/debugger/awsSamDebugConfiguration.gen.ts). After updating ['toolkit/package.json'](../packages/toolkit/package.json), run `npm run generateConfigurationAttributes -w packages/toolkit`
 
-In this repo, the keywords **"shared"** and **"common"** have specific meanings in the context of file/folder names.
+## `web`, `node`, `common`, `shared` naming conventions
 
-### "common"
+This project can run in different environments, eg Web mode (in the browser with no compute backend), or in Node.js on your desktop (the most common way).
+A problem arises when we use code that is exclusive to one environment, an example being Node.js' Filesystem module which will fail in Web mode.
 
-Code within a folder/file that has the "common" keyword implies that it can run in any environment. Examples of environments are: Web mode, or Node.js (local desktop)
+To ensure developers use compatible code for their environment we have subfolders in each topic which contains environment specific code in a single place.
 
-We need this distinction since not all code can run in any environment. A common example is filesystem code, where the actual implementation used could work in Node.js but not in Web mode.
+Using this file tree as reference, here are the rules:
 
-### "shared"
-
-Code within a folder/file that has the "shared" keyword implies that it is intended to be reused wherever it can be. This is generalized code that "Feature A" or "Feature B" could use if it works for their use case.
-
-An example is the `waitUntil()` function which continuously invokes an arbitrary callback function until it succeeds.
-
-> NOTE: Something that is "shared" does not mean it is "common", as it could be reused in different places but only work in Node.js for example.
+```
+src/
+├── myTopic/
+│   ├── {file}.ts
+│   ├── node/
+│   │   └── {file}.ts
+│   └── web/
+│       └── {file}.ts
+└── shared/
+```
 
-### How to apply this
+-   `myTopic/` is the general name of the folder, eg `request` for http requests.
+-   `myTopic/{file}.ts` is for code that works in any environment, we refer to this as `"common"` code.
+-   `node/{file}.ts` is for code that works exclusively in Node.js.
+-   `web/{file}.ts` is for code that works exclusively in Web mode.
+-   `shared/` is for code that is intended to be reused, i.e general purpose utils.
+    -   Note environment specific code should be place in to a `web/` or `node/` subfolder.
+    -   If the code is not in a subfolder then it is considered `shared common` code.
 
--   Aim to make code compatible with "common" from the beginning.
--   In a "topic" folder, if you have common code, create a subfolder named "common" and add your common code to there.
-    ```
-    src/
-      |
-      myTopic/
-        |
-        common/
-        nonCommon.ts
-    ```
--   See if yours, or existing code can be moved in to a "shared" folder. Maybe it can be easily modified to become "shared".
--   If there is no "shared" or "common" naming used for the file/folder, then assume it only works in Node.js.
--   In the rare case your code only works in Web mode, create a `web` subfolder for that code.
+> IMPORTANT: The current codebase does not fully follow this convention yet, the transition is being done incrementally. Due to this, code that is `"common"` may not actually be common yet. If you run in to this, please move that code to the appropriate subfolder.
 
 ## Commands
 
@@ -292,22 +290,22 @@ Commands and events are defined on the backend via sub-classes of `VueWebview`.
     ```ts
     client
         .foo()
-        .then(response => console.log(response))
-        .catch(err => console.log(err))
+        .then((response) => console.log(response))
+        .catch((err) => console.log(err))
     ```
 
     The backend protocol is allowed to throw errors. These result in rejected Promises on the frontend.
 
 -   Registering for events:
 
     ```ts
-    client.onBar(num => console.log(num))
+    client.onBar((num) => console.log(num))
     ```
 
 -   Methods called `init` will only return data on the initial webview load:
 
     ```ts
-    client.init(data => (this.data = data ?? this.data))
+    client.init((data) => (this.data = data ?? this.data))
     ```
 
 ## Webviews (non Vue)
@@ -493,8 +491,8 @@ class ExampleWizard extends Wizard<ExampleState> {
             { label: '1', data: 1 },
             { label: '2', data: 2 },
         ]
-        this.form.bar.bindPrompter(state => createQuickPick(items, { title: `Select a number (${state.foo})` }), {
-            showWhen: state => state.foo?.length > 5,
+        this.form.bar.bindPrompter((state) => createQuickPick(items, { title: `Select a number (${state.foo})` }), {
+            showWhen: (state) => state.foo?.length > 5,
         })
     }
 }

docs/icons.md

@@ -1,15 +1,17 @@
 # Icons
 
-All icons that are used in the Toolkit can be found in `resources/icons`.
+All icons that are used in the extensions can be found in `resources/icons`.
 
-A [build script](../../scripts/build/generateIcons.ts) generates Toolkit artifacts:
+A [build script](../scripts/generateIcons.ts) generates extension artifacts in [core/](../packages/core/):
 
 -   `resources/icons/cloud9/generated`
 -   `resources/fonts/aws-toolkit-icons.woff`
 -   `resources/css/icons.css`
--   `contributes.icons` in [package.json](../../package.json)
+-   `contributes.icons` in [amazonq package.json](../packages/amazonq/package.json) and [toolkit package.json](../packages/toolkit/package.json)
 
-This script should be ran using `npm run generatePackage` after making updates. Any changes made to `package.json` should be committed with the relevant icons.
+This script should be ran using `npm run generateIcons` after making updates. Any changes made to `package.json` should be committed with the relevant icons. Type checking in `core/` relies on the entries in `core/package.json`. However, the individual extensions require entries in their `package.json`s as well. Currently, resources (including icons) are shared between `core/` and the individual extensions. If `contributes.icons` in each of the extensions does not match the entry in `core/`, then CI will fail.
+
+To sync the icons to the individual extensions, run `npm run copyFiles && npm run generateIcons` for each extension.
 
 ## Fonts
 

docs/telemetry.md

@@ -139,3 +139,124 @@ Finally, if `setupStep2()` was the thing that failed we would see a metric like:
     ...
 }
 ```
+
+## Adding a "Stack Trace" to your metric
+
+### Problem
+
+Common example: _"I have a function, `thisFailsSometimes()` that is called in multiple places. The function sometimes fails, I know from telemetry, but I do not know if it is failing when it is a specific caller. If I knew the call stack/trace that it took to call my function that would help me debug."_
+
+```typescript
+function outerA() {
+    thisFailsSometimes(1) // this succeeds
+}
+
+function outerB() {
+    thisFailsSometimes(0) // this fails
+}
+
+function thisFailsSometimes(num: number) {
+    return telemetry.my_Metric.run(() => {
+        if (number === 0) {
+            throw Error('Cannot be 0')
+        }
+        ...
+    })
+}
+```
+
+### Solution
+
+Add a value to `function` in the options of a `run()`. This will result in a stack of functions identifiers that were previously called
+before `thisFailsSometimes()` was run. You can then retrieve the stack in the `run()` of your final metric using `getFunctionStack()`.
+
+```typescript
+function outerA() {
+    telemetry.my_Metric.run(() => thisFailsSometimes(1), { functionId: { name: 'outerA' }})
+}
+
+function outerB() {
+    telemetry.my_Metric.run(() => thisFailsSometimes(0), { functionId: { source: 'outerB' }})
+}
+
+function thisFailsSometimes(num: number) {
+    return telemetry.my_Metric.run(() => {
+        telemetry.record({ theCallStack: asStringifiedStack(telemetry.getFunctionStack())})
+        if (number === 0) {
+            throw Error('Cannot be 0')
+        }
+        ...
+    }, { functionId: { name: 'thisFailsSometimes' }})
+}
+
+// Results in a metric: { theCallStack: 'outerB:thisFailsSometimes', result: 'Failed' }
+// { theCallStack: 'outerB:thisFailsSometimes' } implies 'outerB' was run first, then 'thisFailsSometimes'. See docstrings for more info.
+outerB()
+```
+
+### Important Notes
+
+-   If a nested function does not use a `run()` then it will not be part of the call stack.
+
+    ```typescript
+    function a() {
+        return telemetry.my_Metric.run(() => b(), { functionId: { name: 'a' } })
+    }
+
+    function b() {
+        return c()
+    }
+
+    function c() {
+        return telemetry.my_Metric.run(() => asStringifiedStack(telemetry.getFunctionStack()), {
+            functionId: { name: 'c' },
+        })
+    }
+
+    c() // result: 'a:c', note that 'b' is not included
+    ```
+
+-   If you are using `run()` with a class method, you can also add the class to the entry for more context
+
+    ```typescript
+    class A {
+        a() {
+            return telemetry.my_Metric.run(() => this.b(), { functionId: { name: 'a', class: 'A' } })
+        }
+
+        b() {
+            return telemetry.my_Metric.run(() => asStringifiedStack(telemetry.getFunctionStack()), {
+                functionId: { name: 'b', class: 'A' },
+            })
+        }
+    }
+
+    const inst = new A()
+    inst.a() // 'A#a,b'
+    ```
+
+-   If you do not want your `run()` to emit telemetry, set `emit: false` in the options
+
+    ```typescript
+    function a() {
+        return telemetry.my_Metric.run(() => b(), { functionId: { name: 'a' }, emit: false })
+    }
+    ```
+
+-   If you want to add to the function stack, but don't have a specific metric to use,
+    use the metric named `function_call`. Also look to set `emit: false` if you do not
+    want it to emit telemetry.
+
+    ```typescript
+    function a() {
+        return telemetry.function_call.run(() => b(), { functionId: { name: 'a' }, emit: false })
+    }
+    ```
+
+-   If your function name is generic, look to make it unique so there is no confusion.
+
+    ```typescript
+    function a() {
+        return telemetry.my_Metric.run(() => b(), { functionId: { name: 'aButMoreUnique' } })
+    }
+    ```

docs/web.md

@@ -85,7 +85,7 @@ VS Code window, in the background it is running in a Browser context.
 
 ## Adding Web mode specific npm modules
 
-If you need to manage npm modules required for Web mode, such as a [browserfied module](https://www.npmjs.com/package/os-browserify), see [the documentation here](../packages/core/src/web/README.md).
+If you need to manage npm modules required for Web mode, such as a [browserfied module](https://www.npmjs.com/package/os-browserify), see [the documentation here](../packages/core/src/web/README.md#packagejson).
 
 ## Finding incompatible transitive dependencies