build: remove package.json syncing (aws#5629)

Removes the need to copy/sync package JSON contents from core ->
subproject extensions on build and package.

Summary of changes:
- Delete syncPackageJson.ts and handlePackageJson.ts
- Consolidate testing scripts and make small improvements (e.g.
validation, handle multiple directories to import tests from)
- package.json is now fully complete in `amazonq/`, `toolkit/`. Changes
to extensions should be made only the respective file (except for
icons).
- `toolkit/` runs unit, e2e, and integ tests from `core/`. It will also
run any of its own tests (but those don't exist yet).
- Web tests now moved to and run from `amazonq/`. This was decided
because `amazonq/` is more likely to support web than `toolkit/` at this
time. Also, it didn't seem possible to run the web tests from `core/`
like the other tests because it relies on webpack and a separate test
runner.
- No longer create web-based webpacks for `core/`

Follow up work:
- Get [packageJSON
properties](https://github.com/aws/aws-toolkit-vscode/blob/46b409c3353ce0dafefb796f5994c1fa88a23b47/packages/core/src/shared/vscode/env.ts#L61-L74)
from `context.extension.packageJSON` instead.
- Cleanup any remaining unneeded tasks in `.vscode/tasks.json`
- Move icons out of `core/package.json`.
- Continue to investigate: can we run web tests from core like other
tests? Toolkit may have a use in web mode one day.

---

<!--- REMINDER: Ensure that your PR meets the guidelines in
CONTRIBUTING.md -->

License: I confirm that my contribution is made under the terms of the
Apache 2.0 license.

Author: hayemaxi

buildspec/lint.yml

@@ -26,7 +26,8 @@ phases:
     build:
         commands:
             - export HOME=/home/codebuild-user
-            - npm run testCompile
+            - npm run compile -w packages/core
+            - npm run testCompile -w packages/ --if-present
             - npm run lint
             - VCS_COMMIT_ID="${CODEBUILD_RESOLVED_SOURCE_VERSION}"
             - CI_BUILD_URL=$(echo $CODEBUILD_BUILD_URL | sed 's/#/%23/g') # Encode `#` in the URL because otherwise the url is clipped in the Codecov.io site

buildspec/windowsTests.yml

@@ -30,7 +30,8 @@ phases:
 
     build:
         commands:
-            - npm run testCompile
+            - npm run compile -w packages/core
+            - npm run testCompile -w packages/ --if-present
             - npm run lint
             - $env:TEST_REPORT_DIR="$env:CODEBUILD_SRC_DIR/.test_reports"; npm run test
             - |

docs/TESTPLAN.md

@@ -45,8 +45,7 @@ The test suite has the following categories of tests:
 
 ## Test files
 
-Currently, most if not all testing code lives in the subproject `packages/core/`.
-For more information, see [arch_develop.md](./arch_develop.md#monorepo-structure)
+Currently, most testing code lives in the subproject `packages/core/` due to the move to monorepo. See [arch_develop.md](./arch_develop.md#monorepo-structure).
 
 -   `src/test/` : unit tests
     -   `src/test/globalSetup.test.ts` :
@@ -60,6 +59,8 @@ For more information, see [arch_develop.md](./arch_develop.md#monorepo-structure
 -   `.vscode/launch.json` : defines VSCode launch configs useful for Toolkit
     developers, e.g. the `Extension Tests` config runs all tests in `src/test/`.
 
+Many tests required running in an activated extension environment, but the core-lib is just a library and does not have any extension to activate itself. Core-lib tests are run from `packages/toolkit`, and web tests are run from `packages/amazonq`.
+
 ## How we test
 
 VSCode documentation describes an [extension testing](https://code.visualstudio.com/api/working-with-extensions/testing-extension)

docs/arch_develop.md

@@ -23,7 +23,7 @@ If you are considering contributing, please consider whether your implementation
 library or in `packages/toolkit`. If your work could be re-used by other packages (e.g. auth mechanisms,
 utilities), then it may belong in the core library. If instead you are adding something toolkit specific
 (eg. an integration to a new AWS service in the Explorer Tree), consider putting it in `packages/toolkit`.
-To import from the core library, please export your desired code using `index.ts` files, and add an appropriate `exports` statement
+To import from the core library, please export your desired code using `index.ts` files and add an appropriate `exports` statement
 in `packages/core/package.json`.
 
 Unless otherwise stated, the documentation throughout this project is referring to the code and
@@ -51,25 +51,9 @@ Current quirks of the current monorepo status that should be resolved/evaluated
 -   Pre-release only publishes packages/toolkit extension directly. It should be extended to other added extensions. See [`release.yml`](../.github/workflows/release.yml)
 -   VSCode does not support inheriting/extending `.vscode/` settings: https://github.com/microsoft/vscode/issues/15909
 
-Additional quirks introduced by creating a core library from the original extension code:
-
--   Tests are ran from `packages/core/`
--   Extension runs from `packages/toolkit`
--   Extension tests run from the core lib. Since some of the tests require an extension context/sandbox, we initiate a "fake" extension to run these tests. This is also why there are vscode extension properties in the package.json
--   Some of original extension code (that now lives in `packages/core`) depends on the package.json, specifically the contributes section. This section is very large AND needs to be present in both the core library and toolkit extension package.jsons. The core library code needs access to this section to create types, set up SAM debuggers, etc. The toolkit needs this section during packaging/debugging so that the extension can run in vscode. The short term solution was to creat a [build script](../packages/toolkit/scripts/build/handlePackageJson.ts) to copy necessary fields over to the toolkit extension during packaging and debugging.
-
 ### Contributes and Settings
 
-Some components of the core library depend on the `package.json`s of the extensions. One example of this is compile time checking of the extension's settings values. However, VSCode also requires a complete local `package.json` for the individual extensions during packaging. As a temporary workaround to this, we are using scripts to auto-populate the `package.json`s for the individual extensions from the core `package.json`.
-
--   [`packages/toolkit/../handlePackageJson.ts`](../packages/toolkit/scripts/build/handlePackageJson.ts)
-    -   Copies the entirety of the `contributes` and `engine` sections, except for `configuration.properties` relating to `packages/amazon`.
-    -   Restores to the original barebones `package.json` after packaging/debugging, to avoid a large amount of duplicate code.
-    -   To develop for the Toolkit extension: add all changes to `packages/core/package.json`
--   [`packages/amazonq/../syncPackageJson.ts`](../packages/amazonq/scripts/build/syncPackageJson.ts)
-    -   Moves all Amazon Q related `configuration.properties` to the local `package.json` only, overwriting anything that exists with the same name locally.
-    -   Does not restore, it is a superset of what exists in `packages/core` for `configuration.properties`.
-    -   To develop for the Amazon Q extension: add all changes to `packages/amazonq/package.json`, EXCEPT for settings that are references by code in the core library, or settings that already exist in the core `package.json`
+`packages/toolkit/` and `packages/amazonq` have independent extension packageJSON files. They do not rely on `packages/core/package.json`. However, to get typed icons in the core-lib we require a place to store the icon entries. This currently happens in `packages/core/package.json`. See [`icons.md`](./icons.md) for more information.
 
 If you are modifying or registering new debuggers in VS Code via the `debuggers` contribution point, you may need to regenerate the [definitions file](../packages/core/src/shared/sam/debugger/awsSamDebugConfiguration.gen.ts). After updating ['toolkit/package.json'](../packages/toolkit/package.json), run `npm run generateConfigurationAttributes -w packages/toolkit`
 

package-lock.json

@@ -99,6 +99,25 @@
                 "order": 2
             }
         },
+        {
+            "name": "Extension Tests (web)",
+            "type": "extensionHost",
+            "debugWebWorkerHost": true,
+            "request": "launch",
+            "args": [
+                "--disable-extension=amazonwebservices.aws-toolkit-vscode",
+                "--extensionDevelopmentPath=${workspaceFolder}",
+                "--extensionDevelopmentKind=web",
+                "--extensionTestsPath=${workspaceFolder}/dist/test/web/testRunnerWebCore",
+                "${workspaceFolder}/../core/dist/src/testFixtures/workspaceFolder"
+            ],
+            "outFiles": ["${workspaceFolder}/dist/**/*.js", "${workspaceFolder}/../core/dist/**/*.js"],
+            "preLaunchTask": "webRun",
+            "presentation": {
+                "group": "3_ExtensionTests",
+                "order": 3
+            }
+        },
         {
             "name": "E2E Test (current file)",
             "type": "extensionHost",

packages/amazonq/.vscode/launch.json

@@ -109,7 +109,6 @@
             // Since `webpack --watch` never terminates (but finishes packaging at some point),
             // VS Code uses this to parse the CLI output to pattern match something that indicates it is done
             "problemMatcher": "$ts-webpack-watch",
-            "dependsOn": ["copyPackageJson"],
             "options": {
                 "cwd": "${workspaceFolder}/../../packages/core"
             }
@@ -124,7 +123,7 @@
             "label": "webRunTerminate",
             "command": "echo ${input:webRunTerminate}",
             "type": "shell",
-            "dependsOn": ["restorePackageJson"]
+            "dependsOn": []
         },
         // ---------- End: Web Mode Tasks ----------
         {

packages/amazonq/.vscode/tasks.json

@@ -46,19 +46,20 @@
     "browser": "./dist/src/extensionWeb",
     "scripts": {
         "vscode:prepublish": "npm run clean && npm run buildScripts && webpack --mode production",
-        "buildScripts": "npm run generateNonCodeFiles && npm run copyFiles && npm run generateIcons && npm run generateSettings && npm run syncPackageJson && tsc -p ./ --noEmit",
+        "buildScripts": "npm run generateNonCodeFiles && npm run copyFiles && npm run generateIcons && npm run generateSettings && tsc -p ./ --noEmit",
         "generateNonCodeFiles": "ts-node ../../scripts/generateNonCodeFiles.ts",
         "copyFiles": "ts-node ./scripts/build/copyFiles.ts",
-        "syncPackageJson": "ts-node ./scripts/build/syncPackageJson.ts",
         "clean": "ts-node ../../scripts/clean.ts dist/ LICENSE NOTICE",
-        "compile": "npm run clean && npm run buildScripts && webpack --mode development",
+        "compile": "npm run clean && npm run buildScripts && webpack",
+        "compileDev": "npm run testCompile && webpack --mode development",
         "compileOnly": "tsc -p ./",
         "package": "ts-node ../../scripts/package.ts",
         "lint": "true",
         "watch": "npm run clean && npm run buildScripts && tsc -watch -p ./",
         "testCompile": "npm run clean && npm run buildScripts && npm run compileOnly",
-        "test": "npm run testCompile && c8 --allowExternal ts-node ../core/scripts/test/test.ts dist/test/unit/index.js ../core/dist/src/testFixtures/workspaceFolder",
-        "testE2E": "npm run testCompile && c8 --allowExternal ts-node ../core/scripts/test/testE2E.ts dist/test/e2e/index.js ../core/dist/src/testFixtures/workspaceFolder",
+        "test": "npm run testCompile && c8 --allowExternal ts-node ../core/scripts/test/launchTest.ts unit dist/test/unit/index.js ../core/dist/src/testFixtures/workspaceFolder",
+        "testE2E": "npm run testCompile && c8 --allowExternal ts-node ../core/scripts/test/launchTest.ts e2e dist/test/e2e/index.js ../core/dist/src/testFixtures/workspaceFolder",
+        "testWeb": "npm run compileDev && c8 --allowExternal ts-node ../core/scripts/test/launchTest.ts web dist/test/web/testRunnerWebCore.js",
         "webRun": "npx @vscode/test-web --open-devtools --browserOption=--disable-web-security --waitForDebugger=9222 --extensionDevelopmentPath=. .",
         "webWatch": "npm run clean && npm run buildScripts && webpack --mode development --watch",
         "serve": "webpack serve --config-name mainServe --mode development",

packages/amazonq/package.json

@@ -7,9 +7,7 @@ import { runTests } from 'aws-core-vscode/test'
 import { VSCODE_EXTENSION_ID } from 'aws-core-vscode/utils'
 
 export function run(): Promise<void> {
-    return runTests(
-        process.env.TEST_DIR ?? 'test/e2e',
-        ['../../core/dist/src/testInteg/globalSetup.test.ts'],
-        VSCODE_EXTENSION_ID.amazonq
-    )
+    return runTests(process.env.TEST_DIR ?? 'test/e2e', VSCODE_EXTENSION_ID.amazonq, [
+        '../../core/dist/src/testInteg/globalSetup.test.ts',
+    ])
 }