QOL improvements for setting up test projects (#839)

Author: edwardfoyle

.changeset/spotty-roses-notice.md

@@ -0,0 +1,2 @@
+---
+---

.gitignore

@@ -4,6 +4,9 @@ lib
 node_modules
 temp
 tsconfig.tsbuildinfo
+test-projects/*
+!test-projects/README.md
+!test-projects/.eslintrc.json
 
 .idea
 .eslintcache

.npmrc.local

@@ -1,5 +1,5 @@
 # this is a test config file used to publish locally
 //localhost:4873/:_authToken=garbage
 
-# this prevents the script-shell setting set by install:local from being overwritten when copying the localhost config
+# this prevents the script-shell setting set by setup:local from being overwritten when copying the localhost config
 script-shell=bash

CONTRIBUTING.md

@@ -47,24 +47,22 @@ Also check out the README files in the root of each package directory for a brie
 
 Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
 
-## Contribute to create-amplify package
-
-### Set up your local development environment
+## Set up your local development environment
 
 ```sh
 # clone project repo
 cd <project directory>
-npm run install:local
+npm run setup:local
 npm run test
 ```
 
-`npm run install:local` will run `npm install`, then build all packages in the project and run `npm link`
+`npm run setup:local` will run `npm install`, then build all packages in the project. It also links the amplify and create-amplify bin files into the project node_modules folder.
 
 `npm run test` will run all the unit tests in the project
 
-You should now be able to run the new `amplify` CLI.
+You should now be ready to start contributing to the project!
 
-#### Other helpful scripts
+### Other helpful scripts
 
 `npm run watch` will start the tsc server and watch for changes in all packages
 
@@ -76,6 +74,30 @@ You should now be able to run the new `amplify` CLI.
 
 `npm run e2e` will run the E2E test suite. Note: you must have valid AWS credentials configured locally to run this command successfully.
 
+### Testing changes locally
+
+For local testing we recommend writing unit tests that exercise the code you are modifying as you are making changes. Individual test files can be run using:
+
+```sh
+npm run test:dir packages/<package name>/lib/<file-name>.test.ts
+```
+
+> Note: You must rebuild using `npm run build` for tests to pick up your changes.
+
+Sometimes it's nice to have a test project to use as a testing environment for local changes. You can create test projects in the `local-testing` directory using
+
+```sh
+npm run setup:test-project <name>
+```
+
+This allows you to make local changes and immediately try them out in a test project. All projects that you create in this directory are gitignored.
+
+Depending on the scope of the change you are making, integration tests and/or E2E tests may be necessary.
+
+Integration tests are located [here](./packages/integration-tests/src/test-in-memory/). These tests operate mostly like unit tests but they exercise many different components and packages together. However, these tests do NOT make service calls. Test assertions can be set up using the [CDK assertions library](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.assertions-readme.html).
+
+E2E tests are located [here](./packages/integration-tests/src/test-e2e/). These tests exercise full end-to-end features with no mocking or stubbing. These tests do make service calls and therefore require valid AWS credentials to run. Tests should be added to this suite very sparingly as these tests are the most time consuming to run and maintain. If possible, additional features should be tested by adding on to existing tests rather than adding entirely new tests.
+
 ### Creating changesets
 
 This repo uses [changesets](https://github.com/changesets/changesets) for versioning and releasing changes.
@@ -85,7 +107,7 @@ This will start a walkthrough to author the changeset file. This file should be
 
 ### Publishing packages locally
 
-Publishing packages locally allows you to install local package changes as if they were published to NPM. This is useful for testing or demo scenarios.
+Publishing packages locally allows you to install local package changes as if they were published to NPM. This can be useful for testing or demo scenarios.
 
 To set up a local npm proxy and publish the current local state to the proxy, run `npm run vend`.
 This will start a local npm proxy using [Verdaccio](https://verdaccio.org/) and run `changeset version` and `changeset publish`.
@@ -120,45 +142,7 @@ At a minimum, each package needs:
 4. An `update:api` script in the `package.json` file
 5. A `typedoc.json` file
 6. An `.npmignore` file
-
-### Dev
-
-1. Publish to local
-
-You have to publish to local every time after making a change
-
-```sh
-# In amplify-backend repo
-npm run clean
-npm run install:local
-npm run build
-npm run vend
-```
-
-2. Prepare to install
-
-```sh
-# On your local machine
-rm -rf $(npm config get cache)/_npx # clean npm cache globally
-mkdir amplify-project-$(date +%Y-%m-%d) # create an empty folder. Alternatively, you can create a project by running e.g `npx create-next-app next-$(date +%Y-%m-%d)`.
-cd amplify-project-$(date +%Y-%m-%d) # cd into the empty folder
-```
-
-3. Create Amplify Project
-
-```sh
-# In the folder created in the previous step
-npm create amplify
-```
-
-### Test
-
-```sh
-# In amplify-backend repo
-npm run build # Have to build every time after making a change before running test
-npm run test:dir packages/create-amplify # Run all test in create-amplify
-npm run test:dir packages/create-amplify/src/<file-name>.test.ts # Run all test in one file. e.g. npm run test:dir packages/create-amplify/src/get_project_root.test.ts
-```
+7. A `README.md` file that gives a brief description of the intent of the package
 
 ## Licensing
 

package.json

@@ -17,7 +17,6 @@
     "docs": "typedoc",
     "e2e": "npm run test:dir packages/integration-tests/lib/test-e2e",
     "e2e:cleanup-resources": "tsx scripts/cleanup_e2e_resources.ts",
-    "install:local": "npm run set-script-shell && npm install && npm run build && npm run install:local --workspaces --if-present",
     "lint": "eslint --max-warnings 0 . && prettier --check .",
     "lint:fix": "eslint --cache --fix . && prettier --write .",
     "live-dependency-health-checks": "npm run test:dir packages/integration-tests/lib/test-live-dependency-health-checks",
@@ -26,6 +25,8 @@
     "publish": "tsx scripts/publish.ts",
     "publish:local": "tsx scripts/publish_local.ts",
     "set-script-shell": "npm config set script-shell bash --userconfig ./.npmrc",
+    "setup:local": "npm run set-script-shell && npm install && npm run build && npm link ./packages/cli ./packages/create-amplify",
+    "setup:test-project": "tsx scripts/setup_test_project.ts",
     "start:npm-proxy": "tsx scripts/start_npm_proxy.ts",
     "stop:npm-proxy": "tsx scripts/stop_npm_proxy.ts",
     "test": "npm run test:dir $(tsx scripts/get_unit_test_dir_list.ts)",

packages/cli/package.json

@@ -13,8 +13,7 @@
     "#package.json": "./package.json"
   },
   "scripts": {
-    "update:api": "api-extractor run --local",
-    "install:local": "npm link"
+    "update:api": "api-extractor run --local"
   },
   "repository": {
     "type": "git",

packages/create-amplify/package.json

@@ -12,7 +12,6 @@
     "create-amplify": "./lib/create_amplify.js"
   },
   "scripts": {
-    "install:local": "npm link",
     "update:api": "api-extractor run --local"
   },
   "license": "Apache-2.0",

packages/create-amplify/src/get_project_root.test.ts

@@ -1,4 +1,4 @@
-import { afterEach, describe, it } from 'node:test';
+import { afterEach, beforeEach, describe, it, mock } from 'node:test';
 import assert from 'assert';
 import fsp from 'fs/promises';
 import path from 'path';
@@ -8,6 +8,13 @@ import { AmplifyPrompter } from '@aws-amplify/cli-core';
 const originalEnv = process.env;
 
 void describe('getProjectRoot', () => {
+  const fsMkDirSyncMock = mock.method(fsp, 'mkdir', () => undefined);
+  mock.method(fsp, 'stat', () => Promise.reject(new Error()));
+
+  beforeEach(() => {
+    fsMkDirSyncMock.mock.resetCalls();
+  });
+
   afterEach(() => {
     process.env = originalEnv;
   });
@@ -18,32 +25,30 @@ void describe('getProjectRoot', () => {
     assert.equal(projectRoot, process.cwd());
   });
 
-  void it('returns the default project root directory if user do not pass anything', async (ctx) => {
+  void it('returns the default project root directory if user do not pass anything', async () => {
     process.env.npm_config_yes = 'false';
     const defaultProjectRoot = '.';
-    ctx.mock.method(AmplifyPrompter, 'input', () =>
+    mock.method(AmplifyPrompter, 'input', () =>
       Promise.resolve(defaultProjectRoot)
     );
     const projectRoot = await getProjectRoot();
 
     assert.equal(projectRoot, process.cwd());
   });
 
-  void it('returns the user provided project root directory', async (ctx) => {
+  void it('returns the user provided project root directory', async () => {
     process.env.npm_config_yes = 'false';
     const userInput = path.resolve('test', 'root');
-    ctx.mock.method(AmplifyPrompter, 'input', () => Promise.resolve(userInput));
+    mock.method(AmplifyPrompter, 'input', () => Promise.resolve(userInput));
     const projectRoot = await getProjectRoot();
 
     assert.equal(projectRoot, userInput);
   });
 
-  void it('creates the project root directory if the user provided absolute path does not exist', async (ctx) => {
+  void it('creates the project root directory if the user provided absolute path does not exist', async () => {
     process.env.npm_config_yes = 'false';
     const userInput = path.resolve(process.cwd(), 'test', 'root');
-    const fsMkDirSyncMock = ctx.mock.method(fsp, 'mkdir', () => undefined);
-    ctx.mock.method(fsp, 'stat', () => Promise.reject(new Error()));
-    ctx.mock.method(AmplifyPrompter, 'input', () => Promise.resolve(userInput));
+    mock.method(AmplifyPrompter, 'input', () => Promise.resolve(userInput));
 
     const projectRoot = await getProjectRoot();
 
@@ -52,12 +57,10 @@ void describe('getProjectRoot', () => {
     assert.equal(projectRoot, userInput);
   });
 
-  void it('creates the project root directory if the user provided relative path does not exist', async (ctx) => {
+  void it('creates the project root directory if the user provided relative path does not exist', async () => {
     process.env.npm_config_yes = 'false';
     const userInput = 'test';
-    const fsMkDirSyncMock = ctx.mock.method(fsp, 'mkdir', () => undefined);
-    ctx.mock.method(fsp, 'stat', () => Promise.reject(new Error()));
-    ctx.mock.method(AmplifyPrompter, 'input', () => Promise.resolve(userInput));
+    mock.method(AmplifyPrompter, 'input', () => Promise.resolve(userInput));
 
     const projectRoot = await getProjectRoot();
 

scripts/setup_test_project.ts

@@ -0,0 +1,57 @@
+import * as fsp from 'fs/promises';
+import * as path from 'path';
+import { fileURLToPath } from 'url';
+
+const projectName = process.argv[2];
+if (!projectName) {
+  throw new Error('Usage: npm run setup:test-project <name>');
+}
+
+const testProjectDir = new URL(
+  `../test-projects/${projectName}`,
+  import.meta.url
+);
+await fsp.mkdir(testProjectDir, { recursive: true });
+
+// reaching into the create-amplify package like this isn't great, but for a test utility I think it's acceptable
+// if it breaks, it will only break this local script, not anything in production
+const createAmplifyTemplateLocation = new URL(
+  '../packages/create-amplify/templates/basic-auth-data',
+  import.meta.url
+);
+
+// copy getting started project template
+await fsp.cp(createAmplifyTemplateLocation, testProjectDir, {
+  recursive: true,
+});
+
+// create minimal package.json
+// if you want to test out changes in a commonjs package, change the "type" field in the package.json of the test project after it is created
+const packageJson = {
+  name: projectName,
+  type: 'module',
+};
+
+const packageJsonPath = path.join(
+  fileURLToPath(testProjectDir),
+  'package.json'
+);
+
+await fsp.writeFile(packageJsonPath, JSON.stringify(packageJson, null, 2));
+
+// create minimal tsconfig.json
+const tsConfig = {
+  extends: '../../../tsconfig.base.json',
+  compilerOptions: {
+    moduleResolution: 'bundler',
+    module: 'es2022',
+  },
+};
+
+const tsConfigPath = path.join(
+  fileURLToPath(testProjectDir),
+  'amplify',
+  'tsconfig.json'
+);
+
+await fsp.writeFile(tsConfigPath, JSON.stringify(tsConfig, null, 2));

test-projects/.eslintrc.json

@@ -0,0 +1,5 @@
+{
+  "rules": {
+    "check-file/folder-naming-convention": "off"
+  }
+}