Migrate to a pnpm monorepo (#1281)

- closes #945
- closes #1044
- depends on #1322
- depends on #1327

## Checklist

- [x] Split into smaller PRs for ease of review
- [x] Try DX tests again
#1289
- [x] I have updated the
[docs](https://github.com/cursorless-dev/cursorless/tree/main/docs) and
[cheatsheet](https://github.com/cursorless-dev/cursorless/tree/main/cursorless-talon/src/cheatsheet)
- [x] File issue for speeding up cheatsheet / making it not optional
locally #1307
- [x] Set package manager in package.json
https://nodejs.org/api/packages.html#packagemanager
- [x] Re-add lint rule to prevent api types from importing anything
- [x] Try local install
  - [x] Make sure cheatsheet works
- [x] Make sure local extension build depends on building cheatsheet, or
allows no cheatsheet
- [x] Figure out why we're getting so many extra `internal` modules
- [x] Update #931 to
indicate we're now patching instead of swizzling
- [x] Make sure lint rules are actually running in CI
- [x] Check that we've addressed everything in
#945
- [x] Incorporate changes from
#1166
- [x] ~~Make sure this PR doesn't break doc links; see
#942 (comment)
There's no way to avoid these links getting broken if we upgrade
typedoc; filed #1304
to track
- [x] Revert #1284
- [x] I have added
[tests](https://www.cursorless.org/docs/contributing/test-case-recorder/)
- [x] I have not broken the cheatsheet
- [x] Generate cursorless-vscode package.json automatically, and change
the name we use to `@cursorless/cursorless-vscode` in the `package.json`
checked into source control
- [x] Be sure to update meta-updater to check for that instead of name
`cursorless` (see fixme in its `index.ts`)

## Desiderata

- From #1289

### Correctly flagging errors

- [x] `D1`: It is a compile error to import external packages that are
not listed in `package.json`
   - [x] `D1.1`: `vscode` (for some reason this one behaves strangely)
   - [x] `D1.2`: Other packages
- [x] `D2`: It is **not** a compile error to import from local packages
that you depend on, using our preferred syntax (eg `@cursorless/foo`)
- [x] `D3`: It is a compile error to import from local packages that you
depend on, **not** using our preferred syntax (eg `../foo`)
- [x] `D4`: It is a compile error to import from local packages that you
do **not** depend on, either
- [x] `D4.1`: using our non-preferred syntax (eg `../packages/foo`), or
  - [x] `D4.2`: using preferred syntax (eg `@cursorless/foo`)
- [ ] `D5`: It is a compile error to import from anything other than
`index.ts` in another module
  - [x] `D5.1`: **no** `@cursorless/foo/bar`
  - [x] `D5.2`: **no** `../foo/bar`
  - [x] `a`: Even if you depend on the module
- [ ] `b`: Even if `bar` was re-exported in `foo/index.ts`. In that case
it should be required to import `@cursorless/foo`. Untested, but
probably works, and I ran out of steam 😅
- [x] `D6`: It is **not** a compile error to import external packages
listed in `package.json`
   - [x] `D6.1`: `vscode` (for some reason this one behaves strangely)
   - [x] `D6.2`: Other packages

### Auto-import
- [x] `D7`: Auto-import doesn't import from external packages not listed
in `package.json`
   - [x] `D7.1`: `vscode` (for some reason this one behaves strangely)
   - [x] `D7.2`: Other packages
- [x] `D8`: Auto-import imports from external packages listed in
`package.json`
   - [x] `D8.1`: `vscode` (for some reason this one behaves strangely)
   - [x] `D8.2`: Other packages
- [x] `D9`: Auto-import imports from local packages that you depend on,
using our preferred syntax (eg `@cursorless/foo`)
- [x] `D10`: Auto-import doesn't import from local packages that you
don't depend on
- [x] `D11`: Auto-import doesn't import from anything other than
`index.ts` in another package (eg **no** `@cursorless/foo/bar`)
   - [x] `D11.1`: When you depend on the package
   - [x] `D11.2`: When you don't depend on the package

### Other DX
- [x] `D12`: Find references across projects when no file from the
referencing project is open
- [x] `D13`: Rename across projects when no file from the referencing
project is open
- [x] `D14`: Jump to definition across projects
- [ ] `D15`: It is easy to move a file from one package to another
- [ ] `D16`: It is easy to create a new package
- [ ] `D17`: Breakpoints work in Cursorless extension
- [x] `D18`: Breakpoints work on Cursorless root website
- [ ] `D19`: Breakpoints work on Cursorless docs website

Author: pokey

.editorconfig

@@ -22,7 +22,7 @@ insert_final_newline = false
 # with a line that has trailing white space. Many of our recorded
 # tests use strings with trailing white space to represent the final
 # document contents. For example
-# packages/cursorless-vscode-e2e/suite/fixtures/recorded/languages/ruby/changeCondition.yml
+# packages/cursorless-vscode-e2e/src/suite/fixtures/recorded/languages/ruby/changeCondition.yml
 trim_trailing_whitespace = false
 
 [Makefile]

.eslintrc.json

@@ -10,11 +10,9 @@
     "ecmaVersion": 6,
     "sourceType": "module"
   },
-  "plugins": ["@typescript-eslint", "unused-imports"],
+  "plugins": ["@typescript-eslint", "unused-imports", "import"],
   "rules": {
-    // Note: you must disable the base rule as it can report incorrect errors
-    "no-restricted-imports": "off",
-    "@typescript-eslint/no-restricted-imports": "error",
+    "import/no-relative-packages": "error",
     "@typescript-eslint/consistent-type-assertions": [
       "error",
       {
@@ -51,5 +49,31 @@
     "no-throw-literal": "warn",
     "semi": "off"
   },
-  "ignorePatterns": ["**/vendor/**/*.ts", "**/vendor/**/*.js"]
+  "overrides": [
+    {
+      "files": ["*.tsx"],
+      "rules": {
+        "@typescript-eslint/naming-convention": [
+          "warn",
+          {
+            "selector": ["function"],
+            "format": ["PascalCase", "camelCase"]
+          }
+        ]
+      }
+    }
+  ],
+  "settings": {
+    "import/parsers": {
+      "@typescript-eslint/parser": [".ts", ".tsx"]
+    },
+    "import/resolver": {
+      "typescript": {
+        "alwaysTryTypes": true, // always try to resolve types under `<root>@types` directory even it doesn't contain any source code, like `@types/unist`
+
+        "project": ["tsconfig.json", "packages/*/tsconfig.json"]
+      }
+    }
+  },
+  "ignorePatterns": ["**/vendor/**/*.ts", "**/vendor/**/*.js", "**/out/**"]
 }

.github/workflows/deploy.yaml

@@ -15,24 +15,24 @@ jobs:
       - uses: actions/checkout@v3
         with:
           fetch-depth: 0
+      - uses: pnpm/action-setup@v2
+        with:
+          version: 7
       - uses: actions/setup-node@v3
         with:
           node-version: 16
-          cache: yarn
-      - run: yarn install --frozen-lockfile
-      - run: yarn run compile
-      - run: yarn run prepare-for-extension-publish
-      - name: Initialize cursorless-nx subdir
-        run: npm ci
-        working-directory: cursorless-nx
-      - name: Build cheatsheet html
-        run: npx nx build cheatsheet-local
-        working-directory: cursorless-nx
+          cache: pnpm
+      - run: pnpm install
+      - run: pnpm compile
+      - run: pnpm build
+        env:
+          CURSORLESS_DEPLOY: true
       - name: Publish to Open VSX Registry
-        uses: HaaLeo/publish-vscode-extension@v1
         id: publishToOpenVSX
+        uses: HaaLeo/publish-vscode-extension@v1
         with:
           pat: ${{ secrets.OPEN_VSX_TOKEN }}
+          packagePath: packages/cursorless-vscode/dist
       - name: Publish to Visual Studio Marketplace
         uses: HaaLeo/publish-vscode-extension@v1
         with:

.github/workflows/test-docs.yml

@@ -9,8 +9,11 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v3
+      - uses: pnpm/action-setup@v2
+        with:
+          version: 7
       - uses: actions/setup-node@v3
         with:
           node-version: 16
-          cache: yarn
-      - run: bash -x scripts/docs-build.sh
+          cache: pnpm
+      - run: bash -x scripts/build-and-assemble-website.sh

.github/workflows/test.yml

@@ -25,17 +25,40 @@ jobs:
       VSCODE_LOGS_DIR: ${{ github.workspace }}/artifacts/logs
     steps:
       - uses: actions/checkout@v3
+      - uses: pnpm/action-setup@v2
+        with:
+          version: 7
       - uses: actions/setup-node@v3
         with:
           node-version: 16
-          cache: yarn
+          cache: pnpm
       - run: mkdir -p "${{ env.VSCODE_CRASH_DIR }}" "${{ env.VSCODE_LOGS_DIR }}"
         shell: bash
-      - run: yarn install --frozen-lockfile
-      - run: xvfb-run -a yarn test
+      - run: pnpm --color install
+      - run: pnpm --color compile
+      - run: pnpm --color --filter '!cursorless-org' --filter '!cursorless-org-*' build
+      - run: xvfb-run -a pnpm --color test
         if: runner.os == 'Linux'
-      - run: yarn test
+      - run: pnpm --color test
         if: runner.os != 'Linux'
+      - name: Create vscode dist that can be installed locally
+        run: pnpm -F @cursorless/cursorless-vscode populate-dist --local-install
+        if: runner.os == 'Linux' && matrix.vscode_version == 'stable'
+      - name: Test create vsix
+        id: createVsix
+        uses: HaaLeo/publish-vscode-extension@v1
+        with:
+          pat: none
+          packagePath: packages/cursorless-vscode/dist
+          dryRun: true
+      - run: mv ${{ steps.createVsix.outputs.vsixPath }} cursorless-development.vsix
+        if: runner.os == 'Linux' && matrix.vscode_version == 'stable'
+      - name: Upload vsix
+        uses: actions/upload-artifact@v3
+        if: runner.os == 'Linux' && matrix.vscode_version == 'stable'
+        with:
+          name: vsix
+          path: cursorless-development.vsix
       - name: Archive logs
         uses: actions/upload-artifact@v3
         with:

.gitignore

@@ -8,3 +8,37 @@ node_modules
 
 # TypeDoc output
 docs/contributing/api/
+
+# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
+
+# dependencies
+.pnp
+.pnp.js
+
+# testing
+coverage
+
+# next.js
+.next/
+
+# production
+build
+
+# misc
+*.pem
+
+# debug
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+.pnpm-debug.log*
+
+# local env files
+.env*.local
+
+# vercel
+.vercel
+
+# typescript
+*.tsbuildinfo
+next-env.d.ts

.meta-updater/main.mjs

@@ -0,0 +1,3 @@
+import { updater } from "../packages/meta-updater/dist/index.cjs";
+
+export default updater;

.npmrc

@@ -0,0 +1,2 @@
+save-workspace-protocol=rolling
+save-prefix=""

.pre-commit-config.yaml

@@ -24,15 +24,16 @@ repos:
       - id: detect-private-key
       - id: end-of-file-fixer
         exclude_types: [svg]
+        exclude: patches/.*\.patch
       - id: fix-byte-order-marker
       - id: mixed-line-ending
       - id: trailing-whitespace
         # Trailing whitespace breaks yaml files if you use a multiline string
         # with a line that has trailing white space. Many of our recorded
         # tests use strings with trailing white space to represent the final
         # document contents. For example
-        # packages/cursorless-vscode-e2e/suite/fixtures/recorded/languages/ruby/changeCondition.yml
-        exclude: ^packages/cursorless-vscode-e2e/suite/fixtures/recorded/.*/[^/]*\.yml$
+        # packages/cursorless-vscode-e2e/src/suite/fixtures/recorded/languages/ruby/changeCondition.yml
+        exclude: ^packages/cursorless-vscode-e2e/src/suite/fixtures/recorded/.*/[^/]*\.yml$
   - repo: https://github.com/pre-commit/mirrors-prettier
     rev: "v2.7.1"
     hooks:

.prettierignore

@@ -1,7 +1,6 @@
 **/vendor
 
 # We use our own format for our recorded yaml tests to keep them compact
-/packages/cursorless-vscode-e2e/suite/fixtures/recorded/**/*.yml
+/packages/cursorless-vscode-e2e/src/suite/fixtures/recorded/**/*.yml
 
-# This was swizzled from a Docusaurus core class
-docs-site/src/theme/SearchBar/index.js
+pnpm-lock.yaml