Fix more broken tests and add documentation for repo tools

Author: leeyi45

.vscode/settings.json

@@ -6,8 +6,15 @@
       "url": "./lib/buildtools/src/build/modules/manifest.schema.json"
     }, 
     {
-      "fileMatch": ["tsconfig.*.json"],
-      "url": "http://json.schemastore.org/tsconfig"
+      "fileMatch": ["tsconfig.json", "tsconfig.*.json"],
+      "schema": {
+        "allOf": [{ "$ref": "http://json.schemastore.org/tsconfig" }],
+        "properties": {
+          "typedocOptions": {
+            "allOf": [{ "$ref": "https://typedoc.org/schema.json" }]
+          }
+        }
+      }
     }
   ]
 }

docs/.vitepress/config.ts

@@ -46,7 +46,8 @@ const sidebarConfigs: Record<string, VitePressSidebarOptions> = {
   buildtools: {},
   modules: {},
   lib: {},
-  '/lib/modules-lib': {}
+  'lib/modules-lib': {},
+  repotools: {}
 };
 
 const sideBarOptions = Object.entries(sidebarConfigs).map(([startPath, options]): VitePressSidebarOptions => ({

docs/src/index.md

@@ -5,15 +5,6 @@ layout: home
 hero:
   name: Modules Developer Documentation
   tagline: Developer documentation for the Source Academy modules repository
-  actions:
-    - theme: brand
-      text: Get Started
-      link: /modules/1_getting-started
-    - theme: brand
-      text: Repo Overview
-    - theme: alt
-      text: Developer Instructions
-      link: /api-examples
 
 features:
   - title: Your first bundle or tab
@@ -30,4 +21,5 @@ features:
 
   - title: Repository Tools
     details: Details for the tools used to aid in developing SA Modules
+    link: /repotools
 ---

docs/src/repotools/devserver/devserver.md

@@ -0,0 +1,17 @@
+# Modules Development Server
+
+This server relies on [`Vite`](https://vitejs.dev) to create a server that automatically reloads when it detects file system changes. This allows Source Academy developers to make changes to their tabs without having to use the frontend, and have it render code changes live.
+
+The components here were copied from the frontend's playground and then simplified.
+
+## Compiled Tabs vs Hot Reload
+Normally, the Source Academy frontend uses the bundled version of the tabs. However, the development server relies on the raw Typescript files by default so as to enable hot reloading. If you wish to test the compiled versions, use the settings button to toggle using compiled tabs.
+
+## `mockModuleContext.ts`
+For compiled tabs, `js-slang/context` is not an import that should be required. However, in hot-reload mode, because some tabs rely on their corresponding bundles, `js-slang/context` is an import that needs to be provided.
+
+The `vite` config aliases `js-slang/context` to the `mockModuleContext.ts` file, which can then be used to simulate the evaluation context.
+
+## Vitest Testing
+The `vitest` framework integrates with `playwright` and allows us to actually simulate user actions on the browser page. This is used to test the functionality of the development server.
+

docs/src/repotools/docserver/docserver.md

@@ -0,0 +1,60 @@
+# Developer Documentation Server
+
+Originally most of the developer documentation was contained within the Github repository's wiki. However, as the documentation became more comprehensive in scope and complex in design,
+it was necessary to migrate to a more advanced and configurable solution.
+
+This documentation server is powered by [Vitepress](https://vitepress.dev), which takes Markdown files and renders them into a static site.
+
+## File Structure
+All pages for the server are contained under `src`.
+
+[`vitepress-sidebar`](https://vitepress-sidebar.cdget.com) is being used to generate the sidebar for the server. This means that entries in the sidebar appear sorted according to the file/folder names from which they originate. This is why
+some folder have contents that are labelled `1-something`, `2-something` etc. Each item's value is taken from file's frontmatter, or if not present, the file's first header.
+
+Folders produce item groups, the name of which can be customized using an `index.md` file within that folder.
+
+```txt
+bundles
+├── index.md
+├── 1-getting-started
+│   ├── 1-overview.md
+│   ├── 2-start.md
+│   ├── 3.cheat.md
+│   ├── 4-faq.md
+│   └── index.md
+└── 2-bundle
+    ├── 1-overview
+    │   └── 1-overview.md
+    ├── 2-creating
+    │   └── 2-creating.md
+    ├── 3-editing.md
+    ├── 4-documentation
+    │   └── 4-documentation.md
+    ├── 5-type_map.md
+    ├── 6-compiling.md
+    └── index.md
+```
+
+The above file structure produces the menu below:
+![image](./menu.png)
+
+Each item takes its title value from either the frontmatter (if available) or the first heading of each page.
+
+`index.md` files are only used to title the menus, they are not intended for navigation, though are navigable to if the user enters the address manually, so it is still recommended to keep some basic
+documentation in those pages or create rewrites so that users are automatically redirected away from those pages. Here are the contents of `2-bundle/index.md`:
+
+<<< ../../modules/2-bundle/index.md
+
+Instead, the links in the sidebar link to the page (if it is a markdown file), or to a child within the folder that has the same name as that folder.
+
+The specific configuration for this behaviour can be found in the Vitepress config file, and the documentation for those options can be found on the Vitepress website:
+
+<<< ../../../.vitepress/config.ts {33-43}
+
+## Documentation for `modules-lib`
+The documentation for `@sourceacademy/modules-lib` is automatically generated by Typedoc. The configuration options for this generation are found in that folder. The documentation for `@sourceacademy/modules-lib` should be built before this server is built.
+
+::: details help pls
+Supposedly Vitepress can embed React components, which would be nice for us to have working demonstrations of our React components. Unfortunately, I have never
+been able to get that to work. You can start [here](https://github.com/vuejs/vitepress/discussions/2183) if you want to help figure this out.
+:::

docs/src/repotools/docserver/index.md

@@ -0,0 +1,5 @@
+# Repository Development Tools
+
+- [Development Server](./devserver/)
+- [Docs Server](./docserver/docserver)
+- [Yarn](./yarn)

docs/src/repotools/docserver/menu.png

@@ -0,0 +1,10 @@
+# VSCode Integration
+This page is dedicated to explaining all the VSCode integrations available from this repository.
+
+Since most developers here are assumed to be using Visual Studio Code, this repository automatically provides some extra tooling that integrates with VSCode, all found within the `.vscode` folder.
+
+## JSON Schemas
+By default, VSCode doesn't apply any kind of validation to JSON files that don't have explicit schemas. Using the `settings.json` file, we can apply JSON schemas to specific JSON files.
+
+For this repository, this has been configured for `tsconfig.*.json` files to extend both the original `tsconfig` schema, but also to include the Typedoc schema. Bundle manifest files
+should also automatically receive validation and IntelliSense autocompletion.

docs/src/repotools/index.md

@@ -0,0 +1,40 @@
+# Yarn
+
+This page is dedicated to explaing how Yarn has been configured to work in this repository.
+
+## Workspaces
+Yarn, this repository's package manager, supports a feature called [workspaces](https://yarnpkg.com/features/workspaces#global-scripts), which are a way for packages to refer to one another within the same repository.
+
+This has several benefits:
+- **Reduced install size.** Yarn workspaces support focused installs, which means that a single team working on a single bundle or tab won't have to install the entire repository's worth of packages that won't be relevant to them
+- **Dependency Management.** If tabs (or other bundles) rely on a specific bundle, Yarn's dependency resolution automatically determines what needs to be rebuilt and in what order.
+- **Automatic Worktree Detection.** If so desired, Yarn automatically detects which bundles/tabs have changed with respect to the main git branch and won't rebuild those that haven't.
+- **More Configurability:** Each bundle/tab can maintain its own set of dependencies now, separate from other bundles. This allows us to have patches local to a specific bundle (like the `@jscad/modeling` patch for `csg`). Also each bundle/tab can customize its own `tsconfig` etc..
+
+At the root repository, this is defined using the `workspaces` field in `package.json`.
+
+## Yarn Constraints
+Yarn also supports a feature known as [constraints](https://yarnpkg.com/features/workspaces#constraints) for workspaces. As of this moment, the feature can be used to ensure that all workspaces within a repository have fields in their `package.json`s set to specific values. We use this feature to ensure that:
+
+### 1. All workspaces have `"type": "module"`.
+As part of ensuring consistency, everything in this repository has been designed to use ESM first and to move away from legacy reliance on CommonJS modules.
+
+### 2. Dependencies, if shared, have the correct versions.
+Tabs, for example, should all be using the same versions of `react` and `react-dom`: the one in use by the frontend. If a dependency is specified in the root `package.json`, then the constraints file
+requires that all child workspaces use the version of that dependency.
+
+For example, the root package specifies `react@^18.3.1` as a depdendency, so all workspaces that require React must also use that version spec.
+
+This validation is not carried out across child workspaces, however. Two different bundles could use two different versions of the same package. This should not cause an issue **unless** the bundles are somehow dependendent on each other.
+
+## Parallel Execution of Scripts
+Using the various options of the `yarn workspaces foreach` command, you can execute multiple tasks in parallel, which is how many of the commands in the root repository have been set up.
+
+The `-t` option runs the scripts in topological order, which means you don't have to manually figure out which packages have to be built before others; Yarn figures it out for you.
+
+::: details ESLint out of memory?
+For some reason when I migrated this repository over to Yarn workspaces, I kept finding that linting all the bundles at once in parallel
+would cause NodeJS to exceed its default memory limit of 4906MB. I guess if you wanted to run linting for all bundles in parallel you could
+just run NodeJS with `--max-old-space-size`, but I just found that limiting the number of jobs running in parallel using `-j` to 5 was 
+sufficient to get around this issue.
+:::