Stable hats (#1225)

- Fixes #432 

## Checklist

- [ ] I have added
[tests](https://www.cursorless.org/docs/contributing/test-case-recorder/)
- [ ] Add test for case where enabled hat styles change, both adding or
removing an enabled hat style
- [ ] I have updated the
[docs](https://github.com/cursorless-dev/cursorless/tree/main/docs) and
[cheatsheet](https://github.com/cursorless-dev/cursorless/tree/main/cursorless-talon/src/cheatsheet)
- [ ] I have not broken the cheatsheet

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Author: pokey

.vscode/launch.json

@@ -127,6 +127,18 @@
         "!**/node_modules/**"
       ]
     },
+    {
+      "name": "Docusaurus Build (Debug)",
+      "type": "node",
+      "request": "launch",
+      "cwd": "${workspaceFolder}/docs-site",
+      "runtimeExecutable": "npm",
+      "runtimeArgs": ["run", "build"],
+      "resolveSourceMapLocations": [
+        "${workspaceFolder}/**",
+        "!**/node_modules/**"
+      ]
+    },
     {
       "name": "cursorless.org client-side",
       "type": "chrome",

cursorless-talon/src/cheatsheet/get_list.py

@@ -16,7 +16,6 @@ def get_list(name, type, descriptions=None):
 
 
 def get_lists(names: list[str], type: str, descriptions=None):
-
     return [item for name in names for item in get_list(name, type, descriptions)]
 
 

cursorless-talon/src/vendor/jstyleson.py

@@ -28,7 +28,6 @@ def dispose(json_str):
     former_index = None
 
     for index, char in enumerate(json_str):
-
         if escaped:  # We have just met a '\'
             escaped = False
             continue

docs-site/docusaurus.config.js

@@ -38,6 +38,17 @@ function remarkPluginFixLinksToRepositoryArtifacts() {
         return;
       }
 
+      // Hack; see https://github.com/cursorless-dev/cursorless/issues/1243
+      let match = link.match(/^(\.\.\/)+docs\//);
+      if (match != null) {
+        link = "/docs/" + link.substring(match[0].length);
+        if (link.endsWith(".md")) {
+          link = link.substring(0, link.length - 3);
+        }
+        node.url = link;
+        return;
+      }
+
       let repoRoot = path.resolve(__dirname, "..");
       let artifact = path.resolve(file.dirname, link);
       let artifactRelative = path.relative(repoRoot, artifact);
@@ -65,7 +76,7 @@ const config = {
   baseUrl: "/docs/",
   favicon: "/docs/favicon.ico",
   onBrokenLinks: "throw",
-  onBrokenMarkdownLinks: "throw",
+  onBrokenMarkdownLinks: "warn",
   trailingSlash: true,
 
   plugins: [

docs/user/hatAssignment.md

@@ -0,0 +1,38 @@
+# Hat assignment
+
+Every time you move your cursor, edit a document, or scroll, Cursorless assigns hats to the tokens in each visible editor. When selecting hats, Cursorless attempts to give "good" hats to tokens near your cursor, while at the same time keeping hats from moving around too much.
+
+Hats are considered "good" that require fewer syllables to say, so for example, a gray dot is the best kind of hat, because you just say the letter it sits on, whereas a colored shape (eg `"blue fox"`) is the worst kind of hat, because you need to say both color and shape. Each hat style has a penalty associated with it that indicates how many syllables it requires to say:
+
+| Hat Style     | Example spoken form | Penalty |
+| ------------- | ------------------- | ------- |
+| gray dot      | `"air"`             | 0       |
+| colored dot   | `"blue air"`        | 1       |
+| gray shape    | `"fox air"`         | 1       |
+| colored shape | `"blue fox air"`    | 2       |
+
+## The algorithm
+
+Every time you move your cursor, edit the document, or scroll, Cursorless does a pass through all visible tokens, assigning them hats. It does so by walking through the tokens one by one in order of their "rank". A token's rank is determined by how close it is to your cursor: tokens near your cursor are considered higher rank and get to pick their hats first. For each token, Cursorless proceeds as follows:
+
+### 1. Discard any hats that are considered unacceptable
+
+When a token is picking its hat, it will have a pool of available hats based on
+
+- Which hats are enabled,
+- What characters make up the token,
+- Which hats have already been taken by a higher ranked hat.
+
+One of these candidate hats might be the hat the token was already wearing before this pass started (if it had one), and some of them will be hats that lower ranked tokens were wearing before the pass started.
+
+The token will start by figuring out the penalty of the best candidate hat (eg is there a gray dot? Are they all colored shapes? etc). Based on that penalty, it will consider only the candidate hats whose penalty is sufficiently close to the best hat. The definition of "sufficiently close" is determined by the user setting `cursorless.experimental.hatStability`:
+
+| Setting value          | Behaviour                                                                                                                                                                                                                                                           |
+| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `greedy`               | Only consider hats whose penalty is just as good as the best candidate hat. This setting will result in less stable hats, but ensure that tokens near the cursor always get the best hats.                                                                          |
+| `balanced` _(default)_ | If the best candidate hat has a penalty below 2 (eg it is a gray shape or colored dot), then discard all hats whose penalty is 2 or greater. This setting results in fairly stable hats, while ensuring that all tokens near the cursor have a penalty less than 2. |
+| `stable`               | Don't discard any hats. Always keep existing hat if it wasn't stolen, and don't steal hats unless there are no free hats left to this token. Note that if you have no shapes enabled, then this setting is equivalent to `balanced`.                                |
+
+### 2. Select a hat from the remaining hat candidates
+
+Once the "unacceptable" hats are discarded, then if the token's existing hat is amongst the remaining acceptable hats, it will keep it. If not, it will pick a hat that won't require it to steal from a lower ranked token if any such hats are left. If it can't keep its own hat or avoid stealing a hat, it will steal a hat from the lowest ranked token.

package.json

@@ -687,6 +687,21 @@
               "url"
             ]
           }
+        },
+        "cursorless.experimental.hatStability": {
+          "markdownDescription": "As you scroll, edit, and move your cursor, this setting determines how much Cursorless will move hats around to ensure that the best hats are near the cursor.  See https://www.cursorless.org/docs/user/hatAssignment/",
+          "type": "string",
+          "default": "balanced",
+          "enum": [
+            "greedy",
+            "balanced",
+            "stable"
+          ],
+          "markdownEnumDescriptions": [
+            "Always put the best hats near the cursor",
+            "Only move hats to avoid having colored shapes near the cursor (eg `\"blue fox\"`); otherwise leave hats where they are",
+            "Only move hats to ensure that the tokens near the cursor have a hat at all, no matter how bad the hat is.  Note that if you have no shapes enabled, then this setting is the same as `balanced`"
+          ]
         }
       }
     },

src/apps/cursorless-vscode-e2e/suite/backwardCompatibility.test.ts

@@ -17,7 +17,7 @@ async function runTest() {
 
   editor.selections = [new vscode.Selection(0, 0, 0, 0)];
 
-  await graph.hatTokenMap.addDecorations();
+  await graph.hatTokenMap.allocateHats();
 
   await vscode.commands.executeCommand(
     CURSORLESS_COMMAND_ID,

src/apps/cursorless-vscode-e2e/suite/breakpoints.test.ts

@@ -24,7 +24,7 @@ suite("breakpoints", async function () {
 async function breakpointHarpAdd() {
   const { graph } = (await getCursorlessApi()).testHelpers!;
   await openNewEditor("  hello");
-  await graph.hatTokenMap.addDecorations();
+  await graph.hatTokenMap.allocateHats();
 
   await runCursorlessCommand({
     version: 1,
@@ -51,7 +51,7 @@ async function breakpointHarpAdd() {
 async function breakpointTokenHarpAdd() {
   const { graph } = (await getCursorlessApi()).testHelpers!;
   await openNewEditor("  hello");
-  await graph.hatTokenMap.addDecorations();
+  await graph.hatTokenMap.allocateHats();
 
   await runCursorlessCommand({
     version: 1,
@@ -79,7 +79,7 @@ async function breakpointTokenHarpAdd() {
 async function breakpointHarpRemove() {
   const { graph } = (await getCursorlessApi()).testHelpers!;
   const editor = await openNewEditor("  hello");
-  await graph.hatTokenMap.addDecorations();
+  await graph.hatTokenMap.allocateHats();
 
   vscode.debug.addBreakpoints([
     new vscode.SourceBreakpoint(
@@ -110,7 +110,7 @@ async function breakpointHarpRemove() {
 async function breakpointTokenHarpRemove() {
   const { graph } = (await getCursorlessApi()).testHelpers!;
   const editor = await openNewEditor("  hello");
-  await graph.hatTokenMap.addDecorations();
+  await graph.hatTokenMap.allocateHats();
 
   vscode.debug.addBreakpoints([
     new vscode.SourceBreakpoint(

src/apps/cursorless-vscode-e2e/suite/containingTokenTwice.test.ts

@@ -16,7 +16,7 @@ suite("Take token twice", async function () {
 async function runTest() {
   const { graph } = (await getCursorlessApi()).testHelpers!;
   const editor = await openNewEditor("a)");
-  await graph.hatTokenMap.addDecorations();
+  await graph.hatTokenMap.allocateHats();
 
   for (let i = 0; i < 2; ++i) {
     editor.selection = new vscode.Selection(0, 1, 0, 1);

src/apps/cursorless-vscode-e2e/suite/crossCellsSetSelection.test.ts

@@ -23,7 +23,7 @@ async function runTest() {
   // editor
   await sleepWithBackoff(1000);
 
-  await graph.hatTokenMap.addDecorations();
+  await graph.hatTokenMap.allocateHats();
 
   await runCursorlessCommand({
     version: 1,