extend the markdown syntax for graphql (#3490)

* extend the markdown syntax for graphql
* more grammar support, add ruby
* apply suggestions from vscode discussion

Author: acao

.changeset/clean-spies-sip.md

@@ -0,0 +1,10 @@
+---
+'vscode-graphql': patch
+'vscode-graphql-syntax': patch
+---
+
+- add ruby syntax support
+- add graphql syntax support in markdown codeblocks for js, ts, jsx, tsx, svelte, vue, ruby, rescript, reason, ocaml, php and python
+- make textmate injectors more performant and specific, eliminate redundant config
+
+Big thanks to [@RedCMD](https://github.com/RedCMD) and [@aeschli](https://github.com/aeschli) for your help!

.eslintignore

@@ -5,3 +5,4 @@ working-group/
 packages/codemirror-graphql/src/__tests__/schema-kitchen-sink.graphql
 CHANGELOG.md
 **/CHANGELOG.md
+packages/vscode-graphql-syntax/tests/__fixtures__/

.eslintrc.js

@@ -30,6 +30,7 @@ module.exports = {
     'changesets/**/*.md',
     '**/CHANGELOG.md',
     'functions/*',
+    'packages/vscode-graphql-syntax/tests/__fixtures__/*',
   ],
   overrides: [
     {

custom-words.txt

@@ -94,6 +94,7 @@ typedoc
 vite
 vitest
 vitejs
+vsix
 wonka
 urql
 tsup
@@ -228,3 +229,4 @@ unparsable
 randomthing
 codicon
 edcore
+tokenizes

packages/vscode-graphql-syntax/README.md

@@ -4,180 +4,52 @@ Adds full GraphQL syntax highlighting and language support such as bracket
 matching.
 
 - Supports `.graphql`/`.gql`/`.graphqls` highlighting
-- [Javascript, Typescript & JSX/TSX](#ts) & Vue & Svelte
-- ReasonML/ReScript (`%graphql()` )
-- Python
-- PHP
-- [Markdown](#markdown)
-- [Scala](#scala)
+- Javascript, Typescript & JSX/TSX (examples: [test.js](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/tests/__fixtures__/test.js) & [test.ts](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/tests/__fixtures__/test.ts))
+- Vue (examples: [test-sfc-comp.vue](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/tests/__fixtures__/test-sfc-comp.vue) & [test-sfc.vue](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/tests/__fixtures__/test-sfc.vue))
+- Svelte (example: [test.svelte](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/tests/__fixtures__/test.svelte))
+- ReasonML/ReScript (`%graphql()` ) (example: [test.re](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/tests/__fixtures__/test.re))
+- Python (example: [test.py](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/tests/__fixtures__/test.py))
+- PHP (example: [test.php](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/tests/__fixtures__/test.php))
+- Markdown (examples: [test.md](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/tests/__fixtures__/test.md) & [test-py.md](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/tests/__fixtures__/test-py.md))
+- Scala (example: [test.scala](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/tests/__fixtures__/test.scala))
+- ruby (example: [test.rb](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/tests/__fixtures__/test.rb))
 
 You'll want to install this if you do not use `graphql-config`, or want to use
 the highlighting with other extensions than `vscode-graphql`
 
-## Embedded Language Usage
-
-<span id="ts">
-### Javascript & Typescript
-
-The following delimiters are accepted for syntax highlighting. If you are using
-any of these patterns and they do not work, please open an issue!
-
-#### Template Literal Expressions
-
-```ts
-const query = gql`
-  {
-    id
-  }
-`;
-```
-
-you can use these template tag literal expressions anywhere you like of course
-
-```ts
-useFancyGraphQLClient(
-  graphql`
-    {
-      id
-    }
-  `,
-  {
-    networkStrategy: '🚀',
-  },
-);
-```
+## Contributing
 
-```ts
-const query = gql.experimental`{ id }`;
-```
-
-and in typescript, template tag expressions with type arguments
-
-```ts
-const query = gql<MyType>`
-  {
-    id
-  }
-`;
-```
-
-#### Function Expressions
-
-as well as normal function expressions with template literals
-
-```ts
-gql('{ id }');
-```
-
-```ts
-graphql(`
-  {
-    id
-  }
-`);
-```
-
-there is a bug with function expressions with type arguments like these that we
-need to fix:
-
-```ts
-gql<MyType>('{ id }');
-```
-
-Note, inline `""` and `''` string literals could also be delimited if needed,
-but we currently only delimit graphql template strings for obvious reasons
-
-#### Comment-Delimited patterns
-
-```ts
-const query = /* GraphQL */ `
-  {
-    id
-  }
-`;
-```
-
-```ts
-const query = `#graphql
- { id }
-`;
-```
-
-For full autocompletion, validation and other features, you can install
-`GraphQL.vscode-graphql`, which depends on this extension
-
-<span id="markdown">
-
-### Markdown
-
-#### backtick code blocks
-
-````markdown
-# Hello Jan
-
-```graphql
-query MyQuery {}
-```
-````
-
-#### embedded graphql in js & ts codeblocks
-
-simple js/ts`gql` & `graphql` template tag expression support inside any
-backtick codeblocks.
-
-````markdown
-# Hello Jan
-
-```js
-const myQuery = gql`
-  {
-    its {
-      query
-      time
-    }
-  }
-`;
-```
-````
-
-#### Scala
-
-Using a `graphql`, `gql` or `schema` string interpolator:
+Feel free to open a PR to fix, enhance any language support, or even add new
+languages 😍
 
-```scala
-val query = graphql"""
-  { id }
-"""
-val query2 = gql"""
-  { id }
-"""
-val query3 = schema"""
-  { id }
-"""
-```
+see:
 
-Using a comment-delimited pattern:
+- [the grammars](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/grammars/)
+- [the applicable vscode docs](https://code.visualstudio.com/api/language-extensions/syntax-highlight-guide)
 
-```scala
-val query = """#graphql
- { id }
-"""
-```
+### Contributor Guide: Improving a Language
 
-## Other languages
-
-We actually support other languages than this! just need to extend this readme
-even further! 🥵
+Whether fixing a bug or adding a new delimiter for your language, here are a few tips for you:
 
-## Contributing
+1. TDD approach: add your bug case or new delimiter example to the relevant file in `tests/__fixtures__`
+1. run `yarn test -u` in the syntax extension workspace, and observe whether vscode-textmate tokenizes your example properly
+1. fix/update/add the relevant pattern, and repeat the above to see if the tokenization changes. you should see `meta.embedded.block.graphql`
+1. to test manually, run `yarn vsce:package` in the workspace and right click to install the bundled vsix extension, and open the fixture file
 
-Feel free to open a PR to fix, enhance any language support, or even add new
-languages 😍
+<span id="adding-a-lang"></span>
 
-see:
+### Contributor Guide: Adding a Language
 
-- [the grammars](grammars/)
-- [the applicable vscode docs](https://code.visualstudio.com/api/language-extensions/syntax-highlight-guide)
+1.  add a file to [grammars](https://github.com/graphql/graphiql/blob/main/packages/vscode-graphql-syntax/grammars/) following our other examples.
+1.  be sure to add it to `package.json` contributions as well, in the `grammars` section. the `text.html.markdown` is for applying to markdown codeblocks
+1.  use a scope ala `source.{lang}` from a vscode-provided syntax grammar, or a popular, official contributed grammar. To find the name of the scope for any token's highlighting, use `Developer: Inspect Editor Tokens & Scopes` from the vscode command palette.
+1.  name it `inline.graphql.{lang}` for consistency
+1.  add a test file `tests/__fixture__` to document example usage, and a test spec to `__tests__` to assert the snapshot, pointing to the source you created
+1.  run `yarn test -u` in the workspace to add the snapshot
+1.  use the snapshots to ensure your capture groups are working and serializing the graphql as expected
+1.  in the test fixture, document all working cases and non working cases with Todo comments for common usage in your language. be sure to think of cases such as string interpolation and generics for typed languages.
+1.  add it to the list above in the readme, with links to your test fixtures as usage documentation
+1.  to manually test it in vscode itself, run `yarn vsce:package` in the syntax extension workspace and right click and install the bundled vsix file, then view the test fixture
 
 ## Usage Note
 

packages/vscode-graphql-syntax/grammars/graphql.js.json

@@ -1,24 +1,10 @@
 {
-  "fileTypes": [
-    "js",
-    "jsx",
-    "mjs",
-    "cjs",
-    "es6",
-    "es",
-    "esm",
-    "ts",
-    "tsx",
-    "vue",
-    "svelte",
-    "cts",
-    "mts"
-  ],
-  "injectionSelector": "L:source -string -comment",
+  "scopeName": "inline.graphql",
+  "injectionSelector": "L:(meta.embedded.block.javascript | meta.embedded.block.typescript | source.js | source.ts | source.tsx | source.vue | source.svelte) -source.graphql -inline.graphql -string -comment",
   "patterns": [
     {
       "contentName": "meta.embedded.block.graphql",
-      "begin": "\\s*+(?:(?:(Relay)\\??\\.)(QL)|(gql|graphql|graphql\\.experimental)|(/\\* GraphQL \\*/))\\s*\\(?\\s*(`)",
+      "begin": "\\s*+(?:(?:(Relay)\\??\\.)(QL)|(gql|graphql|graphql\\.experimental)|(/\\* GraphQL \\*/))\\s*\\(?\\s*(`|'|\")",
       "beginCaptures": {
         "1": {
           "name": "variable.other.class.js"
@@ -36,7 +22,7 @@
           "name": "punctuation.definition.string.template.begin.js"
         }
       },
-      "end": "`",
+      "end": "(`|'|\")",
       "endCaptures": {
         "0": {
           "name": "punctuation.definition.string.template.end.js"
@@ -46,7 +32,7 @@
     },
     {
       "contentName": "meta.embedded.block.graphql",
-      "begin": "\\s*+(?:(?:(Relay)\\??\\.)(QL)|(gql|graphql|graphql\\.experimental))\\s*\\(?\\s*(?:<.*>)(`)",
+      "begin": "\\s*+(?:(?:(Relay)\\??\\.)(QL)|(gql|graphql|graphql\\.experimental))\\s*\\(?\\s*(?:<.*>)(`|'|\")",
       "beginCaptures": {
         "1": {
           "name": "variable.other.class.js"
@@ -61,7 +47,7 @@
           "name": "punctuation.definition.string.template.begin.js"
         }
       },
-      "end": "`",
+      "end": "(`|'|\")",
       "endCaptures": {
         "0": {
           "name": "punctuation.definition.string.template.end.js"
@@ -72,7 +58,7 @@
     {
       "name": "taggedTemplates",
       "contentName": "meta.embedded.block.graphql",
-      "begin": "(`|')(#graphql)",
+      "begin": "(`|'|\")(#graphql)",
       "beginCaptures": {
         "1": {
           "name": "punctuation.definition.string.template.begin.js"
@@ -81,14 +67,13 @@
           "name": "comment.line.graphql.js"
         }
       },
-      "end": "(`|')",
+      "end": "(`|'|\")",
       "endCaptures": {
         "0": {
           "name": "punctuation.definition.string.template.end.js"
         }
       },
       "patterns": [{ "include": "source.graphql" }]
     }
-  ],
-  "scopeName": "inline.graphql"
+  ]
 }

packages/vscode-graphql-syntax/grammars/graphql.markdown.codeblock.json

@@ -1,16 +1,32 @@
 {
-  "fileTypes": [],
-  "scopeName": "markdown.graphql.codeblock",
-  "injectionSelector": "L:markup.fenced_code.block.markdown",
+  "scopeName": "inline.graphql.markdown.codeblock",
+  "injectionSelector": "L:text.html.markdown -meta.embedded.block",
   "patterns": [
     {
-      "contentName": "meta.embedded.block.graphql",
-      "begin": "(gql|graphql|GraphQL)(\\s+[^`~]*)?$",
-      "end": "(^|\\G)(?=\\s*[`~]{3,}\\s*$)",
+      "begin": "(^|\\G)(\\s*)(\\`{3,}|~{3,})\\s*(?i:(graphql|gql|GraphQL)(\\s+[^`~]*)?$)",
+      "name": "markup.fenced_code.block.markdown",
+      "end": "(^|\\G)(\\2|\\s{0,3})(\\3)\\s*$",
+      "beginCaptures": {
+        "3": {
+          "name": "punctuation.definition.markdown"
+        },
+        "4": {
+          "name": "fenced_code.block.language.markdown"
+        },
+        "5": {
+          "name": "fenced_code.block.language.attributes.markdown"
+        }
+      },
+      "endCaptures": {
+        "3": {
+          "name": "punctuation.definition.markdown"
+        }
+      },
       "patterns": [
         {
           "begin": "(^|\\G)(\\s*)(.*)",
           "while": "(^|\\G)(?!\\s*([`~]{3,})\\s*$)",
+          "contentName": "meta.embedded.block.graphql",
           "patterns": [
             {
               "include": "source.graphql"

packages/vscode-graphql-syntax/grammars/graphql.php.json

@@ -1,6 +1,6 @@
 {
-  "fileTypes": ["php"],
-  "injectionSelector": "L:source -string -comment",
+  "scopeName": "inline.graphql.php",
+  "injectionSelector": "L:(meta.embedded.block.php | source.php -string -comment)",
   "patterns": [
     {
       "contentName": "meta.embedded.block.graphql",
@@ -126,6 +126,5 @@
         }
       ]
     }
-  ],
-  "scopeName": "inline.graphql.php"
+  ]
 }