|
| 1 | +  |
| 2 | +## VSCode Textmate grammar test |
| 3 | + |
| 4 | + |
| 5 | +Provides a way to test textmate grammars against a vscode engine using user-friendly plaintext files. |
| 6 | + |
| 7 | +Demo: |
| 8 | + |
| 9 | +[](https://asciinema.org/a/QoGS5fPsxDOHl1T43zzmFxJAU) |
| 10 | + |
| 11 | +Inspired by [Sublime Text syntax tests](https://www.sublimetext.com/docs/3/syntax.html#testing) |
| 12 | + |
| 13 | + |
| 14 | +### Installation |
| 15 | + |
| 16 | +As a project dependency: |
| 17 | + |
| 18 | +```bash |
| 19 | +npm i --save-dev vscode-tmgrammar-test |
| 20 | +``` |
| 21 | + |
| 22 | +Or as a standalone command line tool: |
| 23 | + |
| 24 | +```bash |
| 25 | +npm i -g vscode-tmgrammar-test |
| 26 | +vscode-tmgrammar-test --help |
| 27 | +``` |
| 28 | + |
| 29 | + |
| 30 | +### Unit tests |
| 31 | + |
| 32 | +```scala |
| 33 | +// SYNTAX TEST "source.scala" "sample testcase" |
| 34 | + |
| 35 | +// line can start with a <comment token> and not have a valid assertion |
| 36 | + |
| 37 | +class Stack[A] { |
| 38 | +// <----- keyword.declaration.scala |
| 39 | +// ^ - keyword.declaration.scala entity.name.class.declaration |
| 40 | +// ^^^^^ entity.name.class.declaration |
| 41 | +// ^ source.scala meta.bracket.scala |
| 42 | +// ^ entity.name.class |
| 43 | +// ^ meta.bracket.scala |
| 44 | +// ^ punctuation.section.block.begin.scala |
| 45 | +``` |
| 46 | + |
| 47 | +To write a unit test: |
| 48 | + |
| 49 | +* include a header line: |
| 50 | + |
| 51 | +``` |
| 52 | +<comment token> SYNTAX TEST "<language scope>" "optional description" |
| 53 | +``` |
| 54 | + |
| 55 | +* Require tokens to have specific scope by using `^` : |
| 56 | + |
| 57 | +```scala |
| 58 | +private var elements: List[A] = Nil |
| 59 | +// ^^^^^^^^ variable.other.declaration.scala |
| 60 | +``` |
| 61 | + |
| 62 | +* Get into those pesky first few characters by using `<-`: |
| 63 | + |
| 64 | +```scala |
| 65 | +var x = 3 |
| 66 | +// <--- keyword.declaration.volatile.scala |
| 67 | +// the length of '-' determine how many characters are matched from the start of the line |
| 68 | +x=5 |
| 69 | +// <~- keyword.operator.comparison.scala |
| 70 | +// you specify offset from start by using '~' character, just in case |
| 71 | +``` |
| 72 | + |
| 73 | +* To ensure that tokens **don't** have undesired scopes put `-` symbol before them: |
| 74 | +```scala |
| 75 | + / ensure comment start with two double slashes |
| 76 | + ^ - comment.line.double slash.scala |
| 77 | + |
| 78 | + / or you can combine both positive and negative scopes |
| 79 | + ^ source.scala - comment.line.double slash.scala |
| 80 | +``` |
| 81 | + |
| 82 | +Lines which start with a `<comment token>` and assertion symbol are ignored by the textmate grammar. |
| 83 | + |
| 84 | + |
| 85 | +Note, that scope comparison takes into account relative scope's position. |
| 86 | +So, if required scopes are `'scope1 scope2'`, the test will report an error if a grammar returns them as `'scope2 scope1'`. |
| 87 | + |
| 88 | +To run a unit test: |
| 89 | +```bash |
| 90 | +vscode-tmgrammar-test 'tests/unit/**/*.test.scala' |
| 91 | +``` |
| 92 | + |
| 93 | + |
| 94 | +### Snapshot tests |
| 95 | +Snapshot tests are like `functional tests` but you don't have to write outputs explicitly. |
| 96 | +All you have to do is to provide a source files, scopes of which you want to test. Then on |
| 97 | +the first run `vscode-tmgrammar-snap` will generate a set of `.snap` files which are an |
| 98 | +instant snapshot of lines of the source files together with corresponding scopes. |
| 99 | + |
| 100 | +Then if you change the grammar and run the test again, the program will output the changes between |
| 101 | +the `.snap` file and the real output. |
| 102 | +If you satisfied with the changes you can `commit` them by running |
| 103 | +```bash |
| 104 | +vscode-tmgrammar-snap --updateSnapshot .... |
| 105 | +``` |
| 106 | +this will overwrite the existing `.snap` files with a new ones. |
| 107 | +After this you should commit them alongside with the source code test cases. |
| 108 | + |
| 109 | +You can read more about them at [snapshot testing](https://jestjs.io/docs/en/snapshot-testing) |
| 110 | + |
| 111 | +To run snapshot test: |
| 112 | +```bash |
| 113 | +vscode-tmgrammar-snap 'tests/snap/**/*.scala' |
| 114 | +``` |
| 115 | + |
| 116 | +### Language configuration via package.json |
| 117 | + |
| 118 | +The configuration follows the format of vscode: |
| 119 | + |
| 120 | +```json |
| 121 | +{ |
| 122 | + "contributes": { |
| 123 | + "languages": [ |
| 124 | + { |
| 125 | + "id": "scala", |
| 126 | + "extensions": [ |
| 127 | + ".scala", |
| 128 | + ".sbt", |
| 129 | + ".sc" |
| 130 | + ] |
| 131 | + } |
| 132 | + ], |
| 133 | + "grammars": [ |
| 134 | + { |
| 135 | + "language": "scala", |
| 136 | + "scopeName": "source.scala", |
| 137 | + "path": "./syntaxes/Scala.tmLanguage.json" |
| 138 | + } |
| 139 | + ] |
| 140 | + } |
| 141 | +} |
| 142 | +``` |
| 143 | +The idea is that for the average language extension all necessary information for tests are already included in the `package.json`. |
| 144 | +It is optional, though. If the configuration is missing it is necessary to specify grammars and scopeName of testcases via command line options. |
| 145 | + |
| 146 | +Right now only regular grammars and *Injection Grammars* via `injectTo` directive are supported. |
| 147 | + |
| 148 | + |
| 149 | +### Command Line Options |
| 150 | + |
| 151 | +Unit tests: |
| 152 | +``` |
| 153 | +Usage: vscode-tmgrammar-test [options] <testcases...> |
| 154 | + |
| 155 | +Run Textmate grammar test cases using vscode-textmate |
| 156 | + |
| 157 | +Arguments: |
| 158 | + testcases A glob pattern(s) which specifies testcases to run, e.g. "./tests/**/test*.dhall". Quotes are important! |
| 159 | + |
| 160 | +Options: |
| 161 | + -g, --grammar <grammar> Path to a grammar file. Multiple options supported. 'scopeName' is taken from the grammar (default: []) |
| 162 | + --config <configuration.json> Path to the language configuration, package.json by default |
| 163 | + -c, --compact Display output in the compact format, which is easier to use with VSCode problem matchers |
| 164 | + --xunit-report <report.xml> Path to directory where test reports in the XUnit format will |
| 165 | + be emitted in addition to console output |
| 166 | + --xunit-format <generic|gitlab> Format of XML reports generated when --xunit-report is used. |
| 167 | + `gitlab` format is suitable for viewing the results in GitLab |
| 168 | + -V, --version output the version number |
| 169 | + -h, --help display help for command |
| 170 | +``` |
| 171 | + |
| 172 | +Snapshot tests: |
| 173 | +``` |
| 174 | +Usage: vscode-tmgrammar-snap [options] <testcases...> |
| 175 | + |
| 176 | +Run VSCode textmate grammar snapshot tests |
| 177 | + |
| 178 | +Arguments: |
| 179 | + testcases A glob pattern(s) which specifies testcases to run, e.g. "./tests/**/test*.dhall". Quotes are important! |
| 180 | + |
| 181 | +Options: |
| 182 | + -u, --updateSnapshot overwrite all snap files with new changes |
| 183 | + --config <configuration.json> Path to the language configuration, package.json by default |
| 184 | + --printNotModified include not modified scopes in the output (default: false) |
| 185 | + --expandDiff produce each diff on two lines prefixed with "++" and "--" (default: false) |
| 186 | + -g, --grammar <grammar> Path to a grammar file. Multiple options supported. 'scopeName' is taken from the grammar (default: []) |
| 187 | + -s, --scope <scope> Explicitly specify scope of testcases, e.g. source.dhall |
| 188 | + -V, --version output the version number |
| 189 | + -h, --help display help for command |
| 190 | +``` |
| 191 | + |
| 192 | +### Setup VSCode unit test task |
| 193 | + |
| 194 | +You can setup a vscode unit test task for convenience: |
| 195 | + |
| 196 | +```json |
| 197 | + { |
| 198 | + "label": "Run tests", |
| 199 | + "type": "shell", |
| 200 | + "command": "vscode-tmgrammar-test -c -g testcase/dhall.tmLanguage.json '**/*.dhall'", |
| 201 | + "group": "test", |
| 202 | + "presentation": { |
| 203 | + "reveal": "always", |
| 204 | + "panel":"new", |
| 205 | + }, |
| 206 | + "problemMatcher": { |
| 207 | + "owner": "vscode-tmgrammar-test", |
| 208 | + "fileLocation": [ |
| 209 | + "relative", |
| 210 | + "${workspaceFolder}", |
| 211 | + ], |
| 212 | + "pattern": [ |
| 213 | + { |
| 214 | + "regexp": "^(ERROR)\\s([^:]+):(\\d+):(\\d+):(\\d+)\\s(.*)$", |
| 215 | + "severity": 1, |
| 216 | + "file": 2, |
| 217 | + "line": 3, |
| 218 | + "column": 4, |
| 219 | + "endColumn": 5, |
| 220 | + "message": 6, |
| 221 | + }, |
| 222 | + ], |
| 223 | + }, |
| 224 | + }, |
| 225 | +``` |
| 226 | + |
| 227 | +Notice the `-c` option that will output messages in a handy format for the problemMatcher. |
| 228 | + |
| 229 | +Result: |
| 230 | + |
| 231 | + |
0 commit comments