Some changes to work better with Visual Studio 2019 plus TODO highlighting

[zledas]

Hello,

I tried to use it with Visual Studio 2019, but there were some problems. The main one, is that VS2019 somehow reads to the end of the line AND consumes line ending if \s* is used. So next line is not highlighted properly. So I changed \s* to [^\S\n\r]* in many places where it was used to read till the end of line.

Also I changed variable.other.XXX (and similar) to entity.name.variable.other.XXX so it gets coloured.

What is more, I added TODO highlighting, but I am not sure if I inserted it in all the needed places.

I am attaching .patch file with my changes, if that would help: changes.txt

P. S. I edited xml directly, so don't have YAML for these changes...

[zledas]

Today I saw, that TODO highlighting is not good enough – it does not catch some cases...

[ephread]

Thanks for the changes @zledas!

I'm sorry it took so long to respond, I completely missed this issue.

I won't be able to use your patch as-is, since, as you pointed out, the source of truth is the YAML. But, if changing \s* to [^\S\n\r]* is all you did, then I can probably update the YAML next time I'm touching the grammar. I'll also add entity.name.variable.other.XXX to all patterns using variable.other.XXX so that they can be both matched.

I'll have a look at the TODO changes, but a lot of corner cases can't be matched with the Textmate Grammar. So not handling these cases might be perfectly okay.

[zledas]

Thanks for responding.

Yes, \s* and variable.other.XXX changes are the main ones!

And it would be perfect if at least main cases of TODO highlighting would be available as well :)

[ephread]

I've resumed working on the grammar and made significant progress. There were a lot of other issues, beyond the line-ending thing.

Would you be able to try the latest version and tell me if you still encounter the same issues? (If you do, it would be nice to have a sample of the breaking code, so that I can write a text case against it!)

Thanks a lot for your help!

[zledas]

Tested new version – it works! :)

Didn't find any breakages, but our formatting is quite strict, so not much variation.

On the other hand, variables and function calls are still not coloured. From what I see, variable.other.XXX (and similar) still do not have prefix (entity.name.variable.other.XXX), so if it would be possible to add that (as an additional match?) – it would be perfect.

[ephread]

Tested new version – it works! :)
Didn't find any breakages, but our formatting is quite strict, so not much variation.

Awesome!

if it would be possible to add that (as an additional match?) – it would be perfect.

Of course, I missed it during the refactoring. I'll add the scopes right away.

[ephread]

0.2.2 is out with the scope extensions. Let me know if it works or if I missed something else!

[zledas]

Tested it – extended variable scopes work great.

BUT I found another problem. I think this did not work before as well. Attaching screenshot and repro case (repro.ink.txt).

AND while writing this report I noticed that + [{EXAMINE}] is not highlighted in the right screenshot. Not sure if this is one of the limitations of text mate parsing.

[ephread]

Tested it – extended variable scopes work great.

I'm happy to hear that!

Thanks for providing an example, I'm unable to reproduce the problem with the latest grammar, sadly (attached here, in case I messed up the release: Ink.tmLanguage.zip). I tested with Textmate 2, SublimeText and VS Code, see below.

AND while writing this report I noticed that + [{EXAMINE}] is not highlighted in the right screenshot. Not sure if this is one of the limitations of text mate parsing.

You're right, it should be coloured. But I'm assuming it's related to the other issue.

Which editor are you using?

[zledas]

Thanks for checking it out. I am using Visual Studio 2019 (Community edition).

What I just noticed is that { and } are not highlighted if they are right after true or false keywords. This can be seen in my previously attached screenshot as well. So I think it is line break problem – line break is consumed and so next line is not treated as new line...

[zledas]

Ahhh, dammit... Turns out that when I first checked 0.2.0 version, Visual Studio still used my older, modified .tmLanguage file, so I didn't see that it was broken :( Sorry for that.

So line break problem still exists. Basically all \s* that read till the end of line work differently in VS compared to other editors for some reason. VS reads to the end of the line AND consumes line ending if \s* is used. So next line is not highlighted properly. I can solve it if I replace \s* with [^\S\n\r]* (this explicitly tells not to consume line break symbols).

So I have to change such (and similar) places:
<string>\s*(false|true)\s*</string>
to
<string>\s*(false|true)[^\S\n\r]*</string>

[ephread]

It's really strange that Visual Studio doesn't behave like other editors. I replaced most
\s occurrences by [^\S\n\r] and it doesn't seem to have any negative impacts on other editors. Sorry I didn't make this change sooner, I wanted to make sure there wasn't any other underlying issues.

Here's the latest version, can you give it a try and let me know how it goes?

Ink.tmLanguage.zip

[zledas]

Yes, it was very strange to me as well.
Tested this new one – looks like it works good. Thanks for fixing all these issues!

On the other hand, I found one not handled TODO case. I am copying it here, but if it would be better to have separate issue,I can move it to separate one:

===events===

=exit
    {
    - (variable != 4):
        TODO: ...
        M: Hi.
    }
    -> the_end

[ephread]

Tested this new one – looks like it works good. Thanks for fixing all these issues!

All good!

On the other hand, I found one not handled TODO case. I am copying it here, but if it would be better to have separate issue,I can move it to separate one:

It's fine, thanks for providing an example, I'll have a look.

[ephread]

0.2.3 should fix the issue. 🙂