feat: Add support for custom code templates (fix #987)

[travkin79]

This PR fixes issue #987 and adds

• a preferences dialog for editing custom code templates for languages available through registered TextMate grammars
• first language-independent code templates
• support for adding new custom code templates
• code proposals based on available custom code templates
• syntax highlighting in the code templates preview based an the settings for the registered language grammars

[angelozerr]

@travkin79 to be consistent with other existing class, I suggest that you use TM* instead of Tm4e*

[sebthom]

Please add some screenshots to this PR. Thanks!

[angelozerr]

@travkin79 I noticed that you have changed the Tm4e with Tm, please use TM like other classes (TMPlugin, etc)

[angelozerr]

Please add some screenshots to this PR. Thanks!

I think it should be very cool to update the docs with screenshot.

[travkin79]

Please add some screenshots to this PR. Thanks!

Hi @sebthom,
Here are some screenshots that illustrate the new features implemented so far (it's still work in progress).

New preferences dialog (very similar to the "Templates" preferences in JDT and CDT, due to reusing code from the Eclipse platform):

Users can add custom code templates for registered languages by pressing the "New" button in the preferences dialog. Here is how the dialog for new code templates looks like (again very similar to the corresponding dialog in JDT and CDT):

The context (context types) list is automatically created by checking registered grammars / languages. In the "New Template" dialog, you'll get a list of all available languages / scopes:

Pushing the button "Insert variable" in the "New Template" dialog shows available code template variables (currently only the language-independent ones):

The preview source viewer in the preferences dialog uses the syntax highlighting associated with the registered grammar corresponding to the template's context type, here Java:

In an editor with syntax highlighting activated through TM4E, users get code proposals based on existing code templates:

[angelozerr]

@travkin79 it looks super great! Congrats!

But IMHO I think you should also support vscode snippet. TM4E already supports:

• textmate
• language-configuration.json

that you can share between vscode and Eclipse IDE.

VSCode snippets is the last part which misses.

For instance instead of using cursor or selection, you should use snippet syntax

If you can support that, you could copy/paste for instance https://github.com/redhat-developer/vscode-java/blob/main/snippets/server.json and use it directly.

And it would be very nice to provide an extension point to contribute to TM4E with snippet like we have for textmate and language configuration.

If you can manage that, you could share snippets from all existing vscode extension and it will be very easy for developer to manages snippets for vscode and Eclipse IDE both.

Please note that IntelliJ supports also vscode snippets.

[travkin79]

Hello @angelozerr,

I see the benefits of VSCode syntax support. But I think, it would also demand a lot of work to add VSCode snippet support. For that reason I prefer to begin with a smaller feature set, without VSCode snippet syntax support for now. We can add that feature later with a separate PR (see my issue comment on that).

In my implementation, I tried to reuse as many code template support code from the Eclipse platform as possible and though, it was already quite some work to get that far. Now, I'd like to clarify if I'm on the right way with my concept of using

1. using custom code templates for registered TM grammars / languages
2. using common code templates for comments, ordinary comments and documentation comments (so that such code templates can be used for all languages, no need to specify copies for each language)

In addition, I'd like to finish my implementation and clarify / improve a few things in code. Most of these things can be found in TODO comments in my changed files.

• Does my algorithm in TMTemplateCompletionProcessor#retrieveTemplateContextType(TMToken) correctly detect comments and documentation comments in tokenized source code?
• TMUIPlugin got a new inner class ContextTypeRegistryWrapper for wrapping a deprecated ContextTypeRegistry. The null-safety check (the compiler) complains about incompatible return types in the overriden method contextTypes(). I didn't find a solution, yet. Any tip is appreciated. Are other null-safety annotations and conversions in my PR correctly used?
• In TMUIPlugin#getTemplateContextRegistry() I'm loading registered grammars and registering a new context type for each grammar. Since the list of grammars becomes quite long and I didn't find language names for each of them, are there any grammars that I can / should safely skip in this step? Maybe the grammars embedded in other languages like various languages from code snippets in Markdown or LaTeX?
• In TMTemplateCompletionProcessor#computeCompletionProposals(ITextViewer, int) I have to call the overridden method via Display.syncExec(). Otherwise I get an invalid thread access exception. Am I doing something wrong or might there be a bug in the parent class?
• I'm trying to apply the same syntax highlighting in the code template preview and editing source viewers. Is there an elegant way of adding special syntax highlighting for the code template variables (e.g. ${user}) without having to create a modified version of each registered TM grammar?
• Extensions
  • In case we want to support language-specific code template variables in our code templates, how can we add additional variables to the pre-defined context types (see TMLanguageTemplateContextType)? Maybe that could also be a feature for a separate PR.
  • In case we're going to use some kind of extensions, we might need to override context types by loading more specific ones from extensions or load language-specific variables (or variable resolvers) from extensions.

[angelozerr]

Please add some screenshots to this PR. Thanks!

Hi @sebthom,
Here are some screenshots that illustrate the new features implemented so far (it's still work in progress).

New preferences dialog (very similar to the "Templates" preferences in JDT and CDT, due to reusing code from the Eclipse platform):

Users can add custom code templates for registered languages by pressing the "New" button in the preferences dialog. Here is how the dialog for new code templates looks like (again very similar to the corresponding dialog in JDT and CDT):

The context (context types) list is automatically created by checking registered grammars / languages. In the "New Template" dialog, you'll get a list of all available languages / scopes:

Pushing the button "Insert variable" in the "New Template" dialog shows available code template variables (currently only the language-independent ones):

The preview source viewer in the preferences dialog uses the syntax highlighting associated with the registered grammar corresponding to the template's context type, here Java:

In an editor with syntax highlighting activated through TM4E, users get code proposals based on existing code templates:

Those screenshot and description should be included in the doc that @sebthom has written