String Hints

[jeremycook]

String hints for improving coding experiences. This would not modify the nature of strings in anyway, it would standardize syntactic sugar for improving coding experiences. And just to be extra clear, this has nothing to do with raw string literals #89.

I propose that a string hint can be lowercase letters that precede the opening double quote of a string like string hello = hint"World";. String hints like "sql", "json", "html", "xml", "markdown", "liquid", etc. would just be conventions that compilers, linters, IDEs could watch for to help a coder out. You will still be working with either a String or FormattableString like normal.

SQL Examples:

As an example: an IDE could lookup "ConnectionStrings" in appsettings.json to provide suggestions for table names and columns when it hits the sql string hint.

string query = sql"SELECT * FROM Foo WHERE Bar = 'Baz'";

string bar = "Baz";
FormattableString query = $sql"SELECT * FROM Foo WHERE Bar = {bar}";

JSON Example:

This being hinted as a JSON string, the $schema property can be used to provide code coloring, autocomplete, and linting capabilities.

string propertyName = "title";
string json = $@json"{
    ""$schema"": ""http://json-schema.org/draft-07/schema"",
    ""type"": ""object"",
    ""properties"": {
        ""{propertyName}"": {
            ""type"": ""string""
        }
    },
    ""required"": [
        ""{propertyName}""
    ]
}";

[CyrusNajmabadi]

@jinujoseph public requests for this sort of facility. Note: the json work has already been done, but was rejected by the roslyn team.

[Pretasoc]

Maybe this could also be solved by some magic comment, that would trigger the IDE to thread the string as an other language island. Then there wouldn't be a need for a language change, which probably make this feature easier happen.

[jeremycook]

@CyrusNajmabadi I read a lot of issues including the JSON one you might be referring to before posting this. I feel and hope that this is different from other requests that may seem similar. Note that this is merely syntactic sugar, and is more generally applicable than JSON. What IDEs or VS Code Extensions (as an example that pops into mind) do with the hints is up to them.

[CyrusNajmabadi]

Note that this is merely syntactic sugar, and is more generally applicable than JSON.

Yes, that was how my feature worked as well.

What IDEs or VS Code Extensions (as an example that pops into mind) do with the hints is up to them.

Yes, unfortunately, the extension mechanism i built to support this in a generalized first-class fashion was rejected. Note: you can see the support i built with how VS lights up regex patterns in the last couple of versions of hte product. I expanded on htat to make it a generalized solution (and added support for json to show that), but it was rejected.

[CyrusNajmabadi]

String hints like "sql", "json", "html", "xml", "markdown", "liquid", etc. would just be conventions that compilers, linters, IDEs could watch for to help a coder out.

Unfortunately, this usually is not enough. There needs to be richer informaiton to pass along to the final tooling to decide what sort of support is needed. For example, within the 'json' realm there are actually many dialects of json supported. Same with regex. I enabled support for this in my system by having users write something like:

// lang=regex,opt1,opt,opt3

[jeremycook]

@CyrusNajmabadi Ah, I didn't realize your intention was broader than JSON. I'm sorry to hear it was rejected. Thank you for the work on Regex. Seeing Regex as it is now is part of what inspired me to post this issue. I've wanted and hoped for something like string hints for a long time.

[CyrusNajmabadi]

Maybe this could also be solved by some magic comment, that would trigger the IDE to thread the string as an other language island. Then there wouldn't be a need for a language change, which probably make this feature easier happen.

That already exists. See:

However, having this expand out to other embedded languages was rejected by the team. I've pinged @jinujoseph to let her know there is interest here from other users to help possibly motivate rethinking things here!

[jeremycook]

There needs to be richer informaiton to pass along to the final tooling

I'm sure you have more experience than me in this realm and I recognize that the Visual Studio, Roslyn, .NET folks all work very hard to provide tight tooling experiences. Recognizing that this feature may not feel very tight I would say that my motivation is that some help from the IDE is better than no help, and that how tight this feature feels is very much dependent on the tool that uses it and not this feature. That's why I suggest these minimalistic string hints and leave the rest up to the tooling to take advantage of them.

Take the sql string hint as an example. Tooling could determine the type of database by starting with the connection strings in app.config, appsettings.json, environment variables, etc., and then try various SQL clients until it finds the right one. From their it can further interrogate and provide progressively enhanced tooling depending on how much can be known within the current context. What if a project has connections strings for multiple databases? The tooling may not know which database I am interested in until I start typing a table name or a column name, but the more info I provide the more helpful it will be. My take is that seeing a list of table names that start with "Vis" is going to be more helpful to me than seeing nothing at all. The autocomplete itself could indicate to the developer which connection string was used to get a table or column name.

For a json string hint, if a "$schema" property exists within that string and the IDE fetches it and gets back a JSON schema file then it can be harnessed to provide autocomplete and linting services. Code coloring can be standard for JSON with nothing but the json hint.

My thought on the configuration side is that is a tooling issue and the tool authors would tell developers how to get the most out of their tools or how to handle special cases. Some standards may coalesce but I think that is a tooling issue that can be put off.

Of value, string hints provide more information in situations like the following, and are even helpful to developers that don't have tooling for a particular string hint. In fact, the existence of a hint and lack of an IDE plugin could result in the IDE prompting the user to go find an IDE plugin (like what VS Code does for file extensions now).

liquid"<ul>{% for person in people %}<li>{{person.firstName}} {{person.lastName}}</li>{% endfor %}</ul>"

vs.

handlebar"{{#list people}}{{firstName}} {{lastName}}{{/list}}"

My hope here is for the language to provide the simplest marker possible for tools to begin building some really useful features upon.

[CyrusNajmabadi]

My hope here is for the language to provide the simplest marker possible for tools to begin building some really useful features upon.

I guess my point is this: the language already provides this.

However, there appears to be little motivatoin beyond that for people to actually provide these rich experiences. This is not to say that people don't think it woudl be great. Just that there's no connection between that and actually allocating resources to make it happen.

I've tried to provide this support with my own PR efforts. But they hit a wall in terms of the team being willing to accept them.

[jeremycook]

the language already provides this.

I can imagine how that might be the case but could you fill me in on how the language already provides this?

But they hit a wall in terms of the team being willing to accept them.

Thank you for reading. I Googled and read more issues and PRs you worked on that relate to this. I didn't find one that directly mapped to what I've asked for here but it seems like we are interested in very similar outcomes. Thanks for your efforts in this area. If there are any issues or PRs, open or closed, that you think closely correspond to this request I'd like to see them.

[qrli]

I saw Resharper automatically identifies e.g. regex string, and provides syntax highlight and helps. That at least proves that IDE/analyzers are able to figure out special string type without hint from developers.

Not the best, but that seems good enough. So I understand there is not enough motivation to add this to the language.

[Pretasoc]

@qrli Thats not completely true. Resharper does so, as either an [RegexPatternAttribute] or an external annotation files, tells resharper a function expects a string to be an Regex pattern. So there is still a hint, although it is not really visible.

The same applies to format strings in ReSharper.

[jeremycook]

@qrli @Pretasoc I think the handlebars and liquid examples I provided in my last comment demonstrate where hinting of some sort is necessary and could prove really useful for lighting up features. The alternative is an IDE or extension hooking into special comments like @CyrusNajmabadi implemented for regex strings. What I propose and what @CyrusNajmabadi has done are functionally equivalent. In fact, what I propose could be syntactic sugar that becomes a code comment in the AST for IDEs and tools to interpret. Like:

string pattern = regex"[a-z]+";

becomes

// lang=regex
string pattern = "[a-z]+";

And my liquid and handlebars examples could look like the following to IDEs and tools.

// lang=liquid
string liquid = "<ul>{% for person in people %}<li>{{person.firstName}} {{person.lastName}}</li>{% endfor %}</ul>";

// lang=handlebars
string hb = "{{#list people}}{{firstName}} {{lastName}}{{/list}}";

[CyrusNajmabadi]

And my liquid and handlebars examples could look like the following to IDEs and tools.

We also support:

string pattern = /*lang=regex*/"[a-z]+";

Which is, effectively, the same sort of 'prefix' to inform tooling.

[jeremycook]

Thanks @CyrusNajmabadi I didn't realize that. That is, effectively, the same sort of 'prefix' to inform tooling. I guess the best reason to add the string hint feature would be to provide a standard that IDEs, etc. can depend on.

It seems like a chicken or egg problem. I imagine that for a feature like this to gain traction it would need to be driven by IDE and tool makers, JetBrains, VS Code extension builders, and so on. If there is no interest from those camps then I can understand why similar proposals have been rejected.

[ghord]

I think this could be neatly solved by two features:

1. Allow attributes on any expression that are only trivia but can be used by analyzers. These attributes could support parameters like normal attributes.
2. Allow analyzers to change color of tokens

This way we can write:

string myJson = [Json] @"{ ""item"" : ""value"" }"; 

or

string regex = [Regex("ig")] "myRegex";

or even

public const string Conn = "..";

var sql = [Sql(Conn)] "select * from Table";

And delegate all the highlighting/validation work to custom analyzers.

[Thaina]

Interesting. If we could attribute anything anywhere it might also possible for dynamic to attribute it typed autocomplete too

[Dynamic<MyObject>]
dynamic a = ParseJson(json);

[jeremycook]

@ghord Finding a middle ground between what you suggest, @CyrusNajmabadi lang comments, and what I originally had in mind might look like this:

string myJson = [Lang("json")] @"{ ""item"" : ""value"" }"; 
string regex = [Lang("regex")] "myRegex";
var sql = [Lang("sql")] "select * from Table";
var liquid = [Lang("liquid")] "{{ title }}: {{ description }}";
FormattableString html = [Lang("html")] $"<h1>{model.Title}</h1>";

The point would be to provide a general purpose Lang attribute so as not to require language specific attributes. Certain language specific attributes could be provided by Microsoft or 3rd-party libraries to improve the experience, but would not be required.

What you suggested could be subclasses of the LangAttribute or implementations of the ILang interface. That might look something like this:

namespace System
{
	public interface ILang
	{
		string Language { get; }
	}

	public class LangAttribute : Attribute, ILang
	{
		public LangAttribute(string language)
		{
			Language = language;
		}
		public string Language { get; }
	}
}

namespace System.Text.RegularExpressions
{
	public class RegexAttribute : LangAttribute
	{
		public RegexAttribute(RegexOptions regexOptions)
			: base("regex")
		{
			RegexOptions = regexOptions;
		}
		public RegexOptions RegexOptions { get; }
	}
}

[CyrusNajmabadi]

Why is [Lang("regex")] needed when there is already /*lang=regex*/? In terms of characters it is:

[Lang("regex")]
/*lang=regex*/

The comment is actually shorter. The comment also has an easy way to just extend with things like

/*lang=regex,IgnoreCase*/

This already works today and has already shipped successfully. It's not clear to me why we need a language feature to replace what already works and which doesn't seem to add anything new that we can't already do. Can you clarify the benefit here?

[CyrusNajmabadi]

I think this could be neatly solved by two features:

Allow attributes on any expression that are only trivia but can be used by analyzers. These attributes could support parameters like normal attributes.

I believe this is already in: #2478

However, i don't see why this is needed since we already have such a trivia (and we're already using it today in the IDE).

Allow analyzers to change color of tokens

Sure. This is not a language request though. This would be better requested in the dotnet/roslyn repo as it's a request about either analyzers or IDE tooling. I've pushed strongly for my work to be made public so other languages can participate. However, tehre has been no traction as of yet.

[jeremycook]

@CyrusNajmabadi Regex is was just an example. From a comment on that issue, @eyalsk made some comments that seem relate to this issue #2478 (comment)