User-defined string literal formats

[TheUnlocked]

This proposal would allow for users to define their own string literal formats in a similar form to extension methods. For example:

// using Newtonsoft.Json.Linq

public static class Extension {
   [UserDefinedStringLiteral("json")] // Attribute name not set in stone
   public static JObject ToJson(this string str){  // byte[] would also be a valid type for this; characters like $ and @ are disallowed
      return JObject.parse(str);
   }
}

// Elsewhere:
JObject obj = json@"
{
  a: 'Hi',
  wow: [
    'A string',
    10
  ]
}";

A smart editor would be able to recognize these custom formats and provide syntax highlighting and code suggestions for other languages (of all kinds) within native C#. This could also allow for things like utf8 strings to be implemented in the standard library rather as a special language feature.

An alternative to double quotes could be backticks, if for some reason using double quotes in this context would be a breaking change.

[Joe4evr]

VS2019 has syntax highlighting for JSON strings (as well as Regex strings) in the box now, so this proposal seems superfluous.

[miloush]

@Joe4evr this proposal does not seem to be around syntax highlighting only.

For the parsing part though you can use using static:

using Newtonsoft.Json.Linq;

public static class Extension {
    public static JObject json(string str) => JObject.parse(str);
}

using static Extension; 
...
    JObject obj = json(@"
{
  a: 'Hi',
  wow: [
    'A string',
    10
  ]
}");

So you would save two characters for the price of making a non-constant expression look like a constant one.

[jho1965us]

A clever editor should also be able to do syntax higlighting from static if you add an attribute:

using Newtonsoft.Json.Linq;

public static class Extension {
    public static JObject json([StringSyntax("json")]string str) => JObject.parse(str);
}

To my knowledge ther is no such attribute for json yet but ReSharper has RegexPatternAttribute for regex

[TheUnlocked]

Edit: Another comment was posted while I was writing this, so this is a rough response to the two comments above the previous one.

However, it would also allow people to do things like this without needing to create a language change for every new string literal prefix:

public static class Extension {
   [UserDefinedStringLiteral("b")]
   public static byte[] Id(this byte[] bytes){
      return bytes;
   }
}

b"Hello, World!"
// new byte[]{(byte)'H', (byte)'e', (byte)'l', (byte)'l', (byte)'o', (byte)',', (byte)' ', (byte)'W', (byte)'o', (byte)'r', (byte)'l', (byte)'d', (byte)'!'}

And it also, as a practical example, allows someone to create some kind of React-like library without needing to fork and edit the language itself, and with as little of an extra syntax footprint as possible to avoid detracting from the readability of the XML:

[UserDefinedStringLiteral("html")]
public static ReactyElement ToReacty(this string xmlString){
   return Reacty.toReactyObject(xmlString); // I have to imagine it would be more complicated than this, but I'm not going to write a bunch of code surrounding a library that doesn't exist yet.
}

string text = "Hello, World!";
ReactyElement boldText = html$"<b>{text}</b>";

[CyrusNajmabadi]

A clever editor should also be able to do syntax higlighting from static if you add an attribute:

This work has already been done, without the need for any langauge changes here: dotnet/roslyn#25518

Examples of what it looks like are here: dotnet/roslyn#24110

[CyrusNajmabadi]

I don't understand how this example works at all:

public static class Extension {
   [UserDefinedStringLiteral("b")]
   public static byte[] Id(this byte[] bytes){
      return bytes;
   }
}

b"Hello, World!"

What actually is the compiler doing here? how is it making Hello, World! into bytes?

[TheUnlocked]

@CyrusNajmabadi It would just read the bytes between the quotes rather than even making a string in the first place.

Edit: In the case of string interpolation, the string would be converted into a byte array through UTF16.

[CyrusNajmabadi]

[UserDefinedStringLiteral("html")]
public static ReactyElement ToReacty(this string xmlString){
   return Reacty.toReactyObject(xmlString); // I have to imagine it would be more complicated than this, but I'm not going to write a bunch of code surrounding a library that doesn't exist yet.
}

string text = "Hello, World!";
ReactyElement boldText = html$"<b>{text}</b>";

Note that this sort of thing is very challenging on the IDE side. For example, what if 'text' inside {text} creates html code that change teh meaning of </b>?

This is why, for example we don't support regex-highlighting on an interpolated string. if you have $"(foo{bar}baz)", then bar can (and often does) completely change the meaning of both the parts of the regex that precede and follow it.

[CyrusNajmabadi]

@CyrusNajmabadi It would just read the bytes between the quotes rather than even making a string in the first place.

I don't know waht that means. What bytes between the quotes? The language doesn't have a concept of that. A string is not built of bytes. It's possible you could say that a prefixed-string could be passed to a string accepting method or char[] accepting method. But i can't see how it could be passed to a byte[] method. What byte[] encoding would even be used?

[TheUnlocked]

@CyrusNajmabadi The actual bytes stored in the source file, unless for some reason that would be untenable. char[] would also ideally be valid.

Edit: I concede that my "Hello, World!" example would only work if the source file were UTF8 or ASCII-encoded.

[Joe4evr]

@CyrusNajmabadi The actual bytes stored in the source file,

File.ReadAllBytes()?

[CyrusNajmabadi]

@CyrusNajmabadi The actual bytes stored in the source file, unless for some reason that would be untenable.

That feels like that would be untenable. It would make the C# language change meaning depend on how the file was saved. Right now the language is completely agnostic to that.

Heck, Roslyn doesn't even have the concept of files. You may be given "files" that have no backing on disk at all. It's unclear how that would even work here.

[Joe4evr]

char[] would also ideally be valid.

If you have a string, .ToCharArray() already exists since .NET 1.0.

Edit: I concede that my "Hello, World!" example would only work if the source file were UTF8 or ASCII-encoded.

Utf8String is available soon for creating and managing UTF-8 encoded strings.

[CyrusNajmabadi]

@Joe4evr Right. I could envision it being any type that had an implicit-ish conversion from string. So char[] would be fine. Utf8String as well as that would addres teh encoding concern.

How this would be defined at a language level is flexible to me. Perhaps it might literally be if there is an implicit conversion. Or, it might be if it has a 1-arg constructor that is of type string/char[]/ReadOnlySpan<char>. Maybe both can be supported.

ReadOnlySpan<char> makes a lot of sense to me as well as presumably that would be trivial to get from from teh data-segment of the PE, without requiring an unnecessary intermediate string-allocation.