Proposal: C# code rewrite/lowering at design/compile time

[msedi]

I'm sure someone has asked for this before but I couldn't find any reference to it.

Background
In our development team we are often in situations where the C# language support is not enough and some things can be expressed in a shorter form. An example is the newly introduced Range, where the syntax m..n is translated into a Range struct that can further be used in following methods. This is very handy, but is "hardcoded" in the roslyn process.

As explained, we sometimes need something similar, for example, years ago, we already created something similar to Range with a MATLAB-like syntax, since this is the origin where our team came from. We wanted to make it as simple as possible and recreated the MATLAB/matrix style access by creating our own Matrix class.

The problem was, MATLAB supported a very unique mini language to index the elements inside the matrix. Just as an example, 1:10, 1:1:10, 1:end, etc. The details don't matter, but the problem was that C# doesn't understand this mini language (of course). So we needed to find a way bring this into the language. Since there was no way, we decided to write it as a string and parse the string. So 1:10 was then "1:10" which of course works, but is not the nicest solution, since we only get runtime errors, but no compile time errors.

Proposal
So finally, I suggest to have a roslyn code rewrite/lowering/transform possibility that is able to hook into the parsing process in front of the real roslyn process. Having this, I would be able to transform the code, for example 1:10 can be created in a similar way, Range is rewritten.

The transformation process should be pluggable and always run in the design time compilation and the real compilation. Sure, some problems might occur and the transfomation process should not take too long so that the real roslyn process is not disturbed. But currently tools like PostSharp or Fody, etc. have to attach to the process afterwards and disassemble the code, which might be even worse.

[svick]

I believe writing an extensible parser, which is what would be required for this, would be a really hard task.

Have you considered continuing using the "1:10" syntax, but also writing a Roslyn analyzer, to get compile-time errors?

currently tools like PostSharp or Fody, etc. have to attach to the process afterwards and disassemble the code, which might be even worse.

One problem with transforming code before the compiler gets to is that it affects IntelliSense. In other words, that transformation would have to run on every key press. PostSharp and Fody can't affect IntelliSense, so they can only run when you actually build your code, which means demands on their performance are much lower.

[msedi]

@svick:

Have you considered continuing using the "1:10" syntax, but also writing a Roslyn analyzer, to get compile-time errors?

In the end yes, since this is the only currently available solution. The example above was only some example, but we have more that we need to cover. Another example is physical values which is supported in F#. Something like:

PhysicalValue v = "200mm";

which resolves to

PhysicalValue v = new PhysicalValue(200, KnownPhysicalUnit.mm);

A lot of our code base consists of this "workaround", but has several drawbacks, since we need a lot of parsing at runtime where in general the value can be disassembled into 200 and mm at compile time. This is causing a lot of performance bottlenecks. We have a lot of more examples, these are only two.

One problem with transforming code before the compiler gets to is that it affects IntelliSense. In other words, that transformation would have to run on every key press.

I understand that, and this is also my concern. On the otherhand if I think about it, it could also run on compile time, but I guess that also here IntelliSense would have a problem because the code does not work, since the syntax wouldn't be correct, right?

If I'm correct PostSharp and Fody, for example, only runs on correct and already compiled code, right?

Nevertheless I understand the concerns, if there would be a smarter solution that would also be possible at design time, that would be great.

[jmarolf]

I assume you want something like this?

https://github.com/dotnet/roslyn/blob/master/docs/features/generators.md

[msedi]

@jmarolf : Yes, that's somehow what I need. The only think I'm not sure, if generators can only create code (like T4 text templates) or are they also able to modify existing code?

[jmarolf]

@msedi Generators has not yet been implemented so it may do any number of things someday :). As its design currently stands, it does not allow you to modify code, only generate new code. The problem with being able to modify existing code is it means programmers can no longer reason about what C# code does.

For example: I may know the rules for overload resolution for classic C#, but your plugin could modify how this operation works. This would effectively make any code your plugin touches a dialect of C#. We don't feel that this would ultimately be the best direction for the C# language. We already have enough trouble writing and maintaining compatibility across one language, it seems unlikely we would succeed with potentially infinite dialects of C# :)

That isn't to say we would never do this, but just that we don't see a way to allow it without breaking everything.

[msedi]

@jmarolf

The problem with being able to modify existing code is it means programmers can no longer reason about what C# code does.

That's a fair point and I agree that this shouldn't happen.

Do you see any solution with the current possibilies we have with Roslyn? Let's say a code rewrite afterwards?

How was the range implemented in Roslyn? I guess here you made it a "hardcoded" part of Roslyn, right?

[CyrusNajmabadi]

that is able to hook into the parsing process in front of the real roslyn process.

This is almost certainly a non-starter. Parsing is a non-trivial part of roslyn, and has deep implications for things like being able to do incremental processing quickly and, most importantly, correctly.

Do you see any solution with the current possibilies we have with Roslyn? Let's say a code rewrite afterwards?

Yes, you could certainly write your own tool that uses roslyn that takes in normal c# code (perhaps annotated with string/comments/attributes), and spits out more C# code that is then compiled normally.

[msedi]

Related to: #1812