Dealing with limited breaking changes in C#

[MadsTorgersen]

Dealing with limited breaking changes in C#

Oftentimes new C# language features have edge cases that could conflict with existing features. In the past we’ve avoided breaking existing code even at the expense of making the feature design more complex than otherwise needed. Sometimes the burden is mostly one of implementation complexity borne by the team, but other times aspects of the feature end up being genuinely confusing to developers as a result.

One example is discards. Because _ used to be just an ordinary identifier, the language tries to protect existing usages as such. As a result, _ sometimes represents a discard and sometimes an ordinary identifier, depending on whether or where it is declared. The rules for this have little inherent logic, as they are based purely on "what was there first". This is needlessly confusing to anyone just trying to learn and use discards, and limits the places where they can be usefully employed in code.

What if we changed our stance a little here? C#'s commitment to backwards compatibility is very valuable to its users, but so is simplicity and clarity. Could we adjust our approach so that the existing language can "move over a bit" and give new features room for more consistent and elegant designs?

For this to be a good trade-off, it would require a high-confidence, low-friction mechanism for making existing code resilient to such new features. And it would require a lot of thoughtfulness and restraint on the language design team to only introduce breaking changes with limited scope, strong justification and excellent fixes.

In the following we'll try to outline such a mechanism. We're curious about everyone’s thoughts, both on the specifics of the mechanism, and on whether it represents a balanced approach to considering limited, thoughtful breaking changes in C#.

Example: field access

Field access in auto-properties is a feature we would very much like to add to C#. The feature allows a variable name field to be used in property accessors to refer to the generated backing field:

public string Name { get => field; set => field = value.Trim(); }

However, the above code could already exist, and could for instance be referring to an existing variable called field in an enclosing scope. If that code now changes meaning in e.g. C# 12, this is a breaking change - either it leads to an error or to different behavior of the existing code.

In order to avoid such a break, the current proposal for field introduces the new field keyword conditionally - only if there isn't already another field in scope at the point of the accessor. This "works" in terms of not breaking existing code, but leads to more confusing behavior for the new feature itself. Introducing a field field somewhere outside the property can nullify the language feature and inadvertently change the meaning of occurrences of the field identifier that were intended to refer to the backing field. This behavior is arbitrarily inconsistent with how value works today in property accessors, simply because value was in the language from the start. Finally, the implementation and maintenance costs in the compiler and tooling are likely to be higher because the feature is more complex and the binding behavior less localized.

It would be better if we could just make the lookup of the new field identifier "normal", and consistent with every other identifier in every other place in the language. This is the kind of breaking change that would make the new feature design meaningfully simpler, and leave the language with fewer “gotchas” in the future. Instead of shying away from these kinds of breaking changes entirely, could we give you a mechanism that reliably protects your existing code in the relatively rare cases where the new feature would break it? Could we help you easily find and fix any lines of code that would be vulnerable to the break, before you even adopt the new language version?

Outline of a solution

As a matter of terminology, a breaking change is a change to a newer version of the language that will cause some previously working user code to break. There are two kinds of breaks that can occur that the language concerns itself with:

• New error: Code that used to work now causes an error.
• Silent break: Code that used to work one way now works differently.

The first is actually the most benign. At least you'll know that something is up, giving you a starting point for chasing down the problem. Still, it is not ideal: The error may not easily indicate that the cause is a breaking change in the language, and it may not even occur at the relevant point in the code.

The second is more insidious. Because the behavior change may be subtle it not be caught in testing, and could lead to bugs down the line where they are more costly. At the same time, it is clear from the field example that we can't just limit breaks to new errors.

A mechanism to fully mitigate these problems should do the following:

1. Identify existing code that would break when upgrading to a given new language version.
2. Give diagnostic messages for such code that are expressed directly in terms of the language breaking change.
3. Offer simple, reliable, fully automated code fixes to render existing code resilient to breaks.
4. Show up automatically and early, so that you don't accidentally miss out on breaking change mitigation.

Language design considerations

No matter how automatic and reliable the mechanism to mitigate it, any breaking change is going to have non-zero impact on at least some developers. There should be a high bar for language design decisions that lead to breaking changes:

• Breaking changes should be adopted judiciously and sparingly, and with clear end-user benefits.
• Concrete breaks in user code should be expected to occur rarely. Coopting a specific identifier in a specific syntactic context, as with field, would be reasonable, but it shouldn’t be much more open-ended than that.
• Each concrete break should be easily explainable in diagnostic messages, with a clear location in code; e.g.: "In C# 12, within property accessors the field identifier will refer to a generated backing field for the property."
• Each break should have a default fix that is simple, robust, local, fully automatable, and preserves the meaning of the code into the new version. For breaking occurrences of the field identifier, the fix would replace them with member access expressions, as in this.field and MyType.field.

If we cannot satisfy these requirements, we shouldn't adopt a given breaking change. We should either settle on an alternative design for the new feature, or punt on the feature completely.

Example: Getting ready for field access

Let’s assume that your current source code is in C# 11 or earlier, and has some occurrences of field as an instance field, which are being referenced as simple names from inside property accessor bodies:

public class Entity
{
    private string field;
    public string Field 
    { 
        get => field; // changes meaning in C# 12
        set => field = value.Trim(); // changes meaning in C# 12
    }

    public Entity(string field)
    {
        this.field = field.Trim();
    }
}

Now let’s say that C# 12 is released, and offers the field access feature with the breaking design. Without mitigation, upgrading your project to C# 12 would break the code above.

The default fix for this breaking change would be to turn field inside property accessors into a member access; i.e. this.field for instance members and MyType.field for static members. This fix would change the above property declaration to:

    public string Field 
    { 
        get => this.field; // resilient to C# 12
        set => this.field = value.Trim(); // resilient to C# 12
    }

Of course you might want to fix the break differently, e.g. by renaming the field variable.

How could we help you find and fix these breaks before you upgrade to C# 12? In the following, let’s sketch a user workflow for that in the context of an IDE and a CLI, using the field feature as a running example. Please note that this is just in order to paint a vision; the actual tooling experience will be designed by the appropriate teams, with the close involvement of the language designers. Afterwards we’ll go into more detail about how we could enable those workflows.

The IDE experience

In the example, remember that you’re starting out in C# 11, and with C# 11 being the latest version supported by the compiler your IDE is currently targeting.

Now a new version of the .NET SDK gets released that includes the C# 12 compiler, and you choose to upgrade Visual Studio or the SDK directly. Based on your project settings (in ways we’ll come back to later), the following now happens:

• When you open the project it is still in C# 11, but you now get additional diagnostics (let’s say warnings) on lines of your C# 11 code that would be broken by C# 12. Specifically, whenever field is used in a property accessor, you get a warning along the lines of "In C# 12, field in an accessor will refer to a generated backing field. Consider changing to this.field."
• For any given such warning, the IDE offers you a list of code fixes like the following:
  • Change field to this.field: This is the default fix described above, and requires no further user input. You get the usual additional options to apply the code fix to all occurrences in the file or project, so if this works for you, you can be done very quickly!
  • Rename field: If the field declaration can be edited in this project you might choose to address the issue by renaming it to something else.
  • One-click upgrade: Fix every remaining break with its default fix and automatically upgrade to C# 12. This is as easy as it gets; one click and you're safely in C# 12.
  • Don't fix, and suppress either this or all such breaking change warnings going forward. This is mostly for when you don't care about newer versions of C#, and want to turn off warnings about them in your projects.
• Of course you can also fix some or all of these issues manually, without making use of the suggested code fixes.
• If and when there are no more compatibility issues, you are free to upgrade the language version and/or target framework in the usual manner, whenever you choose.

Separately from this, any IDE gesture that offers (directly or indirectly) to update to a new language version should be gated by the presence of breaking change warnings in your current code. It shouldn't be too easy to accidentally upgrade the language version without heeding the warnings.

CLI experience

Based on the same project settings as the IDE, a CLI build command would report the same warnings that are provided in the IDE. When you get these warnings, you can go and fix your code accordingly.

In addition, a CLI command (or .NET Tool delivered with the CLI that acts as though it is a CLI command) could be offered to apply the default fixes to remaining breaking code, whether across the whole project, or other granularities (directory?, solution?) that make sense.

You can also turn the warnings off if you have no interest in guarding against breaking changes right now.

Detecting breaking code

Detection of breaking code would be the responsibility of the compiler. It should be able to produce meaningful diagnostics on code that is compiled with a given language version "x", but will break in a higher language version "y" that the compiler supports.

It’s a long-standing practice that newer compilers continue to support older versions of the language in a way that is compatible with older compilers. Indeed this capability is used frequently, as most projects specify a fixed language version (often indirectly through the target framework) even while often being compiled by a newer compiler.

With this proposal, a newer compiler can add diagnostics to an existing language version. That’s not something we’ve done before! Needless to say, this behavior needs to be optional; something you can turn on or off. Depending on your viewpoint, such diagnostics are either a nuisance or exactly what you’re looking for. If you are not looking to upgrade your project to newer language versions, why should you care whether your code will work correctly in those versions? In such cases you would turn these diagnostics off. Conversely, if you do plan to upgrade the language version (or target framework) from time to time, then why wouldn’t you want newer compilers to tell you that your code is not forward compatible? In fact, since the compiler knows about the new language features (as it supports those newer language versions), it would seem distinctly unhelpful for it not to tell you what it knows about their impact on your code!

The upshot is that a given compiler version would use knowledge about newer language features to produce warnings in older versions of the language about code that would eventually be broken. If you heed those warnings, then your code will not break when you eventually upgrade to that language version.

Fixing breaking code

High quality code fixes representing the default fix of each breaking change will be implemented and offered as gestures in IDEs and commands in the CLI. While the fixes would not be part of the compiler itself, they should probably be implemented by the compiler team alongside the warnings. Their existence is an important part of the protective setup that would allow us to feel confident about new breaking changes, and building all the pieces at the same time by the same people is likely the most reliable approach.

Project settings

Perhaps the most hand-wavy phrase of the above example is “based on your project settings”. It would seem desirable that the breaking change handling adds or changes as little as possible about project settings, compared to what's there today.

The landscape is a bit complicated. For instance, in an SDK-style project, a project file without an explicit <LangVersion> will implicitly feed the compiler the language version associated with the specified <TargetFramework>. That is indeed the most common setup. But elsewhere (e.g., when directly calling the compiler), leaving the language version unspecified will cause the compiler to use the latest language version that it supports, something that can also be explicitly specified with a language version of latest.

The warning behavior proposed here relies crucially on the compiler being invoked with a lower language version than the latest one it supports. Only then do the breaking change warnings manifest! So for the large number of projects where a specific language version is provided to the compiler (even when it’s implicit in the project file) this will work quite well. When a new compiler comes out, it can start producing useful breaking change diagnostics relative to the latest version it now supports.

Opting in or out

Many projects will not want to ever upgrade, and should be able to opt out of warnings that are useless to them.
We suggest that we introduce a new compiler flag to guide version-related behavior, e.g.:

Opting out:

<UpgradeLanguageVersion>never</UpgradeLanguageVersion>

Opting in:

<UpgradeLanguageVersion>warn</UpgradeLanguageVersion>

We need to decide which of the two is the default behavior if nothing is specified. This flag will "clutter" the project files of whomever does not want the default behavior. This speaks to having the default be "off", since that aligns with projects that already do not want to mess with their project files. Those projects could be left as they are today, and would never have to worry about this whole new breaking change thing that they won't care about.

On the other hand, if the default is "off", will too many existing projects that do upgrade from time to time neglect to turn it on, and miss out on the safety provided by the warnings? Is it really so bad for existing projects that don't want to ever upgrade to get a one-time warning with a fix to explicitly say no to upgrade-related warnings going forward? This way every existing project will be prompted to make a choice: Stay on the language version you're on, or fix code that would break in future language versions.

We'll need to make a choice here, and we're not sure which is the right way to go. One thing to remember is that, while the compiler has a default for what happens if nothing is specified, the SDK may have its own defaults for what is sent to the compiler if nothing is specified in the project files. The SDK also decides what is put in the project files when new projects are created in the first place. Finally, there can be a difference between what is the right default long-term vs what causes the least friction for the projects that exist today.

Latest version

For what's currently the language version of latest, whether specified implicitly or explicitly to the compiler, there’s a conundrum. With the current behavior, latest causes the language version to automatically move up when the compiler is upgraded. But that behavior would side-step the very warnings that would make this upgrade safe!

It seems that you can’t have your cake and eat it too. You can't have a new compiler both warn you about what would happen if you upgrade and immediately upgrade before you see the warnings! What to do?

We think latest no longer makes sense as a "language version" in this scheme. We should find a non-jarring way to retire it, and e.g. lock the default language version to C# 11 when you call the compiler without specifying a language version.

Instead we can allow a stronger setting for the opt-in flag introduced above to express the intent to always be on the latest version:

<UpgradeLanguageVersion>latest</UpgradeLanguageVersion>

The additional effect in the compiler would simply be that if the language version is not the latest and there are no breaking change warnings in the source code, then a warning is issued that you aren't on the latest supported version of the language. An IDE can then hang a fix off of that warning to actually upgrade your project's language version (as well as one to shut the warning up if you don't actually want to be on latest after all).

A variant of latest today is preview, which means latest plus any features currently in preview. We should move that over to the same flag, as in:

<UpgradeLanguageVersion>preview</UpgradeLanguageVersion>

That would enable preview features when the project is already on the latest language version, and have the same effect as latest when it's not.

Accumulated breaking changes

If there is more than one version between the language version passed to the compiler and the latest one that it supports, some things could get gnarly. For instance, the same code could be subject to multiple subsequent breaking changes (hopefully very rarely!), and competing default fixes may apply.

Perhaps the fixes – whether in an IDE experience or a CLI batch fixer – should be sequenced according to the language version that introduced the break, so that a later fix is only shown – or applied – when the earlier breaking change has been dealt with.

Perhaps this is too much of a corner case to even worry about. We can learn along the way and adapt – it will not be an issue until the second time a language version introduces breaking changes!

Fixing past decisions

What about the concessions we've already made in the past in the name of back compat? What about discards, var, nameof, etc. Could we fix those in future versions of C#?

We don't see why not. Making those work more consistently and elegantly could be regarded as new "features" with breaking changes. But we'd have to be just as judicious as with "real" new features: Are the breaking changes worth the hassle to developers, and are we adding too many in any given release?

In some cases, "breaking" use of such identifiers is already discouraged, even today. For instance, using var as the name of a real type is a really bad idea anywhere, except in compiler tests! In such instances we might look more lightly at the cost of the break to users, because we'd assume most people are already not writing code that would be affected.

Building the muscle

In summary, we propose to:

• Allow ourselves to make limited and well-justified breaking changes in new versions of C#
• Provide a mechanism for you to discover when upcoming breaking changes would affect your code, whether in an IDE or CLI context
• Provide local, reliable automatic fixes for any such upcoming breaks

If we adopt a scheme like this we should start small - e.g. with one breaking feature (which should probably be field) - and learn from the impact. Over time we can tweak both the warning experience and the evaluation criteria for new breaking features based on what we learn from real world usage, until we've reached a steady state that represents a new and better balance between stability and evolution of C#.

Any thoughts and reactions will be extremely helpful!

LDM Discussions

• https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-05-15.md#breaking-change-warnings

[lambdageek]

What about something like Haskell's LANGUAGE pragma - if you want to stick with C#11 but pick up the new meaning of field in every .cs file, do something like this:

  <PropertyGroup>
    <LanguageVersion>11</LanguageVersion>
  </PropertyGroup>
  <ItemGroup>
    <LanguageExtension Include="FieldKeyword" />
  </ItemGroup>

Or alternately, if the whole project is C# 12, an individual .cs file could opt out by doing No**Keyword:

  <PropertyGroup>
    <LanguageVersion>12</LanguageVersion>
  </PropertyGroup>

#language NoFieldKeyword

class C {
  private string field;
  public string Property { get => field; set => field = value; } // Property is backed by this.field
}

[Rekkonnect]

Or more elegantly, I would prefer

#language nokeyword field

This syntax would make the intent clearer and offer a better IDE syntax highlighting, much like how it already does for true and false in preprocessor directives, and numbers in warning severity pragma statements.

[CyrusNajmabadi]

I would be opposed to a single project having varying semantics for a language feature. this would be especially confusing if a language feature needed to bridge multiple files.

Our goal here would be that you would only vvery sparingly need to do any work in a project to begin with (we expect 99.9% of users to never even be affected by these changes). So having this level of intricacy would be enormously complex. This contrasts with 'nullable' where we expected it to impact all developers heavily with a clear need to be able to incrementally move projects forward.

[jaredpar]

Agree that a perf file setting is not what we want here. These breaking changes are meant to be very targeted and small. If we did need a per file setting then we've probably chosen items we probably shouldn't be taking breaking changes on.

[lambdageek]

These breaking changes are meant to be very targeted and small.

I think that's a limitation of the current "language version" approach. With switches for individual language features, breaking changes can be much more expansive

[CyrusNajmabadi]

breaking changes can be much more expansive

That is a non-goal, especially as it very likely no longer allows for the creation of simple fixers that always work.

[msedi]

If you ask me, I would accept breaking changes for the sake of going forward and allow improved designs. We carefully watch what the language and runtime team do and adjust our coding accordingly so that we are prepared for new things. That was the reason why the transition to .NET (Core) from the .NET framework was a matter of weeks.

I wouldn't prefer something silent, because I have learned from the past that simply making things not showing a problem, will be a problem later. So I would accept an error with a fix for the edge cases you mentioned. field might be an example. Only people (which for sure exist) might have used field extensively and might get into trouble, but if a code fix is available, that might no be such a big problem for them (others might disagree here).

Meanwhile (while I was typing) @lambdageek already posted something I also would be fine with, namely opting in or out. I don't know about the efforts and how easy it is to add features as options, but this would be something that could be used in a wider scope. Instead of just switching the <LangVersion> you could exclude/include (maybe only exclude?) syntax you cannot currently use (because of too much effort) and shift the transition to this feature at a later time.

From our product code, breaking changes is always a problem since we have strong approval processes in medical imaging. So I would think my other colleagues would disagree here, but they are also still on .NET Framework 4.8....

As said above, this is my opinion, there might be other perceptions.

[HaloFour]

I have concerns about anything that has the potential to create schisms in the ecosystem. The switch to .NET Core is still painful and insurmountable for a huge number of projects. Having the language add more paper cuts for each revision of the language only makes that more difficult.

That's not to say that I'm completely against deprecating and breaking changes to the language. I've argued that this should have been the tactic adopted for discards, for instance. Having the ability to break glass in the back pocket for exceptional cases I think is a good thing in general. But I think we need to be very cognizant that every little break like this makes it harder for projects maintained by less passionate and devoted developers to keep up with new versions.

[jaredpar]

My expectation is that we'd put breaking change warnings end roughly as early as we could. Consider as a concrete example the warning on lower case type names. That was implemented and shipped pretty much as soon as we realized it represented a block we wanted in the future. Yes you had to opt into that particular warning because it was a new thing but it was shipped well before RTM.

That's also why this proposal is finding a way for customers to sustainably opt into this type of notification going forward . That way it's not waiting for an entire release cycle to get a new warning.

[seanblue]

Perhaps selfishly, I wouldn't want to wait a whole extra release cycle for the field feature. It would be an incredibly valuable feature for my team (we have private "caching" fields all over the place and I'd love to cache to field instead), so the sooner the better. I think as long as these warnings are added to the IDE at least 3-6 months in advance it should be fine. There could be cases where it waits a whole release cycle simply because the feature (and its warnings) was completed too late to give proper time for developers to act on the warnings, but I don't think it's necessary to wait a whole release cycle as a rule.

[HaloFour]

Perhaps selfishly, I wouldn't want to wait a whole extra release cycle for the field feature.

I honestly would rather the C# team had a yearly release cycle for this very reason. It tightens the preview/deprecation cycle and makes it much less impactful if a larger feature has to miss a release train due to not being ready if the next train is less than two years away. I guess minor releases do exist, but they're not held to any schedule and I believe can't/don't include any larger feature releases.

Personally, I'd be fine waiting a release cycle to get a "breaking change" protocol in place and stable. I don't think any feature (and especially field, regardless of how much I want it too) is worth getting that wrong.

[seanblue]

Personally, I'd be fine waiting a release cycle to get a "breaking change" protocol in place and stable. I don't think any feature (and especially field, regardless of how much I want it too) is worth getting that wrong.

Yeah, that's totally fair. It would be really bad to get something like this wrong.

[luislhg]

I don't think it should wait a year/cycle because no one is forced to update as soon as available. Every team/project can opt to update at their own time.
A breaking change arrives in a new version, they will see the error/issue/change and opt to fix or just not update now.

This was nicely pointed out here too
#7033 (reply in thread)

[IS4Code]

I don't think preventing hypothetical confusion of developers outweighs confusion and frustration of those who suddenly find their code (or code they had from elsewhere) not working. If you instruct a developer about contextual keywords, they will know about this for the rest of their lives, but I guarantee you will always be able to discover code that broke as a result. We can all move forward here, but all existing code won't.

[MadsTorgersen]

That's probably true. But at least you'd be able to take any existing code to the doctor: compile it with its old language version with a new compiler find out where the breaks are, and fix them.

[CyrusNajmabadi]

of those who suddenly find their code (or code they had from elsewhere) not working.

The point here is that you would not suddenly find your not working :) It will always work, and continue to work with the same semantics as before. The only thing that would change stuff is if you took explicit steps and decided to make it have different meaning.

This keeps the developer in control, with tooling to help them.

but I guarantee you will always be able to discover code that broke as a result.

Not sure what you mean by this. It also sounds like you're saying that despite education, you still run into breaks. The goal here is that you wouldn't need to have to worry about that as any breaks (if htey had the very unlikely chance of hitting your codebase) would always be something you'd be notified about, instead of potentially being silent.

We can all move forward here, but all existing code won't.

The above proposal is not in conflict with htat. If existing code can't move forward, it will never change meaning. The compiler/language is fully committed that if your code doesn't move forward, that it's meaning always stays the same (or at least, for a very very very long time). This is the same compat guarantee as today, and we do nto intend on changing that. This is precisely because we expect there to be a ton of code that cannot move and needs to be safely preserved.

THe proposal above those is for users who do want to move forward and explicitly opt to. It allows for complete safety on doing this, and full awareness and explicitness of what that entails and what impact that may have.

[alexalok]

We can all move forward here, but all existing code won't.

If I understand correctly, existing code won't break unless the project is upgraded to a newer C# version, either explicitly via LanguageVersion or implicitly by updating to a newer SDK. In other languages it is often expected to spend some time adapting your code after such an upgrade.

[Jure-BB]

Assume more code is yet to be written than it currently exists. Confusion and frustration caused by inferior language (due to not improving specific areas to avoid breaking changes), will compound over time. As long the cost of upgrading existing code is minimal, I think improvements will pay off in the long run.

[iam3yal]

Generally I think that this is a good thing and a step in the right direction for the language, I know the LDT is pretty mindful and careful about the changes so personally I'd welcome it.

It would be nice to have a document where all these breaking changes are documented per version of the language as well as treating these changes similar to features where they are discussed with the community, I know many of the LDT members are doing it actively just thought to bring this up because I haven't seen it in the OP. :)

[CyrusNajmabadi]

FWIW, we do doc this here: https://github.com/dotnet/roslyn/tree/main/docs/compilers/CSharp

You can see all the breaking changes docs. I'm sure there's space for potential improvement here in terms of organization and discovery though. Thanks for the feedback on this!

[iam3yal]

@CyrusNajmabadi That's really great, some of us spend more time browsing this repo than Roslyn so it might be reasonable to add "Breaking Changes" as an item on the list of features starting with C# 12 for convenience.

[CyrusNajmabadi]

Makes sense to me:-)

[RikkiGibson]

They both link to the same stuff.

[iam3yal]

@RikkiGibson What exactly do you mean? Maybe I'm not explaining myself as good as I should but right now I don't think there is an item on each version of C# in the Language-Version-History.md that says "Breaking Changes" or something similar and link to any of the files here, I don't think you need to do it retroactively but it might be a good idea to do it in the future. Sorry if I sound like a broken record. 😃

[jnm2]

This proposal is spot on.

As a library maintainer, I've also long wished that tooling could present warnings and automatically fix up code during library package version upgrades, for example to move away from removed deprecated library APIs. Both of these aspects would be important:

• Any tooling that updates to a new library version should be gated by the presence of breaking change warnings in your current code. It shouldn't be too easy to accidentally upgrade the library version without heeding the warnings.

• One-click upgrade: Fix every remaining break with its default fix and automatically upgrade.

[KathleenDollard]

The current proposal is for C# changes only, not deprecated APIs. We may consider that, but I think it would be a broader conversation with the community.

[yaakov-h]

That said, at the moment I believe a library could also ship an analyzer to achieve the same effect.

[pinkfloydx33]

Would there be a way to possibly have these new warnings be auto-excluded from warnings-as-errors? Sometimes we aren't in control of the exact installed SDK version, say on a shared build server where we want the updates without resorting to a global.json. I wouldn't want the warnings now appearing from a minor SDK bump to suddenly stop code from compiling at all

[CyrusNajmabadi]

You're welcome! :)

[KathleenDollard]

Also...I am working with Chet Husk on recognizing that we are compiling from an editor to open the door to displaying the warnings by default in an editor, and turning them down when calling dotnet directly. Definitely want feedback on when to opt in (if you do not see the warning prior to upgrading your LangVersion or TargetFrameowrk you will never see it).

[pinkfloydx33]

Oh see that would be nice

[seanblue]

I was also concerned about the warnings-as-errors thing and would like some additional clarification.

1. If we have warnings-as-errors enabled and show our intent (via the new flag) to update to new versions (meaning we want to see warnings), would those warnings show up as errors and fail the compile?
2. If so, will there be a way to force all warning of that class (i.e. all language breaking change warnings, current or future) to stay as warnings or would we have to add exceptions in the WarningsNotAsErrors list (or elsewhere) for each individually? (It's not a huge deal either way, I'm just curious.)

[TonyValenti]

In my world, the following items tend to get upgraded at the same time:
Visual Studio
.Net Runtime
LangVer

Perhaps Roslyn should take a similar support structure as .net with LTS and STS releases.

Imagine this scenario:
I'm running .net 7.0.3 and I have field variables.
The C# compiler for .net 8.0 preview 2 makes field a keyword which will break my code.
When I upgrade to .net 7.0.4, my code generates new warning complete with a fixer.

With this workflow, the upgrade process would be:
Before upgrading major versions, upgrade to the latest supported minor version and run any fixers.

[TahirAhmadov]

I think this makes perfect sense. Especially with the careful approach as outlined in the OP, I don't expect it to be a problem.

One thing I would add, is there a plan to try measuring the impact of the breaking change in real life? Like some sort of big data analysis of all public and maybe even private (per agreement with customers) code, to determine if it's a concern at all? For example, with field keyword, do people really have many variables with that name? I'm not saying they don't, just asking if it's feasible to measure

[MadsTorgersen]

I wouldn't say that there's a plan for anything yet, but we do frequently look at our internal codebases and at public GitHub repos to get a sense of how prevalent a given pattern is. We would certainly do that for any breaking change under consideration.

For field there are some. Consider how many meanings that word can have in particular domains - physics, agriculture, sports...

[TahirAhmadov]

Another thought I had: what if these quick fixes "phoned in" anonymously, providing you guys with stats? (Obviously there's a privacy question here. ) would that help you make a more informed decision WRT how prevalent the to be broken pattern is, and whether users are actually going through with applying the fixes?

I'm asking because, imagine a situation where you have an idea that "field" is widely used, yet most people aren't fixing it (for whatever reason), and knowledge of users ignoring the fix would allow you to react - either increase the communication, or even postpone a feature to a future version, in order to avoid real world harm.

[torbica]

but we do frequently look at our internal codebases and at public GitHub repos to get a sense of how prevalent a given pattern is.

I understand that you probably can not do much more but please consider that most of C# code is not public.
I am writing C# for almost 15 years and never had something public. In fact out of 5 different companies where I have worked not a single one had even a line of their code on public code hosting platform.

[CyrusNajmabadi]

We have many source of private data as well. We're also in communication with many thousands of developers, companies and organizations who work with us and trial out the ideas/prototypes/previews of what we ahve.

in order to avoid real world harm.

Could you give an example of what you mean by "real world harm". The idea here is that nothing should ever break. Code continues to work properly, with the user in complete control about if/when they might even want to adopt certain new language features (the same as in the past). Our commitment to compat remains the same as before. All this is to prevent there from being any harm, while also allowing the language to have simpler semantics (which honestly normally align with how people think it works in the first place :)).

[TahirAhmadov]

Could you give an example

Thinking about it again now, I feel like I was being too cautious. The scenario I imagined was that potentially, some number of users would ignore the warnings; then upgrade to the new version and experience a number of compilation errors/runtime bugs, and blame the language for it, creating an overall negative impact on the ecosystem.
However, the principle of "personal responsibility" applies here and the language cannot possibly account for users' poor choices.
In general, I support this proposal, and in fact, I think it's vitally important to ensure continued improvement of the language; it's mathematically impossible to continue the development, as the "debt" piles up over many years.

[hez2010]

How about introducing something like the concept of edition in Rust?
That's say, we can have different language editions and there may be many breaking changes between editions.
For example,
<LanguageEdition>2017</LanguageEdition>
represents the compatibility requirement of the feature set in 2017, and it may still receives new features that are non-breaking changes. Users can still adopt new features that being backported without concerning about compatibility issues. It means the old edition of language can still receive new features from newer language versions, if there is no compatibility concerning.

When you want to make some major breaking changes to the language, you can bump the edition, such as 2022 (assuming C# release a new edition every 5 years), so users can choose which edition of the language at their own risk. An update assistant can also be developed to help users fix source compatibility issues during edition upgrade automatically. As for the language team, maintaining only three editions can you already support users from a range up to 15 years.

This also enables the ability to withdraw some bad designs that already existing.

Typically, we want C# to clean up some old features that being superseded by new features. For example, when we have roles and extensions, we don't need extension methods anymore, so we would like C# to remove the old extension methods so that it can prevent the language from bloating.

We don't need plenty of ways to express a same thing. But as the language evolves it is not avoidable, so we need a mechanism that allows us to remove outdated things from a language.

With editions, the language can always revise the existing features again, fixing or even removing outdated/misdesigned features recursively, in a period of several years. This is essential and can keep a language clean and efficient.

[MadsTorgersen]

I'm not in favor of deprecating features from C#. It puts people in an unnecessary binary choice between keeping old code and using new features. I like soft approaches like Everyday C# to keep an updated list of currently recommended practices, and perhaps tool that with analyzers, so that you can voluntarily limit yourself to a more curated subset of the language.

[orthoxerox]

I really like the way Rust handles breaking changes with editions. Instead of spreading small breaking changes across the releases they bundle them into editions that are currently spaced three years apart (in C# years that's like 10 years apart) and you can automatically upgrade to the next edition with a single cargo fix command most of the time.

[MadsTorgersen]

I think that's fine too. It puts breaking changes on a more coarse-grained schedule than other language changes, which means you have to deal with them more rarely, but when you do it'll be a bigger deal.

In .NET we don't really have as much of a "version upgrade" experience, except from .NET Framework to Core based .NET. In this proposal I tried to steer clear of anything like that, not because I am particularly against it, but because it would easily turn into a "boil the ocean" kind of thing, where everyone's experience changes.

I feel like the proposal as currently written is agnostic to such bigger upgrade experiences. It doesn't presuppose them but should work fine with them. And it's also independent of batching policy for breaking changes: if we chose to do breaking changes more rarely, even on a predetermined schedule, there just wouldn't be any new breaking change warnings in the meantime.

Currently I think I lean more towards a finer granularity, but this is something we can tune along the way as we gain experience with it.

[yaakov-h]

How would this work with the VS IDE experience?

Whilst I'm not familiar with the exact intricacies, it is quite clear that the in-IDE analysis seems to use whatever latest version of MSBuild and Roslyn that it shipped with, even when the actual project compilation uses a different version thanks to global.json.

I wouldn't want to be in a position where I have a project on SDK 7 that uses 7 defaults (no explicit UpgradeLanguageVersion and no explicit LangVersion, with a TFM of net7.0[-*]), but when upgrading the IDE to whatever ships with .NET 8, I now get C# breaking change warnings in my .NET 7 / C# 11 project.

To take that one step further, let's say I have a .NET 7 / SDK 7 / C# 11 project, and I set it to <UpgradeLanguageVersion>warn</UpgradeLanguageVersion> and use global.json to build with SDK 8 / C# 12, which introduces some breaking changes. Then I install the SDK 9-equivalent build of the VS IDE. In this scenario I'd only want to get the SDK 8 (C# 12) warnings, not the SDK 9 (C# 13) warnings.

[yaakov-h]

So for context, my particular concern is that my company has a very large codebase and many areas are stable and reliable and don't have a developer actively working on them in VS every day, week, or even month because it does what it needs to do already. We also build with warnings as errors.

I wouldn't want to be put in a position where I can't update Visual Studio on either CI machines or developer machines without going and updating every single breaking change in all of our code, i.e.:

• Building older code that uses an older SDK and older TFM, even it doesn't explicitly specify a LangVersion, should continue to build without new warnings/errors after updating VS. This realistically shouldn't be a problem as dotnet build doesn't involve the IDE, but I just want to flag it anyway.
• Opening an older solution that uses an older SDK and older TFM, even if it doesn't explicitly specify a LangVersion, shouldn't introduce any new "breaking changes" warnings/errors. We already get new warnings for dotnet/roslyn-analyzers changes and that's annoying enough.

IMO should be some kind of per-repo/sln/project action required to trigger the new warnings. There's already an action required to update to a new language version - either by changing LangVersion or by changing the TFM, that doesn't just happen by itself... hmmm.

Definitely needs more thought in this area.

[CyrusNajmabadi]

I wouldn't want to be put in a position where I can't update Visual Studio on either CI machines or developer machines without going and updating every single breaking change in all of our code,

This should not be a concern. We have no desire to break anyone, and our existing compat story stays the same (and likely will for a very very very long time). You should be able to update VS/CI in the same fashion today with the same expectations you have today.

Building older code

It's very important to us that old code, especially code not being actively maintained and worked on, be safe from problems on the upgrade paths.

Opening an older solution that uses an older SDK and older TFM

Same.

--

The areas where you might (and again, i must stress how extraordinarily unlikely this is to hit you anyways), have to deal with a change in semantics, are when you you actually to upgrade to a later langversion. Specifically, not that you installed the new version, but you upgraded your project to use it.

[yaakov-h]

(and again, i must stress how extraordinarily unlikely this is to hit you anyways)

Considering how many minor breaking changes I've already encountered, including two significant Roslyn bugs in VS2015, I don't think my company is as unlikely to hit these are you think we are. 😉

Specifically, not that you installed the new version, but you upgraded your project to use it.

That's good then :)

I'd be fine with that happening now that TFMs have implicit defaults for LangVersion, though we do tend to override them for older TFMs (e.g. <LangVersion>11</LangVersion> for a netstandard2.0 project)

Back on the old SDK when every project was implicit "latest," that was much less fun.

[CyrusNajmabadi]

Considering how many minor breaking changes I've already encountered, including two significant Roslyn bugs in VS2015, I don't think my company is as unlikely to hit these are you think we are.

A roslyn bug is a bug. If it exists, we may choose to fix it (including breaking people). This is not about shipping/fixing bugs. :)

This is about what happens when we would like to make a language change intentionally that could potentially break meaning. Is there a path to do that that is safe for people like yourself, but also available for projects that are ok upgrading. :)

[yaakov-h]

Sure, my point is just that this code is already full of C# edge cases, so I'd be completely unsurprised to find more that the LDM thinks are extremely rare.

[Joe4evr]

In some cases, "breaking" use of such identifiers is already discouraged, even today. For instance, using var as the name of a real type is a really bad idea anywhere, except in compiler tests!

You seem to forget there are (well... allegedly) some code bases out there that on purpose define real types named var/dynamic/etc. because they have guidelines that prohibit any and all use of those keywords. There's an argument that if those types can no longer be named as such, an alternative way of banning that small subset of contextual keywords should exist. I've mentioned before that .editorconfig is probably a prime candidate for this, since it already has options for var usage (which while I'm quite a proponent of var, I will wholeheartedly take any option to ensure dynamic isn't accidentally used).

[MadsTorgersen]

You are right that I didn't go into that. I wasn't exactly forgetting about it so much as waiting to see if anyone brought it up. 😉.

While I understand why people do this, I don't think it is a viable or generalizable mechanism for avoiding certain language features. As you say, editor config is a better place (and AFAIR it already helps with var), and there's always the backstop of writing an analyzer, or grabbing one off the market.

In the case of dynamic, declaring a type doesn't really help you avoid the dynamic type: you can easily get it as a return type, and you can even end up with local variables of type dynamic - through use of var. The better way is through static semantic analysis such as analyzers.

[jaredpar]

You seem to forget there are (well... allegedly) some code bases out there that on purpose define real types named var/dynamic/etc. because they have guidelines that prohibit any and all use of those keywords.

Teams with such policies have a number of other tools available to enforce such policies. For example an analyzer can be authored to ban any particular keyword, C# feature, etc ... We can't keep holding back progress in C# in the ecosystem for teams that want to pick and choose which features are allowed in their code base.

[CyrusNajmabadi]

Right. There's a distinct difference between someone coding normally, who happens to write something that might change meaning wrt a future lang version, versus someone explicitly going out of their way to try to ensure that their code will break if we wanted to codify some semantic-identifier.

If someone intentionally squats on things in this fashion, versus using existing mechanisms that are actually intended to solve these use cases, then that's not something we're bothered by.

Note: those customers will continue to work, even as they upgrade their compiler. We will not break them. We're just no longer going to say that they can upgrade lang version without difficulty. They made their bed :)

[Joe4evr]

We can't keep holding back progress in C# in the ecosystem for teams that want to pick and choose which features are allowed in their code base.

Of course, I was just trying to get something going on if/how those certain cases would move forward and if it would be something more broadly usable. (Evidently the answer seems to be 'no' at this time. ¯\_(ツ)_/¯)

[OJacot-Descombes]

A possible fix for existing field identifiers (instead of this.field) is to use the the verbatim identifier character: @field. The auto-backing field is logically also this.field, so using this. is a bit confusing. Unlike this the @ works for type names, local variables and parameters as well.

[sab39]

My vote is to favor new keywords over contextual cleverness, so that @ can be applied globally as a universal solution when these issues arise. (I didn't realize that @value worked like that, and I don't like it at all - it seems completely counterintuitive to what @ is supposed to do!)

[CyrusNajmabadi]

it seems completely counterintuitive to what @ is supposed to do!)

Here's the thing about @, all it does is say that a word is an identifier. However, value is already an identifier. value is not a keyword to the language (never has been, all the way since 1.0). It's just a bog standard name (not keyword) that is in scope in a property setter. So, adding @ has not ever changed the meaning of it. @ was always supported on all identifiers as a non-affecting annotation. This was intentional and meant that things like code-generators could always prefix with @ whether a word was a keyword or just an identifier.

Given that, it would then be super weird to introduce a new identifier for this use case and have it not behave just like the existing feature that otherwise is pretty much the same thing :)

[sab39]

Yeah, I guess I should have said that value being an identifier rather than a true contextual keyword is the thing I find counterintuitive. I understand that it was intentionally defined that way, but it's an uncomfortable fit for me: correct me if I'm wrong, but I don't think there are any other identifiers that don't have a declaration somewhere, and just magically exist. Wouldn't the rationale for making value an identifier, taken to its logical conclusion, mean making this an identifier that magically exists in non-static contexts?

[Rekkonnect]

In C# 9.0 args was added for top-level programs. But yes, those magically-available identifiers can be discovered in few places.

[CyrusNajmabadi]

but I don't think there are any other identifiers that don't have a declaration somewhere, and just magically exist

There definitely are. Lots of them. "var", "dynamic", "notnull", "unmanaged", "args", etc. etc. etc. :)

mean making this an identifier

We could certainly have done that. However we do not want 'this' to mean something else elsewhere. For example, it would be a bad thing (in our opinion) to allow: var this = 0; inside a static method. You could certainly design a language to allow that. But it's something that could be very confusing.

That is directly different from something like 'value'. We do not want to make that a keyword. As a keyword, you would not be able to use it elsewhere, and that would be a bitter pill given how valuable (har har) that word is.

[CyrusNajmabadi]

@Korporal Thoughts on alternative grammars can be made either in #5311, or in a new discussion if you'd like to bring up new ideas. The idea of going with an alternative grammar (or making changes that would virtually break all programs out there) is wholly out of scope and entirely off topic for this discussion.

[AlwaysHC]

The supported language version needs to be specified in the single file.
What if you include an old C#7.3 file to a C#12 project?
Warinings regarding breaking changes in that file should appear for security reasons

[BobSammers]

it doesn't currently have a mode for looking backwards while already in a newer language version

Only one example, but in the field discussion, is there any reason to retire the warning in any version? If someone is using field inside a property but there is an in-scope identifier of that name explicitly declared, that seems like a warning scenario regardless, or have I misunderstood?

I haven't thought this through for other potential breaking changes, but my expectation is that most changes to the language would merit a warning in perpetuity when developers may fall foul of it.

[CyrusNajmabadi]

, that seems like a warning scenario regardless, or have I misunderstood?

That's a potential option, but it's somewhat out of scope for this discussion. We may or may not choose to nudge people toward/away certain patterns. This nudging could be in the Roslyn compiler itself, the core Roslyn-IDE set of analyzers. Or analyzers delivered by the dotnet/roslyn-analyzers team. :)

[BobSammers]

it's somewhat out of scope for this discussion

I was intending a discussion point for the wider issue of managing breaking changes, not to debate this specific example. Sorry if this wasn't clear.

My point was that in the wider context of introducing occasional breaking changes, I would expect that one of the principles (perhaps one of the costs) is that warnings about potential transgressions would have to be suitably robust when code of different vintages can mean two different things.

[AlwaysHC]

The copy-paste problem is definitely real. I think it applies to any scheme I've ever seen for managing breaking changes in languages. The mechanism I've proposed doesn't address it, and it doesn't currently have a mode for looking backwards while already in a newer language version, as @HaloFour suggests. I think it could be expanded to include that, but likely at considerable complexity cost.

My position for now is that we'd be disciplined enough about which breaking changes we add that the likelihood of getting into trouble because of this issue is low. But we should keep an eye on it regardless, and be willing to consider mitigations if it turns out to be bigger than we thought.

What about a #pragma directive to wrap the code/file from older versions of the language included in the project?
It will permit the compiler/analyzer to show the warnings if required and a "Suppress the not required #pragma directive" message when the included code/file is updated.

Ex.

...
C# 12 code;
...
#pragma enable langversion 11
...
code from SO or old code base
...
#pragma disable langversion 11
...
C# 12 code;
...

[BobSammers]

What about a #pragma directive[...]?

Wouldn't it be quicker to accept one of the default fixes offered by the tools than to surround each offending block? In addition to speed, this would:

• Guarantee that the original intent of the code is maintained
• Avoid the (possibly complex) need for the compiler to support more than one language level in a single cs file
• Remove ambiguity for future code maintainers who a) may not know what the breaking change was or b) may not notice the pragma

[alrz]

Opting in or out

Would it make sense to add breaking change warnings at some (not necessary the latest) level of warning waves?

It's already possible to get those on a specific version since there's no relationship between /langver and /warn.

<!-- we're at 7.0 for some reason but give me all the warnings -->
<LangVersion>7.0</LangVersion>
<AnalysisLevel>latest</AnalysisLevel> 

For a more explicit option, the SDK already provide AnalysisLevel<Category>, so this would be just another category if that's all you want.

[jmarolf]

I definitely think it would make sense for it to be a category so developers can explicitly disable or enable it in all the same scenarios that they can for existing categories. I still think we would have the explicit properties described in the proposal.

[OJacot-Descombes]

New keywords or standard identifiers could be introduced as invalid C# identifiers, like $field. Not very beautiful but at least not a breaking change.

[CyrusNajmabadi]

Design of a particular feature (like semi-auto-props) should take place in that feature. This is not the place to discuss that. Of note, we considered many different variants of that feature, much of which would not have any conflicts with existing code. However, the crux of the issue is htat we found all those options undesirable. They were all unpleasant in some way and went against how we felt the feature should be designed (esp. if we were doing this in c#1). That's a core reason this discussion is starting. Because we'd like to be able to design the features cleanly, though in a way that is completely safe for the community in the unlikely case that their code will conflict with it.

If you want to state that you would prefer that all new features must come with syntax that cannot conflict with existing code, that's ok to stay here, and your opinion is definitely heard. However, while that's the strategy we've currently employed up till now, this is the discussion about seeing if we can relax that in ways that work well for both us and the community.

[gerneio]

This might already be a thing, but having some sort of source-control level analyzer notifier of these sorts of breaking changes, where the owners of the embeded repos are emailed when their repository is detected to have old lang ver projects that will will break in future versions. It could theoretically inspect all the same settings as mentioned in the OP and maybe even auto-submit PR's/issues for fixes, but the important bit is just getting the email warnings. In my mind, this would work in a similar way as the github dependabot, just customized and tailored to the C# breaking changes experience. Not sure if this possible/practical, but something that would be needed for main stream source control providers (i.e. Github and Azure DevOps). Might require a one-time setup per org, not sure.

The end goal being to ease an organizations upgrade process to new lang vers with breaking changes, since at an org/team level, we will have, potentially, multiple dozens of projects under our scope.

[jmarolf]

Not a bad idea, perhaps we could talk to the dependabot folks and contribute something over here. I know they aren't interested in adding "new ecosystems" at this time but perhaps they could be persuaded. Regardless, I think we need to ensure that we have all the apis in place so if someone wanted to build a service like this the heavy lifting would already be done for them.

[robkeimig]

This proposal sounds acceptable to me. I started out not liking it, but the 4 criteria established above under Language Design Considerations are enough for me to see the light. The first thing that pops into my mind when we see a breaking change - "who asked for this and why is their use case more important than mine?"

The other edge of the sword is that the ever-forward progress of the C# ecosystem has gently nudged us towards higher-quality code. Biggest among these progressions being null warnings-as-errors and related corrective measures.

Backwards compatibility is a really important topic for me, but I recognize trying to achieve 100% coverage would become extremely prohibitive to the roadmap (i.e. stuck in a local minimum).

[CyrusNajmabadi]

Backwards compatibility is a really important topic for me, but I recognize trying to achieve 100% coverage would become extremely prohibitive to the roadmap

Note taht we do not offer total back compat even today. We make judicious changes to the language where we've found it appropriate (applying the bar that it really should impact almost on one, while being a net win for almost everyone). We have silently (in that no tooling tells you about it, not that we didn't announce it) taken these changes in the past and found it was fine for the ecosystem.

[seanblue]

I definitely don't want to derail this by going too deep into a specific example, but I figured I'd at least mention the inconsistency between is SomeType (is-type construct) and is not SomeType (pattern matching) as a potential case for making a breaking change. It probably occurs very infrequently, but as long as the proper warnings and IDE suggestions can be implemented, doing this change may be another of these "small impact, but makes the language more consistent" cases, similar to the _ discard case.

See the following for more details:

• is not SomeType fails to compile if there is a member with the same name roslyn#50671
  • Specifically is not SomeType fails to compile if there is a member with the same name roslyn#50671 (comment) for the explanation of why it's technically by design right now and is not SomeType fails to compile if there is a member with the same name roslyn#50671 (comment) saying it's actually a "feature request".
• Add is not type constraint (different from is not pattern) [breaking change!] #4533
• "is Foo" works but "is not Foo" doesn't when there is a Type Foo and other member with the same name Foo #4743

Again, I'm not requesting any discussion on this specific example here (that can be done in either of the two discussions linked to above), but I think having more examples is helpful since there's probably a lot of nuance around what kind of breaking change is worth including in this proposal and what isn't.

[RikkiGibson]

I can tell you I don't get a lot of joy out of reviewing the code which decides whether expr is Item and is an incomplete conjunctive pattern or is declaring the variable and.

[miyu]

Just break things. I know that's not going to happen, but I'll ask for it anyway.

The cost of debt is infinite over time. C#/.NET are 23 years old and long overdue for breaking changes to right the past... I hope in 2050 we aren't still dealing with today's legacy brokenness from 2003, but as of today we are 100% on that path. Minor breaks are a good start.

Would collections or reflection look the same if redesigned today? Probably not - they'd probably at minimum have better interfaces... How about IL, the build process, or native interop? They'd probably be quite different too; today we can't declare nullable as a constraint or use attributes on loops/variables/statements, which would be nice for GPU / high-performance computing, and presumably codegen would be quite different & projects would be structured differently with multiplat AOT in mind. The native interop story is still pretty painful; the VS tooling is great, but the language support is still barebones. Yes, these are all seemingly minor, but they add up into pretty giant cracks.

I frequently wish what Kotlin did to Java's stdlib would happen to C#, or that .NET were split into .NET Enterprise Version stuck in 2001 alongside a .NET Modern that tries to do things right.

[RikkiGibson]

My recollection is that .NET Core started out by taking relatively large breaking changes from Framework. Customers then had problems adopting it and many of those breaking changes had to be walked back.

I get why the imperative to avoid breaks can be so painful. But it's also a case of: because we are usually so diligent about avoiding them, it can make it hard to see just how disruptive it is when they do occur.

IMO, this discussion/proposal is us owning up to: there is an extent where maintaining compat can be pathological. With things like async await(await async) { } being a valid method or whatever, we are going beyond the standards set by our competitors in a way that may not be adding as much value--so how do we build a gentle ramp which lets us actually break things when we judge the cost/benefit to be worth it.

[CyrusNajmabadi]

It's unfortunate that minor breaks are controversial

That's a fair statement. But it's also the truth of hte situation. And, importantly, it's the truth primarily because of the external pressure the community itself raises. Breaks come with very serious consequences/ramifications, and we have a ton of experience trying to thread this needle, only to find it falling entirely flat because the community response is to simply not accept it, forcing backtracking and additional customer pain and confusion.

Until that actually changes, i don't see us approaching the breaking change discussion differently.

[hez2010]

I don't know if that's true. I imagine the vast majority would not even know the break happened.

Just see how Python broke things from 2 to 3 and the fact is that it doesn't affect Python becoming the most popular language nowadays.

[HaloFour]

@hez2010

Just see how Python broke things from 2 to 3 and the fact is that it doesn't affect Python becoming the most popular language nowadays.

Python 3 was released in 2008. In 2020, when Python 2 reached end of life, it still accounted for more than 50% of all python code in use. Even today, after much more effort to migrate and significantly more libraries supporting Python 3, it's still estimated that more than a third of python code is still python 2.x. Yes, Python is popular as a whole, but the ecosystem is still painfully fractured almost 15 years later.

[DavidArno]

... the ecosystem is still painfully fractured almost 15 years later.

I wonder whether there will be a similar painful fracture between .NET Framework 4.8.x, stuck on C# 7 and .NET 17 & C# 21 in 10 years time with some having to try and maintain both? We use VSTO for example, so parts our code base will remain attached to the Framework. This is similar to the Python issue where much of the code is stuck using V2 libraries that won't ever move to Python.

I know that Guido has said he can't see there ever being a Python 4 due to how painful the transition from 2-3 was, but Python is in a much better position as a language by going through that transition. I understand why C# takes a cautious approach to changes, but it comes with downsides of redundant baggage and what's now seen as bad practice baked in. There's no right answer to this situation.

[BhaaLseN]

Generally in favor...and thats mostly because it wouldn't make a difference for most of my code bases (which use underscore-camelcase for backing fields whenever it isn't an automatic or read-only property).

However: Code style (and its enforcement through .editorconfig and friends) is something that comes to mind. My code bases also use "don't use this. unless absolutely necessary" - just throwing out there that the code fix shouldn't suggest using this.field and at the same time flag the location to "remove this." afterwards (while still on a language version that does not have the field keyword).
For that reason, I'm almost leaning more towards the @field suggestion that came up; but I'm pretty much impartial to either (since it wouldn't affect me at all).

[CyrusNajmabadi]

My code bases also use "don't use this. unless absolutely necessary"

Wouldn't that apply here though? It becomes an 'absolutely necessary' sort of situation, right?

I'm almost leaning more towards the @field suggestion that came up;

This wouldn't fit what we're proposing here as that's a case where we woudl again be designing the lang feature differently than how we think it should be designed. To be clear, there are many designs we have in our arsenal that cause absolutely no issues at all. We were moving forward with (and may still take) a design that has no breaks whatsoever. However, we find these approaches distasteful as they then violate the existing spirit and design of the language so far. For example, @value doesn't work that way in a property, so now we would continue having special cases and subtle differences someone would have to understand in this space

The new design space question is: can we move away from that, designing the lang feature as if it was v1, while not being a problem for users that might overlap that construct?

Our hope is that we can be responsible stewards here, enabling that to happen, while also not causing any problems for users. If it turns out we don't feel that that's feasible, we'll just continue with the existing path of always making sure there's no break, even if the lang feature has slight warts compared to if it was in v1.

[MgSam]

The problem with this proposal is that it treats silent breaking changes and previously-legal-but-now-an-error as nearly equivalent. They are not even remotely equivalent. Errors are loud, easy to find and easy to fix. Silent breaking changes are a nightmare- warnings will not work as most devs have learned to ignore them (they are so often spurious).

User's dealing with breaking code will always be orders of magnitude more expensive than those costs born by the compiler team. The field breaking change proposed seems certain to me to cause silent bugs for people and broken applications. Microsoft's general commitment to backwards compatibility is one of the strengths of the C#/.NET ecosystem- I don't think that should be thrown away. You guys should absolutely avoid silent breaking changes at all costs.

That said, more interesting to me than anything in this breaking change proposal is going the opposite direction- turning some of C#'s keywords into context-sensitive keywords. It is silly that class or params, for example, cannot be the names of local variables, for instance. Furthermore, these v1 keywords to most devs feels sort of like a bug given so many other "keywords" are context-sensitive. I've had more than one occasion where someone asked me confusedly why their variable name was giving them strange errors.

[DavidArno]

I don't think that'd be possible, field is too common of a term in a lot of industries.

Of course it's possible. goalpost is a very common term and - outside of sports fields - they are notoriously easy to move.

But more seriously, field is a terrible term to use for a C# field beyond example code. I accept that making field a reserved word likely will not happen (maybe: I gave up trying to work out whether it might or might not happen from Cyrus's replies yesterday), but - wantonly paraphrasing you - "it would inconvenience those that haven't bothered to think of good names" is a very poor defence for keeping it flexible in my view.

[HaloFour]

Importantly, what this proposal does is open up the conversation about whether or not such breaks are allowed at all. That does not mean a blanket policy as to how those breaks might apply to different language features. I'd expect that for any feature that is reconsidered that it would be done so from the perspective of removing warts and weird "gotchas" due to having to favor some previously existing meaning, not to push a one-size-fits-all approach.

I also don't see this proposal as open-season for relitigating the syntax for every language feature added in the last 20 years.

But more seriously, field is a terrible term to use for a C# field beyond example code.

Or code that does a lot of reflection? I've certainly named fields field. I've named locals local and parameters parameter, too, because those were descriptive names. That said, I've also named events @event, due to the fact that it was a reserved word.

I don't think there's anything wrong with contextual keywords. C# has had them since 1.0, which is why value is also not a reserved word. If field were to have been included in the language in C# 1.0, I'd expect it would have been given similar treatment.

[DavidArno]

Or code that does a lot of reflection? I've certainly named fields field. I've named locals local and parameters parameter, too, because those were descriptive names. That said, I've also named events @event, due to the fact that it was a reserved word.

In my view, reflection, like unsafe, is stepping into "here be dragons" territory. It is code that steps outside of the security of compiler checks and so is prone to breaking without warning. I genuinely do not see "we might break code that uses reflection" as a consideration for any change.

I don't think there's anything wrong with contextual keywords. C# has had them since 1.0, which is why value is also not a reserved word. If field were to have been included in the language in C# 1.0, I'd expect it would have been given similar treatment.

Again, I'm purely speaking from personal experience, but I think value is an example of a wrong decision made in v1 that we keep defending without justification. I am genuinely arguing for promoting value and every other contextual keyword into full-blown keywords. I really think it would simplify the language massively if the team did so. With hindsight, contextual keywords were a mistake and I really hope this proposal leads to the end of them.

[CyrusNajmabadi]

Again, I'm purely speaking from personal experience, but I think value is an example of a wrong decision made in v1 that we keep defending without justification.

In order for me to even slightly start to think there was something wrong with it, we'd need to have heard about a problem there for the lifetime of the language. It still remains something that it does not appear anyone has ever had a problem with. So the only reason to consider it 'a wrong decision' would be using some other sort of appraisal of the situation that doesn't actually align with what how users are using the language and what they're thinking about wrt these features.

I really think it would simplify the language massively if the team did so.

You state this... but i have no idea what the justification for this is. Having written parts of the spec (and someone who has to read it all the time), this would be likely <0.01% of the spec/language that this would simplify (not exaggerating).

There are occasional impl areas where i would actually agree on there being a simplification. For example, in modifier parsing. But from a language/impl perspective simplification, things like value are definitely not even the tiniest blip on the radar.

I really think it would simplify the language massively if the team did so.

This approach means that anything we ever want to do must be a keyword, and will impact someone who ever wants to use that word for it's normal meaning, which would otherwise not be a problem. 'value' is particularly damaging as, just checking basic repos, it shows up in thousands of cases outside of properties. So we'd squat on a word that would actually serve to be quite annoying to people who would otherwise have really preferred to use this. That not only doesn't help the language (see the above assessment on how this would be practically no benefit to us), but would actively hurt customers. This is exactly the sort of breaking change we would not want to take.

For a breaking change to be worthwhile:

1. there should be good/significant benefits to us (and ideally the community) for breaking it.
2. there should not be huge downsides to to the community if it were to be taken.

There are def cases of this. But going and literally breaking things that have never been a problem, don't costs us anything (in spec or impl), and allow people the ability to use words that make sense, is absolutely not something that makes sense. it's all negative, and realistically no positive (though i get you think that it would be).

Where i'm much more amenable to a break is when avoiding the break actually leads to a more complex feature in terms of how to use it. For example, that yield requires return after it, or that await requires an async context around it. For non-conflicting identifier, who naturally slow into the language with the syntax we want people to write anyways, breaking this seems capricious as the feature and user code have no trouble coexisting.

[CyrusNajmabadi]

With hindsight, contextual keywords were a mistake

Honestly, i'm not sure there's anything that indicates that that's the case broadly. I can think of a couple of cases where having some system like what's being proposed here would allow us a slightly different route for a few identifiers. And i'm def open for them for those cases. But in the vast majority of contextual-keyword cases, going the contextual route is absolutely nicer, ensuring that the language feature nicely coexists with users using the same identifiers without any problem. Having multiple approaches be available means that we can take the 'break' approach where potentially better, but we can tke the 'contextual' where it's clearly the superior option.

[KieranDevvs]

Im probably at the extreme and fringe end of the scale with my viewpoint but, i'm all for breaking changes with very little regards to user "happiness". In my opinion changing major versions in software, you should expect breaking changes. I followed .NET framework for a long time and the mantra to "avoid breaking changes at all costs" really hindered the language. .NET Core was a breath of fresh air as it wasnt constrained by the same principles and a lot of the features we're drastically changed from its Framework co-parts (and for the better!). I can't tell you how much I hate having to maintain older .NET framework projects, and going back and doing so, really makes you appreciate how much better .NET Core / 5+ is.

I've seen a few features in .NET core / 5+ that have been introduced, where the implementation team weren't too confident on how well the design of said feature would be received, only to find that once it had been released, it was a mistake or retrospective information allowed people to see the pitfalls of said design. These features then have solidified because they're "shipped" and the API is public so any modifications would be really hard without breaking changes.

Being more liberal with breaking changes would really help in those situations where the API isn't the best and improvements after the fact, can be made.

The only thing that I would require is that breaking changes are not only documented in release notes, but expectations of what to do with the breaking changes are also provided. In the past when niche API's have been obsoleted in favour of new mechanisms or API's, the end user is given no help. In these cases, I dont think a "fixer" is enough as the fix proposed might not be what the end user is wanting to achieve. If the relative teams have time to produce a "fixer" / "analyser" for the problem, then they should have the time to write an in-depth section about why the breaking change has been made, what the expectations / recommendations are to migrate (with examples), and links to relevant resources.

[CyrusNajmabadi]

FWIW, it would be a strong goal of mine that we would only take breaking changes if we could ensure that we would be in the 'hour or two' category. Ideally, from a tooling/upgrade perspective it would be near-instant. But because there might be some changes involved, i would want the docs/info/tooling as clear as possible so that if we had to make a change, you could convince yourself it was safe and good to do in very little time.

[KieranDevvs]

I've found a great discrepancy between what people consider "large". Let's put some numbers down.

I don't know the exact metrics. The main codebase I work on on a day to day basis currently is a 190+ project solution that span a dependency tree with 2-5 root nodes. Most projects I would say at a minimum, has 5 dependencies, whether they're 3rd party (nuget or other external feeds) or projects from the same solution.

And although breaking changes might not happen currently at the language level, I can assure you, nuget dependencies and internal API's definitely get breaking changes all the time and it's not that big of a deal, at least not to me 🤔, so adding more breaking changes to the language in favor of building a better language seems like a no brainer...

[CyrusNajmabadi]

i'd say that trying to extrapolate out from a single case to the entire ecosystem (which likely has hundreds of thousands to millions of projects), isn't likely going to be representative.

Here's a way we can tell. We unintentionally have broken things over the years. After all, the compiler is complex, and we're human. And, there are also times when we make a change thinking it's safe, only to learn it wasn't. In these cases, it's amazing just how quickly we will hear from lots of people who are broken and now unhappy about things :)

and it's not that big of a deal, at least not to me

Sure. I expect there is a contingent of users that are more ok with the pain and aggravation of having to deal with breaks. I've been a developer on platforms where this was more common, and i was able to adjust. But we'll have to gauge this sort of thing with the ecosystem as a whole and balance accordingly. :)

[KieranDevvs]

i'd say that trying to extrapolate out from a single case to the entire ecosystem (which likely has hundreds of thousands to millions of projects), isn't likely going to be representative.

Obviously... But you (not you personally but the company) asked for feedback and individual's opinions on the matter and I was also asked to quantify it with numbers.

Here's a way we can tell. We unintentionally have broken things over the years. After all, the compiler is complex, and we're human. And, there are also times when we make a change thinking it's safe, only to learn it wasn't. In these cases, it's amazing just how quickly we will hear from lots of people who are broken and now unhappy about things :)

Ok, but I would also say that the loudest voices in the room aren't necessarily representative of the popular opinion.
I don't know what the popular opinion is, and I guess you don't either, seeing as you're asking for feedback.

But in any case, we agree some somethings, and not others. Lets just leave it at that, I don't want to spend my weekend trying to defend my position on this further. If people agree with what I've said, they can upvote it, if not, they can ignore it and write their own posts.

[madelson]

I really appreciate .NET's extreme reticence breaking changes. Compared to other ecosystems I work with (e.g. node/npm), .NET upgrades tend to be trivial, which is great (.NET Framework to .NET Core was the exception, but feels justifiable). We upgrade C# versions in particular very aggressively because there is little cost and lots of upside to doing so.

I think that these benefits could be maintained with a small degree of low-impact breaking changes in C#, but I wouldn't want to see a major departure from the current stance.

In the rare cases we have faced breaking changes in the .NET Runtime (e.g. removing InPlaceStringBuilder, making ResourceManager prone to deadlock) they've caused a lot of pain.

[MauNguyenVan]

I support this proposal, in order to move forward, it is necessary to accept the removal of things that are no longer relevant and hinder development, because over time complexity and obsolescence will be a burden. and gradually take up all the space for development. The language will die over time and when it is realized, there is no time to fix it due to too many outstanding problems.
Everything has its price

But make it clear, friendly, easy to switch between versions, and give error messages and documentation. Maybe like kotlin, rust... do.

[theunrepentantgeek]

I don't believe anybody is talking about removing language features.

In fact, I recall this being discussed recently - and the consensus view was that removing features would be enormously disruptive, quite expensive, and provide very little value.

[madelson]

Sorry if I missed it but is there a proposal for how code generators would work in this world? With source generators becoming more common I can see this becoming a new point of friction with upgrades because now you have to get the generator to upgrade in order to upgrade the project.

We have @ as a tool that can be used currently, but I don't tend to see this used for every single identifier in generated code today. Were that to become best practice, I think it would harm the readability of generated code and also it seems very easy to forget. Furthermore, @ only tackles changes related to keywords which is smaller than the universe of potential breaks.

I think the ideal answer would be to allow for a #pragma where you can set the language version for a file so that you can guarantee future-proofing (much like #nullable allowed you to do during the era where the nullable reference type rules were in constant flux).

Something else to consider would be any place you can write C# outside of a project, e.g. tools like LINQPad or inline snippets passed to things like RoslynCodeTaskFactory or CSharpCodeProvider. In these cases, either there may be no way to specify the language version or there is a way but in common usage it typically isn't specified (at least in my experience).

[KieranDevvs]

If the solution is to use #pragma to set the language version for a file, then I would want this behavior to be locked down to source generators only. It would make projects really confusing to have multiple language versions.

[CyrusNajmabadi]

Our position with Analyzers and Source Generators is that they need to move forward with the language as it evolves. They have access to all the information (including language version) and can emit accordingly. We will not preserve trees/semantics/etc. for them (this is already teh case for analyzers today, which already have breaking changes happen to them).

[madelson]

@CyrusNajmabadi is there an official position on other C# tools like LINQPad?

[CyrusNajmabadi]

Yes. The same would hold there.

[pebezo]

Overall I support introducing sensible breaking changes (especially if it helps clean up the code an allow it to evolve). I also have a slight preference to make the code uncompilable rather than issuing a warning that could be ignored. Unfortunately, enabling nullability on a non trivial old code base increases the likelihood that a warning is missed.

Hopefully, all these complications will not delay the field feature even more...

[luislhg]

Hello there, It's june already, .NET coming to final previews.

This issue continues to be "Unanswered".

Are there any specific concerns or feedbacks the dev team is waiting?
I have the feeling a decision must be made and there's no further discussion going on for months, either way, one direction must be taken in order for this and other issues to move on. It has been delayed for years already.

Please let us know what are you plans, or if this issue has simply being left behind.

[DavidArno]

@luislhg,

Keeping track of the team's thoughts on a topic can be hard as things get posted to different locations. Take a look at issue #7189, as there were some updates there a month or so ago. Also, take a look at the minutes from a recent LDM meeting via the discussion #7205 as that has more info.

TLDR: (my interpretation) there's no clear consensus on how to progress with this topic, so further behind the scenes work is happening and we may get an update at some point.

[Neme12]

I think there should be breaking change warnings in the new language version as well, in case the developer upgraded the language version without encountering the breaking change warnings first, e.g. by upgrading the compiler and the language version at the same time (so there's no window of time with the new compiler and the old language version to be able to present the warnings).

In the case of field, this could be a warning present when the field keyword is used in a class that also has a field named "field" (or another identifier with that name from an outer scope). The warning would say that this code would have changed meaning from an older language version.

This would be really useful in cases where the code is still valid but silently changes meaning from one version to another. I think it's really important that nobody could ever miss such a change and have a silent bug introduced by an upgrade. In cases where the code becomes an error, a warning in the new language version wouldn't be necessary as the user would already know that something is wrong and would not get a silent bug.

[dersia]

I totally support this feature and I think it is an intelligent design to face "historical" issues in the language. as for the warnings, I would love to see a new category of warnings that are outlined specifically, because I have seen way too many projects out there, where devs either didn't care or couldn't make the time (because they inherited older legacy project) to fix warnings and therefor have projects/solutions that have already multiple hundreds of warnings. in those cases they might overlook these breaking warnings and I think k it might make sense to introduce a new category maybe called "breaking warnings" or so, that the ide could highlight better.

as for the opt in why couldn't we just make <LangVerision> mandetory and adjust project files? introducing yet another flag in the project files might make things more complicated and I don't see why it couldn't be used for this case. <LangVersion>latest</LangVersion> would then also allow forward compatibility.

[madelson]

because I have seen way too many projects out there, where devs either didn't care or couldn't make the time (because they inherited older legacy project) to fix warnings and therefor have projects/solutions that have already multiple hundreds of warnings

+1 sadly many developers don't have good warning hygiene and as such warnings are not that robust of a mechanism for communication. Not sure what the alternative is, though.