C# LDC / Community Process

[MgSam]

This issue is for community suggestions on how to improve the open source / community participation in the C# language design process. I'll update the head post with any other suggestions the community thinks make sense.

Problems

1. Language Design Meetings (LDM) happen multiple times a week, but design notes from the meetings are typically not posted until months after the fact. This means the community sometimes finds out about decisions after they've already been finalized, minimizing the ability to give any meaningful feedback.

2. Most Language Design Committee members (LDC) don't monitor or participate in Github discussions with any regularity. This results in a number of knock-on problems:

   a) Sometimes this results in LDM that retread points and counterpoints that the community discussed at length and insight from knowledgeable community members can get lost in all the noise.

   b) It also means that the vast majority of impactful design discussions happen offline in Microsoft offices or in Microsoft email. This goes against the whole idea of the language being open source.

   c) Finally, it means sometimes features are finalized and shipped or nearly shipped and only at that late stage of the game the LDC discovers the community hates the feature. This happened with primary constructors and may be the case for the in keyword. Getting that feedback earlier in the game will make the features better and make everyone happier.

Suggestions

1. As the head of the C# team, it seems unlikely @MadsTorgersen can or should be spending his time writing up design notes. Please either designate or hire a junior member of the team who has the responsibility of writing up and publishing the notes in a timely fashion after the meetings.

2. Either a) migrate the language design team to actually use Github for language design discussions or b) designate a single point of contact that monitors Github issues and raises relevant suggestions / points to the LDC.

What are other Microsoft teams doing

• The .NET team posts videos of their API reviews in all their gory detail
• The .NET team interacts extensively with the community on Github discussing and prioritizing issues (feature requests, bugs, etc) as they're posted.
• The ASP.NET posts regular video community standups
• TypeScript keeps an up-to-date roadmap of what the language team is working on and what is next. Crucially, the team is actually using the roadmap internally so it is always current.
• TypeScript posts up-to-date design notes
• VS Code provides monthly iteration plans

Please help in refining these or provide alternative suggestions and skip any finger-pointing or name calling. We should try to provide actionable suggestions in the hope that we can move the needle.

[tannergooding]

FYI. @MadsTorgersen, @jaredpar

[iam3yal]

I've writted an issue about it but I'll post it here.

Releasing design notes to the community needs to be part of the development cycle

Getting the design notes has been a "pain point" for the community for few years now and we really want to understand why it's not baked in to the development process? why do we need to wait for indefinite amount of time before we get it? and when we get it it's really too late in the process for us to react or say anything that can have some impact.

Now, when we look at the development process for other languages such as TypeScript which is developed by Microsoft team, at Microsoft, we wonder why the process is very different? why should it be any different?

We asked for this to change multiple times but as an outsider looking at the process nothing really changed and what I mean by that is the compiler platform is open source, we have a design repo but really the design of the language is still closed source in terms of transparency, now, I'm not going to pretend I understand what it takes to change things but as complex as things are I'm sure you can change it and we're asking you to do it because we want to know, we want to see where you guys are taking the language, we want to be part of this journey but in order to do it, you guys need to let us be part of this, at least a small part of it, don't leave us in the dark.

The TypeScript team are really good at tracking what features are going to be released and in which version and whether it was completed through this document, additionally, the design notes are shared and released regularly as you can see here.

Currently, the only documents that track the status of features for C# that I'm aware of are this document at the Roslyn repo and this document which exists in this repo but the issue is that they only track what was implemented in the compiler and what was released as opposed to actual progress and where things stand.

So I really hope you guys are working on improving the process for us and that one day (soon™) we're going to see the fruits aka design notes released regularly.

[iam3yal]

@MgSam You can add VSCode Iteration-Plans.

[4creators]

IMHO we reached a point where to some extent community got alienated by C# team design decisions or lack of decisions and lack of communication.

The problem is that C# design process has became orthogonal to open source process of .NET Core development to the extent that it does not serve very well .NET devs and .NET users. Expectations of the C# team and .NET devs and users started to diverge significantly and it is a result of the lack of communication between community and core C# team members.

My understanding is that the reason for using this very GitHub repo for work on C# development was to enable common workflow for C# team and C# open source community. As it seems until now nothing like this has ever happened. For example see const parameters feature proposal - based on Issue #744 and Design Review: Hardware Intrinsics for Intel.

There is a large effort in coreclr repo aiming at implementing hardware intrisincs comprising SIMD and other ISA extensions which due to limitations of C# and VB, F# need support of simple syntactic feature - enforcing passing of compile time constant parameters. It can be implemented easily as either analyzer enforced syntax feature (which is IMO bad design) or a language syntax feature where method parameter could have const modifier.

The implementation effort so far is demonstrated by following linked PR's (this can be incomplete list) but community members implementing this feature cannot get simple decision on side of C# team by rising issue in this repo or having it discussed during API Review meeting. Only action taken in Roslyn repo (see When merge window for v15.5 will close? (const parameters)) resulted in @jcouv adding this feature to LDM backlog Compiler: Needs LDM attention and later informing that brief LDM discussion on Oct 31, 2017 concluded the feature can be supported by analyzers. Lack of any further information, lack of any action in this repo regarding #886, lack of LDM design notes for LDM meeting on Oct 31 resulted in attempts to create workarounds for missing C# lang support.

My hope is that community support for implementing of new and IMO very important coreclr features will meet in future with adequate attention from C# language team.

dotnet/coreclr#15341
dotnet/coreclr#15014
dotnet/coreclr#14857
dotnet/coreclr#14861
dotnet/coreclr#14853
dotnet/coreclr#14825
dotnet/coreclr#14809
dotnet/coreclr#14820
dotnet/coreclr#14808
dotnet/coreclr#14796
dotnet/coreclr#14791
dotnet/coreclr#14792
dotnet/coreclr#14779
dotnet/coreclr#14781
dotnet/coreclr#14736
https://github.com/dotnet/corefx/issues/24343
dotnet/coreclr#14730
dotnet/coreclr#14678
dotnet/coreclr#14627
dotnet/coreclr#14628
dotnet/coreclr#14629
dotnet/coreclr#14633
dotnet/coreclr#14632
dotnet/coreclr#14631
dotnet/coreclr#14630
dotnet/coreclr#14634
dotnet/coreclr#14635
dotnet/coreclr#14636
dotnet/coreclr#14637
dotnet/coreclr#14576
dotnet/coreclr#14570
dotnet/coreclr#14572
dotnet/coreclr#14456
dotnet/coreclr#14307
dotnet/corefx#24490
dotnet/corefx#24415
dotnet/coreclr#14225
dotnet/coreclr#14164
dotnet/coreclr#14132
dotnet/coreclr#14020
dotnet/coreclr#13815
dotnet/coreclr#13576

[4creators]

fyi @terrajobst @jaredpar @jcouv @fiigii @sdmaclea

[gafter]

@4creators coreclr features that are exposed to the user as new APIs or runtime options do not generally benefit from language support or attention from the C# language team. The analyzer feature was designed specifically so that API authors can ship an analyzer reference along with the API so that users of the API get all the necessary checking at use sites specific to the API, for example, that some argument expression is given as a compile-time constant. A single analyzer can enforce that restriction for any number of APIs, for example by using an annotation on the restricted parameters.

API authors do not need to wait for the language design team in order to design, implement, and ship such an analyzer. It could have been done years ago. That is not part of the usual design flow for coreclr feature designers, but it is a technique that they should add to their repertoire. If you're unsure where to start, please let us know.

[CyrusNajmabadi]

API authors do not need to wait for the language design team in order to design, implement, and ship such an analyzer.

Indeed. And that's really important. If all these CLR features are gated on getting language traction, then that's going to slow things down enormously. The language has to move at a slow and steady rate. Much caution has to be taken that people won't break, and there are large hurdles to cross to make something into a language feature (as one can see just watching the progress of current language features).

Analyzers allow platform improvements to progress fast, while being decoupled from depending on the actual language changing. This decoupling removes this choke point that would otherwise slow things down greatly.

Then, once a feature has been made available, and has gotten enough adoption, and would really benefit from language changes, work can be done on designing and implementing such a feature.

[iam3yal]

Can we focus on the real issue please? which is how to improve or change the current process so the community would feel they are part of the process and not an outsider scavenge for information.

Thanks.

[HaloFour]

For me it mostly comes down to the timely release of design meeting notes. It so often feels like we're very much not on the same page as the team and have no lead time to digest or comment on a direction explored well over a month ago.

[rikimaru0345]

To add to what has been said here, I'd like to mention the way new features are selected.

Giving feedback on stuff that is being worked on is one thing, but personally I feel there's a bigger issue: what features get selected at all for a new language version.

Sometimes (not always) it feels like the focus on what to implement next is set somewhat arbitrary (mostly based on what the people on the team would like to have or find interesting).

I am sure people who use the language in production every day would prioritize things (slightly) differently than how it is done at the moment.

So my suggestion would be to have some kind of poll that is controlled by the language team.
They would select what goes into the poll (to throw out features that have no merit),
then everyone could vote on what they'd like to see in the language next.

Of course it is not always reasonable to go with the highest voted thing, but at least then there'd be a a centralized location where the language team see what the people actually want.

What we have now - allowing people to post their ideas as github issues - is a pretty good step in the right direction, but it should be expanded.

[iam3yal]

@rikimaru0345 I would really like to see improvements in the following areas:

1. The current/future status of features being developed.

2. When a feature is pushed to the next version or rejected then "why?".

3. Last but not least I would like to see the design notes as early as possible.

However, I disagree with a voting like system, I really think that the design team should decide on what features gets in and have final say on that and I suspect that some people will get disappointed when others won't favour their feature or something similar and this will only add to their frustration.

[gafter]

The current status of features under development is maintained here: https://github.com/dotnet/roslyn/blob/master/docs/Language%20Feature%20Status.md

The language features under consideration for different releases can be seen by filtering this list by different milestones (here filtered to C# 7.3 candidates): https://github.com/dotnet/csharplang/issues?q=is%3Aopen+is%3Aissue+label%3A%22Proposal+champion%22+milestone%3A%227.3+candidate%22 . There are active efforts (some just ramping up) on

• Champion: relax ordering constraints around ref and partial modifiers on type declarations #946 "Relax ordering constraints around ref and partial modifiers on type declarations" (possibly will be treated as a 7.2 bug fix)
• Champion: by value vs in overload tiebreaker (15.6, as a 7.2 bug fix) #945 "by value vs in overload tiebreaker" (ditto)
• Champion: ref local reassignment (15.7) #933 "ref local reassignment"
• Champion "Support for == and != on tuple types" (15.7) #190 "Support for == and != on tuple types"
• Champion "Auto-Implemented Property Field-Targeted Attributes" (15.7) #42 "Auto-Implemented Property Field-Targeted Attributes" (already completed)
• Champion "Expression variables in initializers" (15.7) #32 "Expression variables in initializers" (scheduled for LDM discussion)

Also there are a number of features under active development for C# 8:

• Champion "default interface methods" (16.3, Core 3) #52 "default interface methods"
• Champion "Pattern Matching" (16.3, Core 3) #45 "Pattern Matching"
• Champion "Nullable reference types" (16.3, Core 3) #36 "Nullable reference types"

Features are generally pushed to a later release because they haven't been done. When we explicitly reject proposals, we add comments about why.

For language features that you'd like included sooner rather than later, your best bet is to volunteer implementing it.

[alrz]

For language features that you'd like included sooner rather than later, your best bet is to volunteer implementing it.

I presume you mean features that are already championed. and then #384 (comment)

It seems like most language design proposals don't finalize on a design until the compiler team is ready to have someone actively work on or implement it.

[iam3yal]

@gafter Yes, fair enough although I already covered this in first post, the document you posted tracks features in terms of implementation in the compiler whereas I was speaking about the current/future status of features in term of design and engagement of the team with the community about it, finally, the lack of design notes and sometimes the late design notes that we get turn into a frustration on our side.

[gulshan]

Some categorization of issues like mentioned in dotnet/vblang#8 and by myself in #726 will also help to engage the community in a structured way in my opinion.

[Pzixel]

@gafter you say

#42 "Auto-Implemented Property Field-Targeted Attributes" (already completed)

But its status is just Discussed in LDM. And you are saying that it's already completed. Thus it seems that documentation do not correspond to actual feature state.

[yaakov-h]

@gafter There's still a lot of information that we don't have.

For example, take blittable. I don't have any particular attachment to it, it's just an example.

It appears in the Language Feature Status roadmap for C# 7.2, which shipped earlier today. It is not in C# 7.2.

The issue for it was moved from 7.2 candidate to 7.x candidate in early October, then a week later to 7.3 candidate, without any sort of clear roadmap or explanation.

Is it actually a 7.3 candidate, or is it likely to slip to 7.4 or 8.0 or 9.13 or whatever?

[gafter]

For example, take blittable. I don't have any particular attachment to it, it's just an example.

The issue for it was moved from 7.2 candidate to 7.x candidate in early October, then a week later to 7.3 candidate, without any sort of clear roadmap or explanation.

The explanation was that it didn't get done for 7.2, but it is still something we're interested in. It went into the 7.x bucket (because it could happen in any point release, and it is in 7.x rather than x.x because we prefer sooner rather than later). It was then triaged into the 7.3 release by the LDM, which means it will go into 7.3 if it gets done.

Is it actually a 7.3 candidate, or is it likely to slip to 7.4 or 8.0 or 9.13 or whatever?

I can't say how likely it is to slip, but it is actually a 7.3 candidate.

[MadsTorgersen]

I agree that we could have better communication, and at the core of this is the delay of good design notes getting posted. Currently this rests on me, and I have too many things to juggle to have been able to keep up on a regular basis. I feel bad about the current state of things, and I apologize, but this is currently limited by my capacity. I am working on getting a dedicated person to take it over, but I don't have an ETA on when that can happen. In the meantime I'll try to work on hacks to see that I can get more information out sooner.

Most of what is asked for here would flow from good, early design notes. While we do keep the milestones updated, as Neal mentions, the rationale for changes is captured in design notes. So I completely agree that this is the core issue to address.

To a few of the concerns about how we decide what features to focus on: it is not arbitrary. We study the repo, but we also talk to the compiler and IDE implementers, the framework teams, runtime teams, other teams, lots and lots of industry developers, as well as designers and users of other languages, researchers, managers, MVPs, etc., etc. We look at many sources for problems and solutions, and based on that we make experiments, prototypes and ultimately judgment calls on what is good to do in C#, and can be done well in a way we understand how to do.

Thanks,

Mads (hoping for some quiet days over the holidays to reduce the pile of pending design notes)

[HaloFour]

@MadsTorgersen

I was just typing out that while we love your well curated design notes it's clear that you don't have the time to reorganize and format them, and then you went and merged a bunch of raw notes. I think that's great. It gets the conversation started, anyway.

Going forward I think it's clear that this can't fall on you. You've simply got too much on your plate. I'd say that the proposal champions should probably own it but they're probably in a similar boat. But perhaps they can at least direct that effort through other members of the team. I'd have to assume that these decisions are getting persisted in some manner somewhere, at least towards the end, to help coordinate efforts between the compiler and tooling. It'd be nice if Github were that source of truth.

[iam3yal]

@MadsTorgersen

I am working on getting a dedicated person to take it over, but I don't have an ETA on when that can happen. In the meantime I'll try to work on hacks to see that I can get more information out sooner.

Can you tell us at least whether this will happen after the next major release or before that? any kind of estimation would be better than none, this way we can create a new issue about it without nagging you about it for the time being.

[orthoxerox]

Any updates? Has a dedicated person been found?

[YairHalberstadt]

Would the language design committee members feel uncomfortable with having the meetings recorded whether by video, or simply a voice recorder?

As someone who's fascinated with language design, I would absolutely love to watch/listen to the meetings. It also would give a much more thorough overview of the issues involved than the notes.

On the other hand I fully understand if the LDC want to be able to talk freely, without knowing everything they say is being recorded and scrutinized, or simply value their privacy

[theunrepentantgeek]

Would the language design committee members feel uncomfortable with having the meetings recorded whether by video, or simply a voice recorder?

I suspect this is extremely unlikely to happen. The notes that come out of the LDM quite deliberately summarize the conclusions drawn without any individual attribution (except in very very rare cases).

[YairHalberstadt]

https://sound.stackexchange.com/questions/37821/forensic-proof-voice-changer😉