XML docs are so 1999. Let's replace them with something less verbose.

[ekolis]

Ironically enough, this is one thing that Java has always done better than C# - its code comment documentation is much cleaner looking than C#'s XML docs! I don't know what to replace XML docs with but maybe look to Java for inspiration?

[sharwell]

Related to:

#878
#891
#2394

[miloush]

While "do something with the comment documentation" appears to be a popular suggestion, I would argue that something being "so 1999" is not a valid argument. An impactful change like this should be solving an actual problem (e.g. verbosity) so bad, that it is worth for the compiler to keep two separate implementations of the comment parser, practically forever. Adding a new syntax every 5 or 10 years just because it is popular at the time isn't very sustainable.

[sharwell]

@miloush I've updated my proposal to clarify that I'm proposing delegating processing to a "documentation analyzer" that can be updated similar to code analysis analyzers.

https://gist.github.com/sharwell/ab7a6ccab745c7e0a5b8662104e79735#language-and-compiler

[MgSam]

While "do something with the comment documentation" appears to be a popular suggestion, I would argue that something being "so 1999" is not a valid argument. An impactful change like this should be solving an actual problem (e.g. verbosity) so bad, that it is worth for the compiler to keep two separate implementations of the comment parser, practically forever. Adding a new syntax every 5 or 10 years just because it is popular at the time isn't very sustainable.

Keeping bad syntax around "because that's the way we've always done it" is not sustainable. If we used that reasoning, we should all be writing Fortran or Pascal. Using XML for code documentation was as bad of an idea in 1999/2000 as it is today, and vastly inferior to the documentation syntax used most other mainstream modern languages.

What is true is that the language team has shown zero interest in making any changes here. I chalk that up to no one in Redmond outside of documentation teams ever actually having to write XML docs.

[tannergooding]

Using XML for code documentation was as bad of an idea in 1999/2000 as it is today, and vastly inferior to the documentation syntax used most other mainstream modern languages.

XML is a decent final format for documentation. It is unambiguous, well-defined, fast to parse, and easily serializable. It is already supported on all major platforms, by most tooling/IDEs, and even browsers. It is fairly trivial to convert from XML to another format (such as HTML, doxygen syntax, markdown, etc) and can be compressed very well using binary XML.

One of the biggest flaws around it is verbosity. It just isn't as nice to be human written or read as something like Markdown or Doxygen comments.

Keeping bad syntax around "because that's the way we've always done it" is not sustainable

I would say, in the same vein, changing the format of documentation every 'x' years (even if only every 20 years) is equally unsustainable. Documentation comments are, for better or worse, part of the language spec and directly understood by the compiler.

What is true is that the language team has shown zero interest in making any changes here.

I don't think this is a language problem. It is instead likely a tooling problem. People don't want to read/write XML. This could be solved, without revving the language, by providing a language service API or IDE feature that displays XML comments in a well-rendered format (such as we do for IntelliSense) and only switching back to XML when editing (dotnet/roslyn#39991).

Likewise, a language service API or IDE feature could be provided that allows you to write your comments in a desired format (say Markdown) but ultimately stores it as the recognized XML format when you finish editing.

Such a path would likely be more sustainable/extensible long-term, would let developers write documentation in their "favorite" format, and would require no changes to the actual language (which would be more broadly impactful in terms of tooling, code consistency, etc).

[MgSam]

As an output, I agree, XML is fine. Honestly, I couldn't care less what the output format of the docs is. But reading/writing XML in source code is horrible. It is overly verbose to read and write. It requires escaping common characters. It is such a poor user experience that it almost actively encourages developers to not use.

I'm not a fan of using IDE solutions as a kludge around poor language features. Modern C# can be written in many different IDEs, not just Visual Studio. Trying to improve the editing experience in all of them is unlikely to happen. Not to mention, Microsoft hasn't improved the editing experience for XML docs in VS in 20 years, seems unlikely that they'd do so anytime soon.

I don't see how improving the doc syntax is any different from the many other feature evolutions that take place in C#, or any other language, over time. Anonymous delegates were almost immediately made obsolete via lambda syntax. Array type inference made array initialization inference obsolete and confusing (var a = new[] { 1, 2, 3} vs var a = {1, 2, 3}. Switch expressions may soon make switch statements largely obsolete.

[sharwell]

Such a path would likely be more sustainable/extensible long-term, would let developers write documentation in their "favorite" format, and would require no changes to the actual language (which would be more broadly impactful in terms of tooling, code consistency, etc).

It's not possible to address some of the parts of my proposal without language/compiler changes. The main issue that needs a language change is the ability to create a semantic reference from a comment within a method to some other code element such that it participates correctly with Find All References and Rename scenarios.

[tannergooding]

I don't see how improving the doc syntax is any different from the many other feature evolutions that take place in C#, or any other language, over time.

I think one of the differences is that documentation and comments in general are very stylistic in nature. Once is all said and done, documentation isn't really a language problem. You aren't trying to tell a computer how to accomplish a particular task. Instead, you are trying to explain to another developer what a given thing does and the contract it follows.

If we stick with XML some people will not like it due to verbosity. If we choose 'x', some people will complain that it isn't 'y' instead. If we choose 'z' today, people might be complaining about how it is outdated 5, 10, or 20 years from now. Even today, there are plenty of people that complain about 'x', 'y', 'z', and even 'w' (whether it is verbosity, or readability, or number of syntax variations, etc).

I also don't think we want to end up in a scenario where we have to support exposing syntax for multiple documentation variations: XML documentation and future documentation format 'x' or where we get "locked" in because 'x' was converted to XML as 'y' and now its potentially a language breaking change to modify that.

I'm not a fan of using IDE solutions as a kludge around poor language features.

The language feature itself isn't poor, it's just verbose. This is good, IMO, since you are taking generally unrestricted input and trying to map it to a well-defined parsable format. People don't like typing, even with the existing IntelliSense completion, which is the primary reason people don't like the existing format (another is it can be hard to read when you start getting into lists, tables, etc).

When people don't like typing or reading, it really becomes a tooling problem and how it can be better presented to the user. Multiple IDEs needing to support a feature is also already a problem (VS, VS Code, VS for Mac, 3rd party IDEs, etc) and one that is at least partially solved by having Roslyn as the base language service. This, in general, makes it simpler to support certain features across a range of tools.

[tannergooding]

It's not possible to address some of the parts of my proposal without language/compiler changes. The main issue that needs a language change is the ability to create a semantic reference from a comment within a method to some other code element such that it participates correctly with Find All References and Rename scenarios.

I think that is just a language service problem, not necessarily even a compiler/language problem.

We already validates and exposes relevant information for symbols within the relevant XML documentation tags. The language service could likewise (similarly to how it does with regex and json string literals now) independently validate arbitrary comments and expose additional information from them.

I don't think its a good idea for the compiler itself to try and otherwise parse out arbitrary comment text and it certainly shouldn't be spec'd in the language.

[sharwell]

The language service could likewise ... independently validate arbitrary comments and expose additional information from them.

The minimum required change to support this is:

1. Relaxing CS1587 (XML comment is not placed on a valid language element) to only report a warning if the first non-whitespace character of the comment is <
2. Updating the parser to not analyze the contents of such a comment as XML (which can result in warnings for incorrect syntax)

This provides validation, navigation, and refactoring support, but leaves us in the unpleasant situation where documentation comments can be clean and simple if and only if they aren't trying to document a code element.

I agree that the language shouldn't be providing the detailed functionality here. My proposal focuses on a shared interop format based on the current XML syntax, and an extensibility mechanism where users can provide their documentation interpretation of choice.

[tannergooding]

and an extensibility mechanism where users can provide their documentation interpretation of choice.

Could you elaborate on that bit? It sounds like we are basically proposing the same thing 😄

I think the difference here comes from your point 2, where it sounds like you are suggesting it be allowed to actually store the cs file with non-xml (hence requiring language changes) and where I am suggesting it be entirely a tooling experience and it is ultimately saved to disk as the existing XML documentation format (thereby requiring no language changes, only tooling changes).

[sharwell]

Could you elaborate on that bit? It sounds like we are basically proposing the same thing 😄

@tannergooding Yes I linked to it above in #2987 (comment)

... it sounds like you are suggesting it be allowed to actually store the cs file with non-xml ...

Yes. To me it seems unnecessary to store the C# file with XML comments since the final rendered AssemblyName.xml file is going to be the source-of-truth for consuming features. I always found the C# language specification for documentation comments to be sorely lacking (meaning it is incomplete with respect to a large number of cases, as opposed to a statement on verbosity). Rather than try and fill in that specification to address the holes, I would prefer to build a more complete specification for the shared documentation output files and allow users to work at a level more comfortable to a specific team.

[miloush]

The minimum required change to support this is

Couldn't you just use standard, non-XML comments instead?

[FiniteReality]

I might sound weird for saying this, but I honestly prefer the C# XML doc format to alternatives, such as Javadoc. It feels more rigid and less sensitive to e.g. whitespace changes and frankly, IMHO, it's actually easier to read.

As a compromise, making the summary element the "default" and supporting Markdown in place of XML in this case could work. For example:

/// This is a *markdown* summary. It contains some
/// `cool` inline text, and [links!](example.com)
/// <param name="param1">The parameter</param>
/// Text after the first valid XML element is an error. It must be wrapped in XML tags.
public void MyMethod(int param1) { throw null; }

[HaloFour]

I work with a lot of C#, Java, Scala and Python. All of which have different documentation comment formats. And for any code that has non-trivial documentation they're all about equally as awful. They're just giant walls of text with assorted different forms of markdown and quirks that you need to know to get them to be interpreted correctly. None of them are designed to be read by a human. It's a failure of the tooling that they're displayed as verbatim source rather than as something machine digested and human readable.

IMO, the writing experience between them is also about equivalent given that the IDE will autocomplete most of the markdown for you.

I'm not arguing against these features because I think that XML is better or that other formats will be worse, I just think that they're all just different shades of lipstick on the same pig. To be honest the idea of removing documentation comments from the language entirely and making it a separate analyzer-like extension sounds the best to me, although I'd want whatever supports that to be able to provide the same tooling experience as we have today, if not better.

[nanoant]

As a compromise, making the summary element the "default" and supporting Markdown in place of XML in this case could work. For example:

Couldn't agree more on that. And this is my biggest problem reported in #2394 that C# code is drowning with <summary>-ies, which is depending on your style consumes 2 lines, without much added value.

We can get into endless discussion about next fine documentation syntax, and bounce the ball between those who hate XML doc and these who don't, but I dare to say there is none that can defend need for <summary>. I have once seen some compilation of the most common (key)words in C# and guess which word was the most popular?

How about removing necessity for <summary> ASAP, before we continue any other activities in XML doc area, so everything that is not wrapped in some top level tag becomes summary by default? Seriously, this would transform XML doc into something I (and probably many others) can live with.

Btw. even .NET runtime itself does not always use XML doc, see:

• https://github.com/dotnet/runtime/blob/master/src/libraries/System.Collections/src/System/Collections/BitArray.cs
• https://github.com/dotnet/runtime/blob/master/src/libraries/System.Collections/src/System/Collections/Generic/Queue.cs

I'd love to hear why is it so.

[branko-d]

For what it's worth, Resharper has a nice renderer for XML docs.

[MgSam]

Just pinging this issue as the feedback from the language team at Build was they need to see sustained interest in issues over time for it to ever get addressed.

To be clear, I'd be happy with doing anything here. XML is horrible to both write and read for humans. I asked ChatGPT to rank the top 15 programming languages based around the verbosity of the documentation syntax and C#/F# were at the very bottom as no other language uses XML. I feel like 30% of my coding time is spent moving cursors between <summary> and <param> tags in XML docs :)

[CyrusNajmabadi]

As a starting point, there would need to be a well vetted library for working with markdown in a reasonably standardized fashion, which could sit on a text-model that is not "give me big fat string to parse" :)

[nanoant]

Yes this would be the right direction that is consistent with wide use of Markdown in other places of Microsoft products. We moved to Markdown with LaTeX for equations and it makes our hard anyway work so much better.

[The-Futurist]

Perhaps too, one could enhance Visual Studio so that XML comments get rendered inline as a small HTML panel with styling and all that as well. Today we can hover over a collapsed XML comment and see:

So displaying a more readable HTML panel as the norm, could make comments easier to perceive and read as we work on code.

Maybe something akin to a code-lens could be added:

So the comments could still be "in" XML but rendered more intelligently and attractively, this could be done for ordinary comments too...

[HaloFour]

That is how IntelliJ renders documentation comments:

You can click on type/member links to navigate also.

[CyrusNajmabadi]

You'll get this experience if you hover over the symbol itself (not the collapsed region). We can definitely look into having the collapsed region show the same rendered view.

[nanoant]

So the comments could still be "in" XML but rendered more intelligently and attractively, this could be done for ordinary comments too...

No, please, let's not go this way putting the makeup (markup) on XML pig. This does not solve the problem, the annoying chore of writing documentation using obscure eye piercing XML syntax remains, unless you throw sophisticated Work-like editor on top of it.

But why?
Please acknowledge Markdown that is almost plain text has proven to increase productivity by having engineers spending their time and energy writing actual algorithms, not markup.

I don't understand why Microsoft embraces simple Markdown everywhere but not here. Is there a guy somewhere who invented this syntax and does not want to let go!?
Look, XML docs were good idea and necessary idea to pave a road to even better solutions Markdown. Can you please now let XML docs go?

I can only be happy that I have less and less to do with C# code, working mostly with C++, Python and Markdown these days. But I still have great empathy for my colleagues that need to maintain C# codebase. And C# is a great language, except... you know what.

[CyrusNajmabadi]

@nanoant moving to markdown is non-trivial. Especially as the tooling required for it. The last time I checked too I could find no good libraries that would fit in Roslyn's concrete Syntax tree world, and which could sit on top of a string abstraction like roslyn does. So this is just a staggeringly large work item that would likely take several devs a very long time to implement. As such, this simply has not proven its value compared to its cost.

It's easy to just say "why had no one done this?" when costs are not considered.

[HaloFour]

IMO the documentation comments themselves should be presentation agnostic, intended to be consumed by tools in order to produce documentation, which could be in HTML, markdown, PDF, or whatever. It's easier to do that starting with a format like XML, which is predictable and easy to parse using tons of off-the-shelf tools for any number of languages.

I also think that the experience in the IDE around documentation comments leaves a lot to be desired. I think if the comments were rendered by the IDE into something nice, with options as to how it appears (e.g. inline, adornment with a tooltip, etc.), a lot of the issues would go away. The IDE could also make the authoring experience nicer by providing a UX, filled out with the structure of what is being documented so you only need to add in the text, or allowing manual editing of the XML directly.

Sure, this doesn't fix what the comments look like in GitHub or PRs, but to be honest I don't find that experience pleasant regardless of the format of the comments.

I would say that if all you typically write are summaries, as I know a lot of people do, a shorthand syntax for that might be nice.

[wrexbe]

Maybe even keep them in a separate file, that looks like a commented interface, and have the IDE pretend they are above the implementation. The majority of the time, if I am working on a file, I don't want to see the comments.

[nanoant]

IMO the documentation comments themselves should be presentation agnostic, intended to be consumed by tools in order to produce documentation, which could be in HTML, markdown, PDF, or whatever. It's easier to do that starting with a format like XML, which is predictable and easy to parse using tons of off-the-shelf tools for any number of languages.

If I substitute /documentation comments/ with /source code statements/, do you also believe that source-code should be written using presentation agnostic CIL instead high-level C#? If not, what makes documentation special? Also in your opinion should developers serve tools or tools (computers) serve developer (humans)?

@nanoant moving to markdown is non-trivial. Especially as the tooling required for it. The last time I checked too I could find no good libraries that would fit in Roslyn's concrete Syntax tree world, and which could sit on top of a string abstraction like roslyn does. So this is just a staggeringly large work item that would likely take several devs a very long time to implement. As such, this simply has not proven its value compared to its cost.

I believe you, but this should be presented as a true and honest reason that XMLdoc is here to stay, rather than defending XMLdoc as still superior to plain-text approaches taken by modern competition to C#. I dare to say in the long term XMLdoc may turn C# into fossil language, paving a way for Rust and Python dominance. I just hope someone there at Microsoft or some external contributor will eventually find an elegant way to inject more humane C# comment syntax, e.g. via some smart transform at the very beginning of AST creation (lexer stage), transforming Markdown back into XMLdoc, so the tooling coming afterwards (parser) remains unaware of the change.

[CyrusNajmabadi]

I just hope someone there at Microsoft or some external contributor

Feel free to contribute :-)

[HaloFour]

If not, what makes documentation special?

That documentation generally extracted and formatted into a separate document, in one of many possible different formats.

I dare to say in the long term XMLdoc may turn C# into fossil language, paving a way for Rust and Python dominance

This isn't a popularity contest, and even if it were, I'd wager there are substantially more important features that would provide a more competitive edge than documentation comments.

[DavidArno]

My thoughts here are that the title has not stood the test of time. Sure, XML docs are "so 1999". But I also feel that doc comments themselves are "so 2022". These days, the smart money ought to be on scrapping the clutter of using comments for docs and instead leaving it up to an AI like ChatGPT to generate these simple, formulaic docs from the code itself. So, the latter part of the title maybe should be more "Let's retire them gracefully as they have served their time".

It feels like it would be a huge waste of resources to spend time changing the current XML-based doc comments to a new form at a time when we must be questioning the very purpose of such comments.

[HaloFour]

These days, the smart money ought to be on scrapping the clutter of using comments for docs and instead leaving it up to an AI like ChatGPT to generate these simple, formulaic docs from the code itself.

You have a much more optimistic view of the state of GPT-like LLMs than I do. 😄

Even if those tools become more sophisticated I would still expect a human to provide input in the form of context as to why the code does what it does. If the comments just spit out a summary of the algorithm they're already pretty pointless.

[DavidArno]

You have a much more optimistic view of the quality of developer created doc comments than I do. In my experience, 99% of those comments blindly parrot the code. ChatGPT is more than capable of matching that.

I would still expect a human to provide input in the form of context as to why the code does what it does...

Absolutely. And much of MS's own learn.microsoft.com site shows the way forward with this: the content is high quality and - importantly - written in doc files, not doc comments. We're a very long way off an AI matching the documentation skills of the likes of Bill Wagner. But AIs have already passed the quality of the overwhelming majority of doc comments. The rest tend to then be so detailed, we really don't want them in the code as they make it hard to find that code amongst reams of documentation.

• Simple tooltip info - leave it to an AI to generate
• Good quality docs that describe the why, the side effects, list examples etc - put in separate doc files
• Doc comments - gracefully retire

[theunrepentantgeek]

instead leaving it up to an AI like ChatGPT to generate these simple, formulaic docs from the code itself.

I about 70% agree with you.

Simple, formulaic docs are useless.

What I want from docs is the stuff that you can't discern from a simple reading of the declaration and nearby context.

This is useless, and I understand why some would blame the syntax.

/// <summary>
/// The name.
/// </summary>
public string Name { get; set; }

But this is very useful indeed:

/// <summary>
/// The unique human readable name of the course, as shown to end users.
/// </summary>
public string Name { get; set; }

An 8k token buffer is woefully inadequate for any LLM even attempting to write these kinds of comments.

[nanoant]

That documentation generally extracted and formatted into a separate document, in one of many possible different formats.

The same thing applies to the regular source code. It is being "extracted" by the linker from individual source code files and put into potentially many different final binaries. Still I don't see anything fundamentally different here, that makes syntactic sugar => replacing more verbose get/set {...} embraced by the maintainers, who are opposing in the same time the idea of dropping required tag <summary> polluting the source code and consuming screen real estate for no good reason.

This isn't a popularity contest, and even if it were, I'd wager there are substantially more important features that would provide a more competitive edge than documentation comments.

No it isn't. Neither beauty contest. It is productivity contest. At least in commercial/business environment what matters is how much effort it takes to build solid, bug-free, future-proof solution, given that some (most?) of the code is still written by humans not AI, and you need to make sure that the people's motivation is not taken away by forcing them to write pointless tokens such as <summary> everywhere. There are good reasons why XHTML failed while Markdown is now used everywhere.

Feel free to contribute :-)

I will and I do, but not to Roslyn. I am barely using C# for new developments. The only reason I am "trolling" here is that I still have quite a few colleagues still working on C# code-base and I am occasionally reviewing their code and see them struggling / unmotivated to write reasonable comments such as in @theunrepentantgeek's example. And I am pretty much convinced this is because of unnatural and annoying syntax of XML doc, because I don't see such problems in other language's codebases. Otherwise C# is pretty neat language for general purpose applications. But this is just my opinion.

Anyways, I think I am starting repeating myself. I wish Roslyn team good luck, because I believe it is good for everyone to have competing technologies. Radio out. Cheers.

[HaloFour]

The same thing applies to the regular source code. It is being "extracted" by the linker from individual source code files and put into potentially many different final binaries.

By one compiler, into one build artifact. The analogy doesn't hold up because nobody is expecting anybody to try to build their own tools to try to take C# source and turn them into a completely different output format. The level of effort required to do so is exorbitant. But to take a format like XML, which has tons of off-the-shelf support in every language, and use it to translate into a different format, is much, much lower effort.

There are good reasons why XHTML failed while Markdown is now used everywhere.

XHTML failed because of HTML 5, but that has nothing to do with documentation comments or even Markdown. HTML is still used vastly more. None of them are well-suited for documentation comments.

you need to make sure that the people's motivation is not taken away by forcing them to write pointless tokens such as

everywhere.

That's what IDEs are for. It's not like you're manually typing all of those keystrokes. Type /// and the skeleton is automatically generated for you based on the signature. All the developer needs to do is fill in the blanks.

The only reason I am "trolling" here is that I still have quite a few colleagues still working on C# code-base and I am occasionally reviewing their code and see them struggling / unmotivated to write reasonable comments such as in @theunrepentantgeek's example. And I am pretty much convinced this is because of unnatural and annoying syntax of XML doc, because I don't see such problems in other language's codebases.

XML has nothing to do with that. Bad documentation comments are common in every language, just as bad source comments are common in every language.

My experience is that C# libraries are generally much better documented than their Java counterparts, despite Javadocs being a lot less verbose.

[gewarren]

The rest tend to then be so detailed, we really don't want them in the code as they make it hard to find that code amongst reams of documentation.

In Visual Studio, you can collapse all /// comments using Ctrl+M+O. In VS Code you can press Ctrl+K+0.

Also, here's an example of how you can keep documentation comments in a separate file using the <include> tag.