a draft blog post about error message style guides

Author: cfbolz

posts/2021/12/error-message-style-guides.txt

@@ -0,0 +1,100 @@
+.. title: Error Message Style Guides of Various Languages
+.. slug: error-message-style-guides
+.. date: 2021-12-10 16:00:00 UTC
+.. tags:
+.. category: 
+.. link: 
+.. description: 
+.. type: rest
+.. author: Carl Friedrich Bolz-Tereick
+
+================================================
+Error Message Style Guides of Various Languages
+================================================
+
+PyPy has been trying to produce good `SyntaxErrors`_ and `other errors`_ for a
+long time, and thankfully CPython has made an enormous push to `improve its
+errors in the last releases`_. These improvements are great but the process
+feels somewhat arbitrary sometimes. To see what other languages are doing I
+asked people on Twitter whether they know of error message style guides for
+other programming languages:
+
+.. raw:: html
+
+    <blockquote class="twitter-tweet"><p lang="en" dir="ltr">Is there any programming language that has a style guide for its error messages? <a href="https://twitter.com/ShriramKMurthi?ref_src=twsrc%5Etfw">@ShriramKMurthi</a>, do I remember correctly that Pyret has one?</p>&mdash; Carl Friedrich Bolz-Tereick (@cfbolz) <a href="https://twitter.com/cfbolz/status/1466033151315173384?ref_src=twsrc%5Etfw">December 1, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> 
+
+Wonderfully, people answered me with lots of helpful links (full list at the
+end of the post), thank you everybody! All those sources are very interesting
+and contain many great points, I recommend reading them directly! In the
+following I'm, going to try to summarize some common themes or topics that I found
+particularly interesting.
+
+Several guides point out a 80/20 rule: 80% of the time an error message is
+read, the developers knows that message well and knows exactly what to do. For
+this use case it's important that the message is short. On the other end of the
+spectrum, 20% of the time a developer gets a message that they have never seen
+before and are confused and so the message needs to contain enough information
+to allow them to find out what is going on [Flix, Flow].
+
+Almost all guides stress the need for plain and simple English, as well as
+conciseness and clarity [Flix, Racket, Rust, Flow]. Flow suggests to put coding
+effort into making the grammar correct, for example in the case of plurals or
+to distinguish between "a" and "an".
+
+The suggested tone should be friendly and neutral, the messages should not
+blame the Programmer [Flow]. Rust and Flix suggest to not use the term
+'illegal' and use something like 'invalid' instead.
+
+Flow suggests to avoid "compiler speak". For example terms like 'token' and
+'identifier' should be avoided and terms that are more familiar to programmers
+be used (eg "name" is better). The Racket guide goes further and has a list of
+allowed technical terms and some prohibited terms.
+
+The Racket guide proposes to use the following general structure for errors:
+'State the constraint that was violated ("expected a"), followed by what was
+found instead.'
+
+The Rust guides says to avoid "Did you mean?" and questions in general, and
+wants the compiler to instead be explicit why something was suggested. The
+example the Rust guide gives is: '"Compare "did you mean: Foo" vs. "there is a
+struct with a similar name: Foo"."'. Racket goes further and forbids
+suggestions altogether because "Students will follow well‐meaning‐but‐wrong
+advice uncritically, if only because they have no reason to doubt the
+authoritative voice of the tool."
+
+The Rust guide suggests to put all identifiers into backticks, the Flow goes
+further and uses Markdown for the error message.
+
+The Clang, Flow and Rust guides point out the importance of using precise
+source code spans to point to errors, which is especially important if the
+compiler information is used in the context of an IDE to show a red squiggly
+underline. The spans should be as small as possible to point out the source of
+the error [Flow].
+
+I don't really have a conclusion! I wonder whether it would makes sense for
+Python to adopt a (probably minimal, to get started) subset of some of these
+ideas as guidelines for its own errors.
+
+
+Sources:
+
+* Rust: https://rustc-dev-guide.rust-lang.org/diagnostics.html
+
+* Clang: https://clang.llvm.org/diagnostics.html
+
+* Flix: https://flix.dev/principles/
+
+* Racket: https://cs.brown.edu/~kfisler/Misc/error-msg-guidelines-racket-studlangs.pdf
+
+* More about the research that lead to the Racket guidelines (including the referenced papers): https://twitter.com/ShriramKMurthi/status/1451688982761381892
+
+* Flow: https://calebmer.com/2019/07/01/writing-good-compiler-error-messages.html
+
+* Elm: https://elm-lang.org/news/compiler-errors-for-humans
+
+* Elm's error message catalog: https://github.com/elm/error-message-catalog
+
+* Reason: https://reasonml.github.io/blog/2017/08/25/way-nicer-error-messages.html
+
+
+