Update approach to using Unicode in the docs

[xsawyerx]

[Originally requested by mistake under raku repo.]

Problem or new feature

Raku’s ability to use Unicode throughout the language is impressive - very impressive, but many examples in the documentation rely on characters that cannot be typed on most (or in some cases - all) standard keyboards. This makes those examples difficult to reproduce and makes learning Rakudo more difficult. Examples in which I either need to know the Unicode names, the Unicode codepoints, abandon that idea and copy/paste these characters, or lastly, learning to translate what the author wrote (for the sake of novelty) into what I actually will be typing, are unhelpful.

In many places, the primary purpose of these symbols appears to be demonstrating that Raku can interpret arbitrary Unicode in code. It's cool, sure, but it is rarely (if ever) relevant to users trying to understand a language construct.

FWIW, this was one of the reasons I steered away from Rakudo. I had to understand what each character in each example actually meant. ("Is it 《 or is it ⟪ or is it «, and do I use "" or '' or << or (( to replicate it? Why is it part of a say statement?") It was frustrating that every example was trying to show off something unrelated to the topic at hand, and I couldn't see myself writing it.

Partially driving the point home and partially letting out some more steam: if Raku supported syntax written in non-printable control characters or even audio input, those would be fascinating capabilities worth documenting. But they would not belong in everyday examples meant to teach loops, classes, or basic operators.

Suggestions

To improve accessibility and clarity, I suggest:

1. Removing Unicode or novelty characters from examples unless they have a relevant value.
2. Replacing such examples in the rest of the documentation with ASCII-friendly versions that users can type directly or copy/paste cleanly, so users focus on what's being explained.
3. Making sure the Unicode section encompasses all relevant examples that showcase this ability.

That would let Rakudo's main documentation be practical and approachable, and I could stop complaining about it. :)

[lizmat]

In the Rakudo version of the issue it was suggested to have a toggle that would allow one to switch between ASCII and Unicode version of identifiers.

I think this could be implemented in a JS frontend thing with a conversion map, assuming the generated docs would only contain ASCII versions?

By delaying this until the actual rendering, we wouldn't have to worry about:

• adding some logic in rakudoc for handling this
• fixing the renderer to handle that additional logic

Pinging @finanalyst @thoughtstream about this idea.

[librasteve]

I have a few observations. TLDR; I agree.

Firstly

I have been on this journey and ended up at the same place. In the Physics::Measure README.md, I recently wrote:

Note: The caret prefix ^ can now used as an alternative to the libra prefix ♎️ to ease typing. Also the tilde ~ has been added as an alternative to ± to introduce an Error term.

Secondly

Generally, I am against gadgets on our websites. UX is a very hard thing to do well (especially on the web) and we will already have globals controls for Light/Dark and for Lang [EN, FR] (and actually separate ones for interface lang and content lang are currently envisioned... hmmm). It is a very common notion to reach for a light/dark, but it's not a natural UX thing to reach for a dropdown to toggle unicode/non-unicode.

Thirdly

Really, for core documentation principles, I think we need to decide them and then stick to them. An additional negative of a gadget is that it allows us to sit on the fence, where really we need to show the language in its best, simplest light.

Summary

So, I hope that we can achieve strong consensus on this proposal and then (if that's the way it rolls) have a simple mechanism (eg tooltips?) for those who want to opt in to unicode to see that alternative in context - and a unicode glossary addendum to that section of the docs.

PS. I realize that this may be a feature on the new raku.org site, for example:

method area { π * $.radius² }

Please let me know if you think we should fix those instances!

[xsawyerx]

I'd like to echo practically everything @librasteve says. To highlight specifics:

UX is a very hard thing to do well (especially on the web)

No need to add complexity unless it adds value.

Really, for core documentation principles, I think we need to decide them and then stick to them.

Simplicity, consistency.

[...] we need to show the language in its best, simplest light.

💯 %

[...] for those who want to opt in to unicode to see that alternative in context [...]

I think that if this Unicode character as syntax capability was critical to where it's used, it adds benefit. And if doesn't add specific benefit where it's displayed, there's no point in having it there. E.g., if I'm reading about loop constructs, I probably don't care about π, but if I'm reading about how Raku can leverage π (the character and the value) to help with mathematical/scientific calculations, that's really good to see (and quite impressive).

The example @librasteve provides of:

method area { π * $.radius² }

...is a good example of when the Unicode character really adds value, and where a tooltip version (one showing the Unicode, one not - no opinion on which should be the tooltip, except that it stays consistent) can be very helpful.

I would even just put it in a commented line:

method area { π * $.radius² }
# Alternatively: method area { ... }

(I would've written the content, but I have no idea how to express these in ASCII.)

[jubilatious1]

FWIW, I've been fairly happy cutting-and-pasting from the Raku docs into iTerm2, MacVim, etc.

Set Functions just look way better in Unicode than ASCII.

~$ raku -e 'my $fruits = set <peach apple orange apple apple>;  \
            say "apple" ∈ $fruits;'
True

Also, if you're interpolating strings or playing with regexen then Unicode names are your friends. Time to learn them!

~$ raku -e 'my $fruits = set <peach apple orange apple apple>; 
           say "\"apple\" \c[ELEMENT OF] ($fruits)";'
"apple" ∈ (orange peach apple)

TL;DR

It's a big feature of the language and it works really well so it should be front-and-center. So I disagree with @xsawyerx .

@doomvox

[xsawyerx]

It's a big feature of the language and it works really well so it should be front-and-center.

If the goal is to support people who are still learning the language, I think using Unicode sparingly while they are getting familiar with the core language constructs makes more sense.

I do not have empirical data to cite, but I have heard this concern raised fairly often. I am simply sharing this recurring piece of feedback I have encountered, and one I also share, when learning Raku.

If you find the Unicode characters in the syntax (when learning about non-Unicode related stuff) is helpful for beginners learning language constructs, there's no reason to change it.

[xsawyerx]

FWIW, though, I think the "apple" ∈ $fruits is a good example of when Unicode is relevant, just like @librasteve's example of the area subroutine. The goal wasn't to remove all cases, but only irrelevant ones.

[librasteve]

actually "apple" ∈ $fruits is one of the cases I was going to raise - because, as you say, it is a big improvement on some kind of loop and test and very easy to read and gather the intent --- but for the Set ops the ascii versions are less easy to read and since we write once, read many that is a cost worth paying in this case

[arkiuat]

I think that docs issue 1520 needs to be mentioned in this context. It may be very old, but it's still open, and 4732 (this issue) and 1520 seem as if they might be at cross-purposes to one another.

On the other hand, there's a lot of discussion in that issue of allowing doc users to choose whether they are shown Unicode or Texas versions, which would also achieve the goals sought in this issue (I think?), so perhaps they're really one issue and not at cross-purposes at all.

There are also a couple of closed issues mentioned in that one that have discussions of some of the same matters touched on in this one: 526, which is mentioned prominently up-front there, and the lengthier discussion at 2201.

[tbrowder]

I also agree with the simple approach so as not to distract from the objective. One of my pet peeves is the Saruman example in the failure discussions: what the heck is causing the failure, the spell? Who or what is Saruman? (I do love the book and the movies, but IMHO, the example is too cute.)

[2colours]

I'm not sure if it has been pointed out yet but much related to arkuiat's comment, the style guide of this repo itself suggests the use of non-ascii syntax in examples. Whether that is a good thing or not.

[xsawyerx]

Just to give my $0.02 as a non-Raku developer on the examples in the guide:

When I read an example with Inf, I think "okay, that's a constant representing infinity. Simple idea. Very cool!" When I read an example with ∞, I think "no way will I write in this language," and step away from it.

I understand there are cases (like those in the comments above) where a Unicode character can be more elegant. but versus a clear constant name Inf that I can just type and read? No. Things like that makes me take a hard turn away from Raku. The mere fact that the language developers find it "more comfortable" to show me characters I am not likely to remember, let alone know how to type, and will have to google and copy/paste, shows me the language was not written for me.

The next example is not much better.

• <a b c > (^) <c d e> -> "okay, interesting operator here, I'll need to understand it better."
• <a b c > ⊖ <c d e> -> "what the... what is that even? this definitely isn't for me."

Perhaps mathematicians and Unicode-keyboarded engineers are the prime audience (get it, prime?). In that case, Unicode it up. But if anyone else is the audience, this is a remarkable hindrance IMHO.

[eiro]

hello there,

Just to give my $0.02 as a non-Raku developer on the examples in the guide: When I read an example with Inf, I think "okay, that's a constant representing infinity. Simple idea. Very cool!" When I read an example with ∞, I think "no way will I write in this language," and step away from it. • <a b c > ⊖ <c d e> -> "what the... what is that even? this definitely isn't for me."

The usage I have of raku is basically to do some math and I have the very opposite feeling: I was really happy to learn about the symmetric difference https://en.wikipedia.org/wiki/%E2%8A%96 I saw from time to time in some math papers, so happy to be able to write stuff like `( ½, 1 … ∞ )` and remember the day I read "resitance is futile" (https://perl6advent.wordpress.com/2012/12/18/day-18-formulas-resistance-is-futile/) because I was like "there is no more readable code than the one you would have used on a paper to explain the problem to your colleague". so the ability to define operators and use unicode for this is really important to me and the people I try to convince to use raku also like it mostly. the only thing I join you on is that I use vim (and the :h digraph-table) on a terminal tat is unicode ready but I imagine other modern editors like vs code have plugins but I don't know how hard is it to find and install them for newbies. maybe we should to both: always texas notation + utf-8

Perhaps mathematicians and Unicode-keyboarded engineers are the prime audience (get it, prime?).

It does have a fun factor :) regards

…

-- Marc Chantreux