Render glyphs with GLSL for hinting + anti-aliasing?

[snickell]

I'm guessing one of the major use cases for Primrose is (as I'm using it) rendering a text editor in XR, where pixels are jenky and ginormous. Displays were of course all 100% jenk in the 90s and a couple techniques from that era might be of interest:

1. truetype hinting (snapping 'ink' to the pixel grid to produce fewer grey pixels)
2. sub-pixel antialiasing (leveraging the RGB sub-pixels to gain a ~30% - 50% edge resolution boost in one axis on some headsets)

To apply these you have to know where the pixel grid is relative to what you're rendering, which really calls for a pixel shader in the webgl context. IMO, applying both of them can significantly boost text readability in XR, I've been playing with the very cool https://github.com/astiopin/webgl_fonts as a starting point.

Is that something you might be interested in for Primrose? Could you suggest where in the primrose source this might make sense to play with? So far I haven't narrowed down on where in the source glyph rendering is done, guessing its SDF or one of the variants? Obviously this would change how the rendering was done, passing a text buffer with layout coordinates to a GL shader rather than SDF glyph textures with coordinates....

[capnmidnight]

The rendering of the text itself happens in the Primrose class in the renderCanvasForeground() function: https://github.com/capnmidnight/Primrose/blob/master/js/package/src/primrose.js#L216-L271

No, it's not SDF of any kind. I'm using the text rendering facilities of CanvasRenderingContext2D.

However, Primrose doesn't use SDF for lack of trying SDF. There are a number of reasons why this is doesn't really work out:

• The text editor texels don't align with screen pixels.

  • A single texel might span multiple screen pixels, or multiple texels might be packed in a single pixel. Multi-sample anti-aliasing and anisotropic filtering are the techniques that have the most value here.
  • Primrose already has support for MSAA in that the render resolution is scalable with the scaleFactor setting on the text editor control.
  • Anisotropic filtering is outside of the scope of Primrose, as Primrose is "just an image", it doesn't know anything about the 3D scene.
  • Incidentally, CanvasRenderingContext2D is configurable to render with font hinting and sub-pixel AA, so maybe try editing the context configuration to include imageSmoothingQuality (though it's actually already enabled for grey-scale AA).
  • There is also the contextAttributes parameter for HTMLCanvasElement.getContext(), where supposedly disabling alpha transparency kicks Safari and Chrome into using sub-pixel AA.
    • However, that would also require either rendering all of the text editor features to a single canvas (there is currently a separate background, foreground, and "trim" canvas, to avoid repainting text more often than necessary), or clever use of globalCompositeOperation to blend the opaque canvases together.
    • This would probably need a change to the Themes to use different compositing operations for Light vs Dark themes.

• Screen pixels in a VR headset are both extremely large in comparison to screen pixels on even old displays, and have the unique property that they are stationary in the user's view. Sub-pixel anti-aliasing actually works against us in this environment, creating strange depth artifacts when the sub-pixel hinting does not align to the same coverage of sub-pixels in the left vs right displays.

• Shader-based text rendering requires generating glyph maps of some kind.

  • Covering the full range of glyphs for any given font is next to impossible, even outside of WebGL's artificial texture memory constraints.
  • Primrose has some support for rendering Unicode glyphs (currently restricted to left-to-right scripts only, with a goal of eventually being able to render international scripts of all kinds).
  • I've also not seen a lot of examples of SDF rendering different font weights, styles or colors, per-word. That would break the syntax highlighting feature, which is rather much the point of Primrose. Rendering text in a buffer isn't all that hard, it's the tokenizing and layout of highlighted code that is much, much more difficult. Perhaps some sort of adaptation of Multichannel Signed Distance Fields might be possible, with an extra styling map for the text color and style, but my understanding is that that would be greenfield development.