Shaper Driven Itemisation

[taj-p]

Today, the text pipeline is dominated by 2 stages: shaping and font selection as shown in the below profile:

Per @behdad's comment (see 2. in harfbuzz/harfrust#297 (comment)), it appears that a more correct and, likely, faster approach is to perform shaper-driven itemization where:

you simply shape the text with your primary font, then look for unshaped holes in the middle and reshape those holes with the next font, and next font, etc. This sidesteps the classic font fallback itemization.

The hope is that this strategy could consolidate the redundant work happening in shaping and font selection (charmap lookups and normalisation). Moreover, it could help us be more correct since, as Behdad states, it's unlikely for us to succeed in making our itemizer match the same normalisation applied by the shaper.

[behdad]

Thanks. I recommend this approach indeed. @khaledhosny has experience implementing it in LibreOffice, and @drott in Chrome. I'll add a sketch of the algorithm soon.

[taj-p]

I've implemented a naive implementation that passes the text through the shaper with the primary font and recursively fills in holes with fallback fonts.

I'm already seeing 10-15% performance improvement and, from inspecting performance profiles, I think a mature implementation could get to 15-25% improvement. Very promising!

However, @behdad , I noticed that performance regresses significantly when the entire text is a hole - for example, when the primary font is Latin Roboto but we're shaping Arabic (so we shape with Roboto before falling back to a font that can appreciably support most of the text). The regression varies between 50 to 400% depending on how many fall backs we churn through.

I think this might be a caller issue. For my use case, I can always provide the "right" font upfront, but I'm wondering whether this is something that you've found best tackled another way (or whether this is a behaviour best handled by the caller).

[behdad]

Thanks. I'll get back to you soon.

[behdad]

I've implemented a naive implementation that passes the text through the shaper with the primary font and recursively fills in holes with fallback fonts.

Do you have some code to show us?

Will still comment in full later.

[taj-p]

Do you have some code to show us?

Yup - of course. Please consider all of this draft code for me to get a feel for the design.

conor-93#2