[doc](text-shaping): Expand inline-layout page with text shaping

information.

Signed-off-by: Desiatkin Dmitrii <d.desyatkin@innopolis.university>

Author: d-desiatkin

src/architecture/fonts.md

@@ -100,8 +100,11 @@ Lets highlite several important architectural details:
  - To avoid data duplication fonts loaded only once and managed by separate thread. That reduces memory footprint of the program. Another reason is limitations that particular platform typesetting library stack imposes on servo(I.E threadsafety of function calls)
  - Special IPC mechanism that serve as middleman between Script and Compositor threads (processes) provided;
 
+One additional thing that is important to mention is that currently two approaches
+
 ## Fonts sequence diagram
 Lets also analyze how servo initializes cause it is important to understand when fonts is loaded in general pipeline. Sequence diagram bellom demonstrate how significant number of servo threads will be launched.
+
 ```mermaid
 sequenceDiagram
     box rgb(255, 173, 173) main thread
@@ -224,7 +227,7 @@ But I don't know whether it will be accepted so I will explain important concept
 
 Each `FontGroupFamily` represent `FontFamily` that may be represented on device as single font file or a set of font-files if we consider `segmented fonts`. Each `FontFamily` (`FontGroupFamily`) must have set of `FontDescriptor`s that allow to uniquely identify `FontFace` object within particular `font file`.
 
-That means that `FontGroup::FontDescriptor` represents some abstract `FontFace` that must be present in at least one of `FontFamily` objects. In case we will not be able to find it we will start `installed_font_fallback` procedure;
+That means that `FontGroup::FontDescriptor` represents some abstract `FontFace` that must be present in at least one `FontGroupFamily` object. In case we will not be able to find it we will start `installed_font_fallback` procedure;
 
 `Language_id` is the new CSS4 feature that allow us to more accurately control visual representation. Lets say we have two `FontFamily` within specified list which have `FontFace` that will satisfy `FontGroup::FontDescriptor`. In that case old spec asked us to simply pick first one that satisfies the descriptor. CSS4 allows user to setup corresondance between language of the element and particular family that we want to use:
 ```html
@@ -235,20 +238,29 @@ That means that `FontGroup::FontDescriptor` represents some abstract `FontFace`
 <link rel="author" title="Richard Ishida, W3C" href="https://www.w3.org/International/questions/qa-css-lang">
 <link rel="author" title="Desiatkin Dmitrii" href="https://github.com/d-desiatkin">
 <style>
-body 		{font-family: "Times New Roman" "Kai", "KaiTi", "DFKai-SB", "BiauKai", serif;}
+body 		{font-family: "Times New Roman", "Scheherazade", "Kai", "KaiTi", "DFKai-SB", "BiauKai", "Doulos SIL", serif;}
+:lang(ar) 	{font-family: "Scheherazade",serif;
+                 font-size: 120%;}
 :lang(zh-Hant) 	{font-family: Kai,KaiTi,serif;}
 :lang(zh-Hans) 	{font-family: DFKai-SB,BiauKai,serif;}
+:lang(din) 	{font-family: "Doulos SIL",serif;}
 </style>
 <div>
 <p>It is polite to welcome people in their own language:</p>
 <ul>
-    <li>歡迎 欢迎 <span lang="zh-Hans">欢迎</span></li>
-    <li>欢迎 歡迎 <span lang="zh-Hant">歡迎<span></li>
+    <li>Καλοσωρίσατε <span lang="el">Καλοσωρίσατε</span></li>
+    <li>>اهلا وسهلا <span lang="ar">اهلا وسهلا</span></li>
+    <li>Добро пожаловать <span lang="ru">Добро пожаловать</span></li>
+    <li>Kudual <span lang="din">Kudual</span></li>
+    <li>欢迎 <span lang="zh-Hans">欢迎</span></li>
+    <li>歡迎 <span lang="zh-Hant">歡迎<span></li>
 </ul>
 </div>
 ```
 ![alt text](../images/font-lang-based-matching-demo.png)
-On pictures above you can see that body have several conflicting font-families all of them are capable to display particular character. Language allow us to additionaly precisely chose `FontGroupFamily` within `FontGroup`
+On pictures above you can see that `body` have several conflicting font-families all of them are capable to display particular character. Then we declare a list of elements where first wellcome string will use body font-family rules, and second wellcom declared inside `<span>` will use additional language hint for `font-family` style.
+
+So `language_id` allow us to additionaly precisely chose `FontGroupFamily` within `FontGroup`. Differences are easy to spot on arabic and Chinese language versions.
 
 So the task of `find_by_codepoint` is:
 1. Traverse all `font files` that represent particular `FontFamily` on device, and accumulate all possible `FontFace`s within family in question. Get `FontFace`s in form the of list of `FontDescriptor`s (load with help o OS / font third party libraries).

src/architecture/inline-layout.md

@@ -1,8 +1,8 @@
 # Inline Layout
-In modern browsers inline layout consists form the following steps:
+In modern browsers inline layout consists from the following steps:
 ## Layout preparation
 - Divide `BoxTree` into the set of subtree view objects that is called `Block Formatting Context` (`BFC`) and `Inline Formatting Context` (`IFC`); Plese do not confuse this termin with `Independent Formatiing Context` that also can be abbreviated as `IFC`.
-- Accumulate all text within `Inline Formatting Context` inside one `infinite string`; Accumulate all elements with visual representation in `Inline Formatting Context` into container of `Inline Items`; During such aggregation html elements may introduce additional codepoints into string. In example some html elements under special conditions may setup independent `BIDI paragraph object` and to adress this fact it is necessary to introduce special `bidi-control symbols`.
+- Accumulate all text within `Inline Formatting Context` into one `infinite string`(important to properly handle some abstractions that span across several hypertext markup elements - i.e. bidi directionality); Accumulate all elements with visual representation in `Inline Formatting Context` into container of `Inline Items`; During such aggregation html elements may introduce additional codepoints into string. In example some html elements under special conditions may setup independent `BIDI paragraph object` and to adress this fact it is necessary to introduce special `bidi-control symbols`.
 - Then we need to prepare text for `shaping` procedure (the result of shaping is a size and position of each symbol that it will occupy within the string). **OpenType / TrueType specific preparation is described further**: During this process we split aquired infinite string into segments of consequative symbols that is sharing a set properties (same bidi-direction, font, language and script), and memorise such segments as `ranges` within original `infinite string`. For each generated `text segment` corresponding item must be introduced in `Inline Items` container.
 - After we use some third party library that will actually extract information that is contained within font file
 and perform `shaping`
@@ -15,12 +15,61 @@ and perform `shaping`
 ## Detailed Explanation of steps:
 ### Inline Items construction
 TODO
-### Text Segmentation
-TODO
 #### BIDI level computation
 TODO
 #### Font Style matching procedure
+Exact details is provided in servo [fonts module](./fonts.md)
+
+### Text Segmentation & Shaping
+#### What is text shaping?
+Wikipedia provides following definition:
+> Text shaping is the process of converting text to glyph indices and positions as part of text rendering. It is complementary to font rendering as part of the text rendering process; font rendering is used to generate the glyphs, and text shaping decides which glyphs to render and where they should be put on the image plane. Unicode is generally used to specify the text to be rendered.
+
+Microsoft have the following document that describe [Text layout](https://learn.microsoft.com/en-us/globalization/fonts-layout/text-layout). It have section devoted to [Text shaping](https://learn.microsoft.com/en-us/globalization/fonts-layout/text-layout#text-shaping), however clear and short definition is not provided, this section names four important subtasks that any text shping engine must solve:
+- [correct processing of ligatures](https://learn.microsoft.com/en-us/globalization/fonts-layout/text-layout#ligatures)
+- [script-specific replacement of the characters that depends on context](https://learn.microsoft.com/en-us/globalization/fonts-layout/text-layout#contextual-shaping)
+- [combining special characters (i.e diacritics and tone marks) into single visual representation](https://learn.microsoft.com/en-us/globalization/fonts-layout/text-layout#combining-characters)
+- [script-specific (i.e Hindi, Devanagari) reordering of the characters](https://learn.microsoft.com/en-us/globalization/fonts-layout/text-layout#character-reordering)
+
+[HarfBuzz definition](https://harfbuzz.github.io/what-is-harfbuzz.html) of text shaping:
+> Text shaping is the process of translating a string of character codes (such as Unicode codepoints) into a properly arranged sequence of glyphs that can be rendered onto a screen or into final output form for inclusion in a document.
+
+#### What facts everyone must know about text shaping:
+
+First that everyone must know about font shaping is the fact that different shaping approaches exists. The two of which author is aware of is [SIL Graphite](https://graphite.sil.org/) and Opentype / TrueType shaping algorithms. Great writeup on both technologies provided by [article](https://graphite.sil.org/graphite_aboutOT.html) on the SIL Graphite website.
+I also feel obliged to provide a link on [great repository](https://github.com/n8willis/opentype-shaping-documents) that contains a lot of documents regarding the OpenType / TrueType shaping.
+
+Second most important thing that everyone must know is the fact that w3c established [Web Font Working Group](https://www.w3.org/groups/wg/webfonts/). That group created [WOFF fonts](https://www.w3.org/TR/WOFF/) to improve and standartized loading of fonts from web resources, mostly it contains the standart of data compression and additional headers for web straming. If we look at uncompressed font format we will find that it follows simmilar structure and shaping model as OpenType fonts. That means that OpenType shaping models is currently dominating web domain. So the rest of the information in the shaping section will be devoted to shaping operations specific to [OpenType / Truetype shaping models](https://harfbuzz.github.io/opentype-shaping-models.html).
+
+#### OpenType / Truetype shaping model
+It would be unreasonable to copy information about all different Opentype and Truetype shaping models here, so I will just provide the link to [awesome repository](https://github.com/n8willis/opentype-shaping-documents) about shaping again.
+
+Now let's get to more practical side. For web engine developer it is not necessary to know every detail of notorious shaping process cause we are working with third-party crates that provide already written shaping engine. Here servo uses opensource HarfBuzz shaping engine written in C/C++ (if someone is intrested in writing pure Rust OpenType shaping engine, please consider to provide your implementation to servo authors). `harfbuzz-sys` crate is used as FFI between C/C++ implementation and Rust language.
+
+HarfBuzz engine have special requirements to the inputs.
+1. we must create special `hb_font_t` and `hb_face_t` structures. In our case creation of such structures is dictated by CSS styles. Users will define the font through CSS font-family and language; and face through combination of font-size, font-weight, font-style, font-stretch, ...
+2. we must properly segment the whole text string accumulated in IFC to the segments of characters that share common properties (more details at [what harfbuzz doesn't do](https://harfbuzz.github.io/what-harfbuzz-doesnt-do.html)). List of properties provided bellow.
+
+#### Text segment features that is shared by all codepoints
+ - Bidi direction
+ - Language
+ - Script
+ - ***Font*** (particular rigidly defined face within the font)
+
+After segmentation we must provide set of OpenType features to the shaping engine.
+
+### Inline items linebreaking
+Unfortunately I don't have enough understanding of token based linebreaking algorithm to shrtly describe it here.
 TODO
-### Shaping
+
+### Inline items BIDI Reordering
+Bidi reordering should be preformed exactly as stated in [Unicode® Standard Annex #9 Unicode Bidirectional Algorithm](https://unicode.org/reports/tr9/).
+Currently servo have only partial implementation of that algorithm.
+Problems mostly concentrated at per-line reordering rules.
+Rules [L1](https://unicode.org/reports/tr9/#L1)-[L4](https://unicode.org/reports/tr9/#L4) not properly implemented. For example we don't use information about bidi-paragraphs at all.
+
+On conceptual level application developer should allways use icu4c or new pure rust icu crates for such operations.
+
+### Line fragments generation
 TODO