Skip to content

Commit 7ae67b4

Browse files
authored
Clean up GenerateBreakTest (#975)
This fulfils the following action item: UTC-155-A89 Robin Leroy, PAG Document extra classes used for testing characters in the segmentation test HTML files for 11.0. [E.g. ZWJ_FE, CM1_CM, etc.] (Retargeted for 13.0, 14.0, 15.0.) It also fixes #354. It also changes the pair table in LineBreakTest.html to show the three way direct/indirect/prohibited break distinction (across spaces), like the old pair table in UAX14 (see https://www.unicode.org/notes/tn54/alba-2.html?v=9.0.0). As in #970, the test files are not diffable, but I tested them with ICU. I tried improving the stability of the sample generation a little bit, but it is not as fancy as what Mark suggests in https://www.unicode.org/notes/tn54/alba-2.html?v=9.0.0#p478. I might do that in another PR.
1 parent b1e89f4 commit 7ae67b4

File tree

11 files changed

+24043
-23212
lines changed

11 files changed

+24043
-23212
lines changed

unicodetools/data/ucd/dev/auxiliary/GraphemeBreakTest.html

Lines changed: 147 additions & 125 deletions
Large diffs are not rendered by default.

unicodetools/data/ucd/dev/auxiliary/GraphemeBreakTest.txt

Lines changed: 696 additions & 1030 deletions
Large diffs are not rendered by default.

unicodetools/data/ucd/dev/auxiliary/LineBreakTest.html

Lines changed: 2331 additions & 2210 deletions
Large diffs are not rendered by default.

unicodetools/data/ucd/dev/auxiliary/LineBreakTest.txt

Lines changed: 18509 additions & 16363 deletions
Large diffs are not rendered by default.

unicodetools/data/ucd/dev/auxiliary/SentenceBreakTest.html

Lines changed: 132 additions & 116 deletions
Large diffs are not rendered by default.

unicodetools/data/ucd/dev/auxiliary/SentenceBreakTest.txt

Lines changed: 351 additions & 349 deletions
Large diffs are not rendered by default.

unicodetools/data/ucd/dev/auxiliary/WordBreakTest.html

Lines changed: 127 additions & 105 deletions
Large diffs are not rendered by default.

unicodetools/data/ucd/dev/auxiliary/WordBreakTest.txt

Lines changed: 1426 additions & 1306 deletions
Large diffs are not rendered by default.

unicodetools/src/main/java/org/unicode/text/UCD/GenerateBreakTest.java

Lines changed: 138 additions & 1489 deletions
Large diffs are not rendered by default.

unicodetools/src/main/java/org/unicode/tools/Segmenter.java

Lines changed: 159 additions & 96 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)