ICU-23307 Space-sensitive UnicodeSet string literals

[eggrobin]

Checklist

• Required: Issue filed: ICU-23307
• Required: The PR title must be prefixed with a JIRA Issue number. Example: "ICU-NNNNN Fix xyz"
• Required: Each commit message must be prefixed with a JIRA Issue number. Example: "ICU-NNNNN Fix xyz"
• Issue accepted (done by Technical Committee after discussion)
• Tests included, if applicable
• API docs and/or User Guide docs changed or added, if applicable
• Approver: Feel free to merge on my behalf

ALLOW_MANY_COMMITS=meow

[eggrobin]

Somehow this is affecting Han-Latin/Names; Latin-Bopomofo, even though neither appears to contain string literals in UnicodeSets.

[eggrobin]

Latin-NumericPinyin.xml:

($tone) ( [i o n u {o n} {n g}]) → $2 &Pinyin-NumericPinyin($1);

Whyyy??!

[eggrobin]

de-ASCII also does this:

$AE = [Ä {A \u0308}];
$OE = [Ö {O \u0308}];
$UE = [Ü {U \u0308}];

[ä {a \u0308}] → ae;
[ö {o \u0308}] → oe;
[ü {u \u0308}] → ue;

and blt-fonipa-t-blt:

$DIGRAPHS = [{ꪹ  ꪸ} {ꪹ  ꪷ} {ꪹ ꪱ}];

These seem to be the only users of space-insensitivity of string literals.

[macchiati]

An artifact, that we just never noticed before the change

…

On Fri, Jan 9, 2026, 09:30 Robin Leroy ***@***.***> wrote: *eggrobin* left a comment (unicode-org/icu#3839) <#3839 (comment)> Latin-NumericPinyin.xml: ($tone) ( [i o n u {o n} {n g}]) → $2 &Pinyin-NumericPinyin($1); Whyyy??! — Reply to this email directly, view it on GitHub <#3839 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMC3UQT5PIQVFD2URAL4F7QVBAVCNFSM6AAAAACRGSRVNOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTOMRZHEYDMMRSGU> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[eggrobin]

An artifact, that we just never noticed before the change

I added something to the CLDR-design agenda. (Not sure I can make it Monday, I expect this week to be chaotic—I am moving Friday—, but I’m sure you have enough context to discuss this topic.)

[jira-pull-request-webhook]

Notice: the branch changed across the force-push!

• icu4c/source/common/uniset_props.cpp is different
• icu4c/source/test/intltest/usettest.cpp is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

[eggrobin]

@markusicu How do I deal with the transliterator breakages until CLDR fixes its rules? Do I just sit on this PR until then?

[macchiati]

We can get them into v49 if we act in the next few days. I'm away from my computer right now, but can handle that if you have the exact list of needed changes, the follow up with the tests and ST checks.

…

On Fri, Jan 16, 2026, 03:25 Robin Leroy ***@***.***> wrote: *eggrobin* left a comment (unicode-org/icu#3839) <#3839 (comment)> @markusicu <https://github.com/markusicu> How do I deal with the transliterator breakages until CLDR fixes its rules? Do I just sit on this PR until then? — Reply to this email directly, view it on GitHub <#3839 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMH5IKB53DLWWBM6X434HDDBDAVCNFSM6AAAAACRGSRVNOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTONJZGYYDAOBWGI> . You are receiving this because you commented.Message ID: ***@***.***>

[markusicu]

How do I deal with the transliterator breakages until CLDR fixes its rules?

@macchiati please make sure that the translit rules get fixed.
@eggrobin for ICU, I would manually change the .txt files in ICU, assuming that the next CLDR integration will come in with equivalent fixes.

[macchiati]

Agreed; Robin shouldn't be blocked on this.

[macchiati]

With a quick regex \[[^\]]*\{[^\}]\s, I'm getting the following in transforms, and no other hits in /common, exemplars/, or keyboards/.

So that matches all of the ones Robin found; that will be easy to fix on the CLDR side

blt-fonipa-t-blt.xml
50: $DIGRAPHS = [{ꪹ  ꪸ} {ꪹ  ꪷ} {ꪹ ꪱ}]; 
de-ASCII.xml (6 matches)
13: $AE = [Ä {A \u0308}]; 
14: $OE = [Ö {O \u0308}]; 
15: $UE = [Ü {U \u0308}]; 
17: [ä {a \u0308}] → ae; 
18: [ö {o \u0308}] → oe; 
19: [ü {u \u0308}] → ue; 
Latin-NumericPinyin.xml
29: ($tone) ( [i o n u {o n} {n g}]) → $2 &Pinyin-NumericPinyin($1); 

[macchiati]

The CLDR PR and ticket are at unicode-org/cldr#5297

[eggrobin]

Thanks for doing that @macchiati. I was busy moving so didn’t have the time to look at this.

[richgillam]

LGTM