Skip to content

Reland "[libc] utf8 to 32 CharacterConverter" #144450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 17, 2025
Merged

Reland "[libc] utf8 to 32 CharacterConverter" #144450

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
86d0cb6
Revert "Revert "[libc] utf8 to 32 CharacterConverter" (#144446)"
sribee8 Jun 16, 2025
45d67dc
fixed base_mask to be the correct size
Jun 16, 2025
e7669d3
added casting to avoid integer promotion
Jun 16, 2025
7f24572
fixed formatting
Jun 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion libc/src/__support/wchar/character_converter.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ int CharacterConverter::push(char8_t utf8_byte) {
/* Since the format is 110xxxxx, 1110xxxx, and 11110xxx for 2, 3, and 4,
we will make the base mask with 7 ones and right shift it as necessary. */
constexpr size_t SIGNIFICANT_BITS = 7;
uint32_t base_mask = mask_trailing_ones<uint32_t, SIGNIFICANT_BITS>();
uint8_t base_mask = mask_trailing_ones<uint8_t, SIGNIFICANT_BITS>();
state->total_bytes = num_ones;
utf8_byte &= (base_mask >> num_ones);
}
Expand Down
Loading