Chinese Text Search Limitation with FTS5 Unicode61 Tokenizer

[cjr96720]

• I have searched open and closed issues for duplicates
• I am submitting a bug report for existing functionality that does not work as intended
• This isn't a feature request or a discussion topic

---

Bug description

When searching for Chinese text in Signal iOS, there's a limitation with how search results are matched.

Steps to reproduce

For example, when searching for "你好" (hello):

• Send a message "前言 你好" and another "前言你好"
• Navigate to the search bar and search for the keyword "你好"
• Result:
  • ✅ Messages containing "前言 你好" (with a space) are found
  • ❌ Messages containing "前言你好" (without a space) are NOT found

Actual result: Messages containing "前言你好" (without a space) are NOT found

Expected result: Messages containing "前言你好" (without a space) should be found.

Screenshots

Device info

Device: iPhone 15 Pro

iOS version: 26.1.0

Signal version: 7.84

Link to debug log

[cjr96720]

I think the root cause is that Signal iOS uses SQLite FTS5 with the unicode61 tokenizer for full-text search indexing.

The unicode61 tokenizer treats each Chinese character as an individual token. When searching for "你好", FTS5 looks for two consecutive tokens: "你" followed by "好".

In the text "前言你好", the characters are indexed as four consecutive tokens: "前", "言", "你", "好". While both "你" and "好" exist in the index, they are at positions 3 and 4, and FTS5's phrase search requires finding the exact consecutive token sequence starting from a token boundary. Since "你好" doesn't start at a token boundary in this context, it doesn't match.

So... maybe the most thorough solution would be to implement a custom FTS5 tokenizer that supports Chinese word segmentation?

[sashaweiss-signal]

Hi, thanks for reporting. You're right that there are issues with Chinese text and our in-app search – something we want to improve, and are tracking internally. I'm going to mark as Acknowledged and have linked this to our internal tracking tool! We'll post here when we have an update.