feat(tantivy): add language detection and language-aware tokenization#2771

Merged

yujonglee merged 3 commits intomainfrom

devin/1767448220-tantivy-language-support

Jan 3, 2026

Commits on Jan 3, 2026

feat(tantivy): add language detection and language-aware tokenization

devin-ai-integration[bot]
and
yujonglee
committed
test(tantivy): add comprehensive tests for language detection and tokenization

devin-ai-integration[bot]
and
yujonglee
committed
chore(tantivy): update TypeScript bindings with language field

devin-ai-integration[bot]
and
yujonglee
committed