Skip to content

Add CJK and Eastern-Slavic name romanization round-trip coverage - #2239

Merged
maziyarpanahi merged 5 commits into
masterfrom
feature/om-270-feature-om-cjk-slavic-romanization-roundtrip-sui
Aug 12, 2026
Merged

Add CJK and Eastern-Slavic name romanization round-trip coverage#2239
maziyarpanahi merged 5 commits into
masterfrom
feature/om-270-feature-om-cjk-slavic-romanization-roundtrip-sui

Conversation

@maziyarpanahi

Copy link
Copy Markdown
Owner

Summary

Adds a dependency-free romanize_name helper for deterministic Latin renderings of non-Latin PERSON surrogates. It reuses the existing Unicode script detection and segmentation utilities, with algorithmic Kana, Hangul, Cyrillic, and Greek handling plus deterministic Han-name readings and an explicit ASCII fallback for unknown Han characters.

Adds synthetic offline regression coverage for Japanese, Chinese, Korean, Russian, and Greek language packs. The tests exercise detect → replace → re-detect stability, romanized PERSON re-detection, and exact offsets on mixed Latin/non-Latin lines. Unwired language packs skip with a clear wiring reason.

Validation

  • .venv/bin/python -m pytest tests/unit/core/test_name_romanization_roundtrip.py -q — 15 passed
  • .venv/bin/ruff check openmed/core/translit.py tests/unit/core/test_name_romanization_roundtrip.py — passed
  • .venv/bin/ruff format --check openmed/core/translit.py tests/unit/core/test_name_romanization_roundtrip.py — passed
  • git diff --check — passed

All test data and detector responses are synthetic and offline.

Closes #468

@maziyarpanahi maziyarpanahi added roadmap-v2 OpenMed V2 roadmap backlog improvement Hardening / refactor of existing code P2 Medium labels Aug 5, 2026
@maziyarpanahi

Copy link
Copy Markdown
Owner Author

Thank you @maziyarpanahi. I reviewed this against #468 and current master, then added maintainer follow-up d2164502.

What changed:

  • Kept the mixed-script name round-trip coverage fully offline and deterministic.
  • Exercised transliteration and exact surrogate re-detection without adding network or restricted-data dependencies.

Validation:

  • .venv/bin/python -m pytest tests/unit/core/test_name_romanization_roundtrip.py -q — 15 passed.
  • .venv/bin/python -m pytest tests/unit -q — 11,593 passed, 89 skipped on the exact aggregate release tree.
  • Ruff lint and format checks passed.

This now satisfies the issue and is ready to merge.

@maziyarpanahi
maziyarpanahi merged commit 5d61414 into master Aug 12, 2026
0 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Hardening / refactor of existing code P2 Medium roadmap-v2 OpenMed V2 roadmap backlog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a CJK and Eastern-Slavic name romanization round-trip test suite

1 participant