Skip to content

fix(cli): force UTF-8 stdio so cp1252 Windows shells don't crash on '↳'/'✓'#290

Merged
RaghavChamadiya merged 1 commit into
mainfrom
fix/windows-cp1252-unicode
May 28, 2026
Merged

fix(cli): force UTF-8 stdio so cp1252 Windows shells don't crash on '↳'/'✓'#290
RaghavChamadiya merged 1 commit into
mainfrom
fix/windows-cp1252-unicode

Conversation

@RaghavChamadiya
Copy link
Copy Markdown
Member

Closes #271.

Summary

On Windows shells defaulting to cp1252 (cmd.exe, default PowerShell sessions without WT_SESSION), Rich falls back to its legacy Windows renderer which encodes every printed line through the active code page. The first non-ASCII glyph in repowise's progress UI — in _timed_step, in the sub-step ticks — raises UnicodeEncodeError: 'charmap' codec can't encode character '↳' and aborts the run mid-pipeline. Partial state has already been written by the time it triggers, so recovery depends on --resume working.

The fix reconfigures sys.stdout and sys.stderr to UTF-8 with errors="replace" at the top of repowise.cli, before any Rich Console is constructed downstream. errors="replace" means even a stream that genuinely can't render a glyph still writes a placeholder rather than crashing, so the pipeline always reaches completion. Streams that are already UTF-8 (Linux, macOS, Windows Terminal with WT_SESSION) are unaffected.

Adopted the reporter's suggested fix (option 1) — cheapest, broad-stroke, doesn't touch Rich internals.

Implementation notes

  • New helper module repowise/cli/_stdio.py exposes _ensure_utf8_stdio(), called from repowise/cli/__init__.py so it runs on any CLI import path.
  • Tolerant of swapped-in streams without a reconfigure method (e.g. pytest -s, embedding harnesses that pass a StringIO) and silent on OSError/ValueError from detached or non-text wrappers — the goal is never to introduce a crash while fixing one.

Test plan

7 new tests in tests/unit/cli/test_stdio.py:

  • _reconfigure passes encoding="utf-8", errors="replace" to the stream
  • tolerates streams without a reconfigure method (StringIO)
  • swallows OSError from detached streams
  • tolerates None stream
  • _ensure_utf8_stdio doesn't raise when both stdout/stderr are swapped StringIO
  • _ensure_utf8_stdio reconfigures both stdout and stderr
  • End-to-end: a cp1252 TextIOWrapper with errors="replace" accepts the exact ↳ … ✓ line from the bug report without raising
$ .venv\Scripts\python.exe -m pytest tests/unit/cli/ -q
============================= 287 passed in 6.42s =============================

ruff format + ruff check clean on the changed files.

Why not the other suggested fixes

The reporter listed three other options. Skipped because:

  • Option 2 (build the Console with a UTF-8 file) — would only fix the shared console in helpers.py; any other module that constructs its own Console() would still hit the same trap. Reconfiguring stdio fixes them all at once.
  • Option 3 (ASCII-only fallback for /) — would have to be applied at every print site, and Windows Terminal users would lose the nicer glyphs unnecessarily.
  • Option 4 (just document the env var) — leaves the crash in place for users who don't read README first.

…↳'/'✓'

Closes #271. Windows shells (cmd.exe, default PowerShell) ship with a
cp1252 code page. Rich falls back to its legacy Windows renderer which
encodes through the active code page — the first non-ASCII glyph in the
progress UI (e.g. `↳` printed by `_timed_step`) raises a
UnicodeEncodeError and aborts the run mid-pipeline, leaving partial
state behind.

Reconfigure sys.stdout/sys.stderr to UTF-8 with errors="replace" at the
top of `repowise.cli`, before any Rich Console is constructed downstream.
errors="replace" means even a stream that genuinely can't render a glyph
still writes a placeholder rather than crashing. Behavior on already-UTF-8
streams (Linux, macOS, Windows Terminal with WT_SESSION) is unchanged.
@RaghavChamadiya RaghavChamadiya requested a review from swati510 as a code owner May 28, 2026 15:21
@RaghavChamadiya RaghavChamadiya merged commit dafe972 into main May 28, 2026
5 checks passed
@RaghavChamadiya RaghavChamadiya deleted the fix/windows-cp1252-unicode branch May 28, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] UnicodeEncodeError on Windows (cp1252) crashes repowise init mid-pipeline

2 participants