fix: handle Chinese and other non-Latin character encodings in terminal output

[roomote]

Summary

This PR fixes issue #6768 where Chinese characters appear garbled in terminal command output.

Problem

When users with Chinese language settings run commands that output Chinese text, the characters appear as garbled text. This is because the terminal was forcing en_US.UTF-8 encoding regardless of the system locale, causing encoding mismatches for systems using GBK or other non-UTF-8 encodings.

Solution

• Detect the system current locale from environment variables (LANG or LC_ALL)
• Extract the language/country code (e.g., zh_CN from zh_CN.GBK)
• Apply the appropriate UTF-8 version of that locale (e.g., zh_CN.UTF-8)
• Fall back to en_US.UTF-8 only when no locale is detected

Changes

• Modified ExecaTerminalProcess to dynamically detect and use the appropriate UTF-8 locale
• Added comprehensive tests for various locale scenarios (Chinese, Japanese, Korean, etc.)
• All existing tests continue to pass

Testing

• Added tests for Chinese (GBK), Japanese (SJIS), Korean (EUC-KR) locale conversions
• Added tests for already UTF-8 locales and fallback scenarios
• All tests pass successfully

Fixes #6768

---

Important

Fixes encoding issue for non-Latin characters in terminal outputs by dynamically setting UTF-8 locale in ExecaTerminalProcess.

• Behavior:
  • ExecaTerminalProcess now detects system locale from LANG or LC_ALL and applies the UTF-8 version.
  • Falls back to en_US.UTF-8 if no valid locale is detected.
• Testing:
  • Added tests in ExecaTerminalProcess.spec.ts for locale conversion to UTF-8 for Chinese (GBK), Japanese (SJIS), Korean (EUC-KR), and fallback scenarios.
  • Ensures existing environment variables are preserved.
• Misc:
  • All existing tests continue to pass.

^{This description was created by for 1f1abd2. You can customize this summary. It will automatically update as commits are pushed.}

[roomote]

Reviewing my own code feels like debugging in a mirror maze - everything looks backwards but somehow still broken.

Review Summary

The implementation correctly addresses the core issue of Chinese character encoding in terminal output. The approach of detecting the system locale and converting it to UTF-8 is sound. Here are some suggestions for improvement:

Important Suggestions

1. Regex pattern could be more robust - The current regex /^([a-z]{2}_[A-Z]{2})/i only matches standard locale formats like zh_CN but might miss valid locales with:

   • Three-letter language codes (e.g., cmn_CN for Mandarin Chinese)
   • Locales with variants (e.g., en_US.UTF-8@euro)
   • Special locales (e.g., C.UTF-8 or POSIX)

2. Missing edge case handling - The code doesn't check if the locale is already UTF-8. Locales like C.UTF-8 or POSIX would incorrectly fall back to en_US.UTF-8.

3. Consider locale validation - The code assumes any matched locale is valid, but it might be worth checking if the locale actually exists on the system before applying it.

Minor Improvements

4. Extract regex pattern as a constant - The regex pattern could be defined as a constant at the class level for better maintainability.

5. Add logging for locale detection - It would be helpful to log which locale was detected and applied, especially for debugging encoding issues.

6. Consider caching the locale detection - Since the system locale rarely changes during runtime, the UTF-8 locale could be computed once and cached rather than recalculated on every command execution.

Testing

Great test coverage! Consider adding test cases for locales with variants (e.g., en_US.UTF-8@euro) to ensure edge cases are handled correctly.

Overall, the changes are focused and directly address the reported issue without introducing unrelated modifications.

[daniel-lxs]

Closing this PR as the issue cannot be reproduced on the main branch. The encoding handling appears to be working correctly in the current version. Please see the comment on issue #6768 for follow-up.