Skip to content

Comments

Normalize CRLF; add English (en_version) support and language flags#15

Draft
Cbphcr wants to merge 1 commit intomainfrom
codex/update-code-for-multilingual-database-support
Draft

Normalize CRLF; add English (en_version) support and language flags#15
Cbphcr wants to merge 1 commit intomainfrom
codex/update-code-for-multilingual-database-support

Conversation

@Cbphcr
Copy link
Contributor

@Cbphcr Cbphcr commented Feb 13, 2026

Motivation

  • Reduce noisy diffs by restoring original CRLF endings for environment files that were unintentionally converted to LF.
  • Add language support so dataset loading and environment tools can use English variants of data files and return English messages when requested.
  • Make the environment and tooling configurable via a --lang flag to run experiments/evaluations in zh or en mode.
  • Fix small API/behavior issues discovered while enabling language variants (string parsing, closed/open handling, and data mapping).

Description

  • Restored CRLF line endings for: chinatravel/environment/world_env.py, chinatravel/environment/tools/attractions/apis.py, chinatravel/environment/tools/restaurants/apis.py, and chinatravel/environment/tools/poi/apis.py without changing logic.
  • Added language handling in dataset loader: new helper functions _get_language and _normalize_query_language in chinatravel/data/load_datasets.py to swap language-specific fields (e.g. *_en) into base keys when lang == 'en'.
  • Introduced en_version flags to the environment and tool classes (WorldEnv, Poi, Attractions, Accommodations, Restaurants, IntercityTransport, Transportation) and used a file_suffix = "_en" convention to load language-specific files when en_version=True.
  • Propagated language selection through CLI and runtime: added --lang argument to run_exp.py, run_tpc.py, eval_exp.py, and eval_tpc.py, and pass en_version=args.lang == 'en' into WorldEnv construction.
  • Improved IntercityTransport by keeping city name mappings (city_cn_to_en, city_en_to_cn) and accepting either Chinese or English city names in select.
  • Made Poi.search return English or Chinese error messages based on en_version, and adjusted metro-station labeling in Transportation.goto to use English labels when en_version is enabled.
  • Fixed Restaurants.id_is_open to treat both Chinese "不营业" and English "closed" as closed, and ensured hard_logic_py from the HF dataset is ast.literal_eval-parsed only when it's a string.
  • Replaced several pop-based transfers with assignments to avoid mutating intermediate maps unnecessarily.
  • Commit message: Normalize line endings for env files back to CRLF for the EOL normalization commit.

Testing

  • Verified EOL differences are non-functional using git diff --ignore-cr-at-eol which showed no content changes for the normalized files (success).
  • Ran python -m compileall on the touched environment files which completed without errors (success).
  • Ran custom scripts to inspect and convert line endings (python one-liners used to count and rewrite EOLs) and validated the conversions (success).
  • Confirmed repository status and made a commit with the normalization changes (commit recorded).

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant