- This repository uses Yarn 4 (Berry) in classic
node_modulesmode (Plug'n'Play is disabled). Install dependencies withyarn install --immutable. - A
postinstallDevscript runs Beemo with the@niieani/scaffoldconfiguration to materialize local config files after install. If you ever need to regenerate those scaffolds manually, runyarn postinstallDev. - The primary verification flow matches the CI workflow in
.github/workflows/ci-cd.yml: runyarn buildfollowed byyarn testwhen practical. Theyarn testcommand runs formatting, type-checking, linting, and unit tests. - Use the existing formatting and linting tools instead of manual formatting. Prettier and ESLint configurations are already provided. Make sure to run
yarn formatbefore committing changes.
To run tests, or run any command that requires the internet, you must execute the task with {with_escalated_permissions: true}.
- The following outputs are generated by scripts and must not be edited manually:
src/bpeRanks/**/*.js— generated from the.tiktokenfiles indata/viayarn codegen:bpe(src/codegen/generateJsBpe.ts).src/model/**/*.ts— generated from the encodings insrc/encoding/viayarn codegen:models(src/codegen/generateByModel.ts).src/modelsChatEnabled.gen.ts— generated fromsrc/models.tsviayarn codegen:chat-enabled(src/codegen/generateChatEnabled.ts).
- Generated files now contain a header comment describing the owning script and the command to regenerate them. Always modify the source templates or data (e.g. files under
src/codegen/,src/encoding/, ordata/) and rerun the corresponding script instead of editing generated outputs directly.
- TypeScript source lives under
src/. Common entry points aresrc/main.tsand the modules insrc/encoding/. - Tests are written with Vitest (
yarn test:code). Snapshot files reside next to their test files insidesrc/__snapshots__. - Keep the repository compatible with both ESM and CommonJS builds; avoid introducing Node-specific globals without appropriate guards.
- Include clear descriptions of functional changes and reference any relevant commands run locally.
- When adding new scripts or codegen steps, update this file if they produce additional generated artifacts.