perf(cli): move the compiler and tokenizer out of the executable - #4469
Draft
Daksh (sudodaksh) wants to merge 2 commits into
Draft
perf(cli): move the compiler and tokenizer out of the executable#4469Daksh (sudodaksh) wants to merge 2 commits into
Daksh (sudodaksh) wants to merge 2 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Preview this PR's installerThe hermetic install E2E suite passed for this commit. These commands run the PR's installer scripts against the latest published CLI release — they preview installer behavior, not unreleased binaries. curl -fsSL "https://raw.githubusercontent.com/ComposioHQ/composio/96749f5aaae7be1d60c801cfd0d1cc36608a9f34/install.sh" | sh
curl -fsSL "https://raw.githubusercontent.com/ComposioHQ/composio/96749f5aaae7be1d60c801cfd0d1cc36608a9f34/install.sh" | COMPOSIO_INSTALL_SHELL=none shThe first command is the default flow: it installs the CLI and configures your shell automatically. The second installs only, without touching shell startup files. Shell-specific setup routes (the override points the variant at this PR's base installer): curl -fsSL "https://raw.githubusercontent.com/ComposioHQ/composio/96749f5aaae7be1d60c801cfd0d1cc36608a9f34/install/bash.sh" | COMPOSIO_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/ComposioHQ/composio/96749f5aaae7be1d60c801cfd0d1cc36608a9f34/install.sh" sh
curl -fsSL "https://raw.githubusercontent.com/ComposioHQ/composio/96749f5aaae7be1d60c801cfd0d1cc36608a9f34/install/zsh.sh" | COMPOSIO_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/ComposioHQ/composio/96749f5aaae7be1d60c801cfd0d1cc36608a9f34/install.sh" sh
curl -fsSL "https://raw.githubusercontent.com/ComposioHQ/composio/96749f5aaae7be1d60c801cfd0d1cc36608a9f34/install/fish.sh" | COMPOSIO_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/ComposioHQ/composio/96749f5aaae7be1d60c801cfd0d1cc36608a9f34/install.sh" sh |
Daksh (sudodaksh)
added this pull request to stack #4465
September 13, 2026 10:42
This was referenced Sep 13, 2026
`composio --version` goes from 288ms to 199ms, peak RSS from 97.8MB to 77.3MB, and the executable from 85.9MB to 79.7MB. Every command benefits. A compiled Bun binary parses its whole embedded bundle before the first line of JavaScript runs, and #4468 had already made sure the TypeScript compiler and the tokenizer rank table were never *evaluated* unless `generate`, `run`, or a large `execute` response needed them. They were still *parsed* on every start: the compiler alone was 44% of the executable's JavaScript and the o200k rank table another 28%, so `--version` spent ~75ms reading code it could never call. Both now ship as companion modules next to the executable, through the mechanism `composio run` already uses for its own runtime helpers: - `generation-runtime.mjs` carries `src/generation/*`, the `composio run` source rewrites, `typescript`, `@composio/ts-builders` and `openapi-typescript`. `generate ts`, `generate py` and `run` load it with `loadInstalledCompanionModule`; from a source checkout the loader resolves the `.ts` next to `run-companion-modules.ts` instead, so tests and `bun run src/bin.ts` need no build step. - `execute-output-encoder-runtime.mjs` carries `js-tiktoken/lite` and the rank table. `execute` loads it only once a response exceeds the 10KB byte pre-filter. A companion bundles its own copy of `effect`, and a fiber cannot run primitives built by another copy of the runtime, so nothing Effect-shaped crosses the boundary: the generation companion exposes plain functions and promises, runs its pipelines on its own runtime, and returns failures as values that `src/generation/errors.ts` rebuilds as the CLI's own error classes, stack included. Generated output is byte-identical to #4468 for `generate ts`, `generate ts --transpiled` and `generate py`. Both modules join `RUN_COMPANION_MODULE_BASENAMES`, so the build, release packaging, install verification, `upgrade` and the self-repair download pick them up unchanged. The three hand-maintained uninstall lists and the upgrade E2E fixture gain the two file names. Two smaller startup costs go with it: - `src/constants.ts` imported `constants` from `@composio/core`'s root entry for two strings and two URLs, which evaluated the whole SDK at startup (~25ms of module-scope work, mostly zod schemas). The four values are spelled out and pinned to core's by a test. - `tool-file-uploads.ts` imported three core helpers at module scope that only a file upload reaches; they are imported on that path now. The binary build gains a guard: after bundling the companions it bundles `src/bin.ts` once more unminified and fails if the executable's graph reaches `typescript`, `js-tiktoken`, core's root entry, `src/generation/*` or a companion entry. Without it a stray static import would put the compiler back into the executable with nothing to notice. Building also surfaced that `assertBundledRuntimeFiles` blanked string literals to same-length runs of spaces, which made the import patterns' `^\s*` backtrack quadratically across the compiler's multi-megabyte embedded lib strings and stalled the build for over ten minutes. String bodies are dropped now. (The check itself has never matched a specifier, since the specifiers it looks for are the string literals it removes; that is left as it was.) Measured on the pinned toolchain, Bun 1.4.1+4661e494f, linux-x64, best of 15, telemetry disabled, both binaries built in the same session: composio --version 288ms -> 199ms tools execute --help 287ms -> 202ms peak RSS 97.8MB -> 77.3MB executable 85.9MB -> 79.7MB executable JavaScript 8.3MB -> 2.1MB (minified) The `execute` tail after `execute.tool_call.end` is unchanged for responses under 10KB (~10ms) and ~20ms slower above it (351 -> 374ms), which is the on-demand parse of the 2.2MB encoder companion. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wx9gEjuiHux2weiHjdNcDs
…s to load A large execute measured its output after the tool call had already succeeded, so a missing encoder companion whose repair download failed turned a successful call into a failed command. Fall back to a byte-based token estimate instead. Load companion modules with Effect.tryPromise so an unloadable file is a typed RunCompanionRepairError rather than a defect, and add the two companions as tsdown entries so the dist build can resolve them. The executable graph check listed @composio/core's root entry with a pattern that could never match Bun's relative module paths. The root entry is still bundled behind the file-upload dynamic import, so drop that entry and correct the constants comment.
Alberto Schiabel (jkomyno)
force-pushed
the
claude/cli-startup-bundle-diet-c7xicz
branch
from
September 14, 2026 12:08
555e495 to
96749f5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
composio --version: 288ms to 199ms. Peak RSS: 97.8MB to 77.3MB. Executable: 85.9MB to 79.7MB. Every command benefits.A compiled Bun binary parses its whole embedded bundle before running any JS. #4468 stopped the TypeScript compiler and the tokenizer rank table from being evaluated at startup, but they were still parsed every time. The compiler was 44% of the executable's JavaScript, the o200k table another 28%. Both now ship as files next to the executable and load on demand.
Fourth PR in the stack. Stacked on #4468; review #4463, #4464 and #4468 first. #4475 builds on this one.
Bun 1.4.1+4661e494f, linux-x64, best of 15, telemetry disabled, both binaries built in the same session:
composio --versioncomposio tools execute --helpAcross the whole stack, from
next:--version612ms to 184ms, peak RSS 167MB to 78MB, executable 95.8MB to 79.7MB.composio executeend to end, against the live backend with a logged-in CLI, best of 7 for the small response and best of 5 for the large one. Tool:HACKERNEWS_GET_ITEM_WITH_ID(no connected account needed) andHACKERNEWS_GET_LATEST_POSTS. "Tail" is the time from theexecute.tool_call.endperf event to process exit.nextThe stack removes ~670ms from a small execute: ~430ms of startup and ~280ms of tokenizer construction that no longer happens. The large response keeps its ~330ms tail because past 10KB the tokenizer is still built; this PR adds ~20ms there for the on-demand parse of the encoder file. The remaining ~1.7s is network the stack does not touch: DNS and TLS to the backend, the preflight round trips before
tool_call.start, and the session create plus execute pair. Wall times move by ±150ms between runs because of that; the tail column is the stable one.Changes
generation-runtime.mjscarriessrc/generation/*, thecomposio runsource rewrites,typescript,@composio/ts-buildersandopenapi-typescript.generate ts,generate pyandrunload it with the newloadInstalledCompanionModule. From a source checkout the loader resolves the.tsfile next torun-companion-modules.ts, so tests andbun run src/bin.tsneed no build step. The specifier is computed at runtime on purpose; a literalimport('./x')gets folded back into the executable. A file that fails to import is a typedRunCompanionRepairErrorasking to reinstall, not a crash. Both companions are also tsdown entries, so thedist/build resolves them.execute-output-encoder-runtime.mjscarriesjs-tiktoken/liteplus the rank table.executeloads it only past the 10KB byte gate from perf(cli): cut 221ms and 44MB RSS off every CLI invocation #4463, and never for executes started bycomposio run. If it cannot be loaded, even after the self-repair download,executeestimates the token count from the byte length (about four bytes per token) instead of failing a tool call that already succeeded.RUN_COMPANION_MODULE_BASENAMES, the mechanismcomposio runalready uses for its helpers, so build, release packaging, install and upgrade verification, and the self-repair download pick them up unchanged. The three hand-maintained uninstall lists and the upgrade E2E fixture gain the two file names.effect, and a fiber cannot run primitives from another copy. So nothing Effect-shaped crosses the boundary. The generation companion exposes plain promises and returns failures as values.src/generation/errors.tsrebuilds them as the CLI's own error classes with fields and stack intact.src/constants.tsimportedconstantsfrom@composio/core's root entry for two strings and two URLs, which evaluated the whole SDK at startup (~25ms, mostly zod schemas). The values are inlined and a test pins them to core's.tool-file-uploads.tsimports its three core helpers on the upload path instead of at module scope.src/bin.tsonce more, unminified, and fails if the executable's graph reachestypescript,js-tiktoken,src/generation/*or a companion entry. Checked that it fires on a stray static import.@composio/core's root entry is not on the list: it is still bundled behind the file-upload path's dynamic import (see Additional context), so the guard cannot exclude it.What changes for users:
generatethe way it already affectedrun: self-repair from the release archive, then an error. A largeexecutealso attempts the repair, and if that fails it falls back to the byte-based estimate rather than failing. Responses under 10KB never touch the encoder.--versionand everything else are unaffected.composio upgradefrom a binary older than this PR copies only the companion files that binary knows about. The firstgenerate,runor largeexecuteon the new version then restores the two new files through the self-repair download.executeresponses over 10KB pay ~20ms more afterexecute.tool_call.end(351 to 374ms), the on-demand parse of the 2.2MB encoder file. Under 10KB, unchanged.Generated output is byte-identical to #4468 for
generate ts,generate ts --transpiledandgenerate py. The 11-invocation help/error diff from #4468 is identical.Found on the way:
assertBundledRuntimeFilesblanked string literals to same-length runs of spaces, and the import patterns'^\s*then backtracked quadratically over the compiler's embedded lib strings. The build hung for over ten minutes. String bodies are dropped now. The check has also never matched a specifier, since the specifiers it looks for are the strings it removes. Left as is, because a corrected version flags false positives inrun-subagent-output-mcp.Type of change
How Has This Been Tested?
Bun 1.4.1+4661e494f, Node 24.20.0, pnpm 11.8.0, linux-x64.
cd ts/packages/cli && pnpm run typecheck && pnpm run validate:boundaries && pnpm run validate:skillspnpm exec vitest run: 129 files, 1335 passed, 1 skipped. New tests cover the mirrored constants, error rehydration and outcome lifting, and the loader resolving both companions from source.pnpm build:binary, then againstdist/composio:generate ts,generate ts --transpiledandgenerate pydiffed against perf(cli): defer the TypeScript compiler and generation pipeline #4468's binary,runwith a trailing expression,executewith 1.6KB and 35KB responses, and the damaged-install cases with files deleted fromdist/.upgrade2 pass,run8 pass,version9 pass,install7 pass on bash and 5 pass on zsh. The install runs used a fixture built the way CI builds it (build:binary:cross,build:binary:package,build:binary:checksums), which also confirms the release zip carries both new files.bun run test/release-workflow.test.tsat the repo root, for the synced uninstall lists.pnpm run typecheck,validate:boundariesand oxlint pass. The execute, companion-loader, constants, generation-runtime,runandgeneratesuites pass (177 passed, 1 skipped), including new tests for the estimate when the encoder cannot load and for the typed load failure. The fallback test fails without the fix.pnpm buildemits both companions, andpnpm build:binarypasses the graph check.Screenshots (if applicable)
Not applicable.
Checklist
@composio/cliis private, so no changeset. Docs: the uninstall lists and the code generation section ofts/packages/cli/AGENTS.md.Additional context
openaiandpusher-js(~0.5MB minified) are still in the executable. Only@composio/core's root entry reaches them, and the two upload guards have no lighter subpath export. A@composio/core/utils/file-upload-guardentry would remove them; that is a core package change.Companion files carry no version stamp, so a companion left from another version is loaded as is. Checking
APP_VERSIONafter import would catch that; it is not done here.The remaining ~95ms of module evaluation is a long tail of eager Schema and command definitions across
src/commands,src/services,effectandsrc/models, not one dependency.🤖 Generated with Claude Code
https://claude.ai/code/session_01Wx9gEjuiHux2weiHjdNcDs