perf(cli): cut 221ms and 44MB RSS off every CLI invocation - #4463
Draft
Daksh (sudodaksh) wants to merge 2 commits into
Draft
perf(cli): cut 221ms and 44MB RSS off every CLI invocation#4463Daksh (sudodaksh) wants to merge 2 commits into
Daksh (sudodaksh) wants to merge 2 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Preview this PR's installerThe hermetic install E2E suite passed for this commit. These commands run the PR's installer scripts against the latest published CLI release — they preview installer behavior, not unreleased binaries. curl -fsSL "https://raw.githubusercontent.com/ComposioHQ/composio/f0872d039d719df08e0ec960a62c044ec02597b3/install.sh" | sh
curl -fsSL "https://raw.githubusercontent.com/ComposioHQ/composio/f0872d039d719df08e0ec960a62c044ec02597b3/install.sh" | COMPOSIO_INSTALL_SHELL=none shThe first command is the default flow: it installs the CLI and configures your shell automatically. The second installs only, without touching shell startup files. Shell-specific setup routes (the override points the variant at this PR's base installer): curl -fsSL "https://raw.githubusercontent.com/ComposioHQ/composio/f0872d039d719df08e0ec960a62c044ec02597b3/install/bash.sh" | COMPOSIO_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/ComposioHQ/composio/f0872d039d719df08e0ec960a62c044ec02597b3/install.sh" sh
curl -fsSL "https://raw.githubusercontent.com/ComposioHQ/composio/f0872d039d719df08e0ec960a62c044ec02597b3/install/zsh.sh" | COMPOSIO_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/ComposioHQ/composio/f0872d039d719df08e0ec960a62c044ec02597b3/install.sh" sh
curl -fsSL "https://raw.githubusercontent.com/ComposioHQ/composio/f0872d039d719df08e0ec960a62c044ec02597b3/install/fish.sh" | COMPOSIO_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/ComposioHQ/composio/f0872d039d719df08e0ec960a62c044ec02597b3/install.sh" sh |
A compiled Bun binary parses its whole embedded module graph before the
first line of JS runs, so bundled-but-unused code is paid for on every
invocation. Verified: a binary that bundles everything but evaluates only
console.log still costs ~235ms, against 15ms for a hello-world build.
Most of this bundle was code `composio execute` never reaches.
Measured on the pinned toolchain (Bun 1.4.1+4661e494f, linux-x64,
best of 7, analytics disabled):
composio --version 970ms -> 749ms (-221ms)
peak RSS 175.8MB -> 132.2MB (-43.6MB)
compiled binary 96MB -> 86MB
bundle 31.24MB -> 15.84MB
Four changes.
run.cmd.ts imported `ts` from ts-morph, which vendors its own copy of the
TypeScript compiler, so the binary carried two of them. The file uses only
createSourceFile, forEachChild, ScriptTarget, ScriptKind and five isX
guards, all available in the typescript copy that
src/generation/typescript/* already pulls in. Sharing one compiler also
means commands/generate and commands/run.cmd no longer evaluate a compiler
each.
js-tiktoken's main entry statically inlines all six BPE rank tables (gpt2,
r50k, p50k, p50k_edit, cl100k, o200k) as string literals; the CLI only ever
uses o200k, via encodingForModel('gpt-4o'). The lite build with that single
table produces identical token-id streams.
prepareExecuteOutput built the o200k rank table on every successful
execute, purely to compare the response against a 10k-token threshold.
Constructing it measured ~390ms in a compiled binary on the pinned
toolchain (390.1, 367.1, 418.6ms across three runs), against ~4ms to
encode a 7.5KB payload once the table exists. A BPE token always covers at
least one UTF-8 byte, so a payload of at most THRESHOLD bytes can never
exceed THRESHOLD tokens; checking byte length first reaches the same
decision without the tokenizer.
That ~390ms is construction cost measured in isolation, not an end-to-end
delta on a real `composio execute`. No credentialed run was available to
measure the whole command before and after, so treat it as the size of the
work removed from the success path rather than a verified wall-clock
saving. `COMPOSIO_PERF_DEBUG=1` reports the gap between
`execute.tool_call.end` and exit for anyone able to run it for real.
ToolsExecutorLive resolved a client through clientSingleton.get()
unconditionally, walking the project context off disk, then discarded it
because every caller on the remote-execute path passes one in.
Three behavioral deltas, none of them the tokenization result or the
inline/file decision:
1. Tiktoken.encode() throws on the literals <|endoftext|> and
<|endofprompt|> appearing anywhere in the response, at any size (a
66-byte payload reproduces it). That throw landed after
"Execution successful" had printed, so the tool ran and the CLI still
exited 1 with nothing on stdout. Responses at or under 10KB no longer
reach encode(), so they now succeed. Larger responses still hit it;
the real fix is passing allowedSpecial 'all' and is not in this commit.
2. TypeScript 6.0.2 (ts-morph's vendored copy) to 6.0.3. Differential
tested: the three real parse helpers over 20 sources covering TSX,
decorators, `using`, `satisfies`, import attributes, optional-chained
calls and unicode gave identical output on all 60 comparisons.
3. Under COMPOSIO_LOG_LEVEL=Debug, ProjectContext's "resolved from ..."
debug lines no longer appear on the remote-execute path. The local-tool
path still calls get() and is unchanged.
Verified with typecheck:src, oxlint, validate:boundaries, the
tools.execute and run command suites (89 tests), and differential tests of
both tokenizers and both TypeScript versions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EzaE7oGVgziJ5nRvBhcci2
Daksh (sudodaksh)
force-pushed
the
claude/cli-binary-performance-c7xicz
branch
from
September 13, 2026 06:25
91dc388 to
bf44c34
Compare
This was referenced Sep 13, 2026
…s once Check the invocation origin before building the tokenizer, since composio run always prints inline, and pass the token count from the threshold check into the stored-output summary instead of encoding the payload twice. Add a test for a response past the byte pre-filter but under the token threshold, and drop the unused ts-morph dependency.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
composio --version: 970ms to 749ms. Peak RSS: 175.8MB to 132.2MB. Binary: 96MB to 86MB. No new dependencies (one unused one removed), no API changes.The compiled binary carried two copies of the TypeScript compiler and all six tiktoken rank tables. A compiled Bun binary parses everything it embeds before running any JS, so every command paid for that.
First PR in a stack of five. Review order: this, #4464, #4468, #4469, #4475.
Bun 1.4.1+4661e494f, linux-x64, best of 7, analytics disabled:
composio --versionChanges
run.cmd.tsimportedtsfrom ts-morph, which bundles its own TypeScript. It now uses thetypescriptpackage the generation code already pulls in. One compiler instead of two, 8.5MB each.js-tiktoken/litewith that one table. Token ids are identical.prepareExecuteOutputbuilt the rank table on every successful execute just to compare against a 10,000 token threshold. A token covers at least one byte, so a payload under 10,000 bytes cannot exceed 10,000 tokens. It checks bytes first and only builds the tokenizer past that. It also checks the invocation origin before building it, sincecomposio runalways prints inline, and a stored response is encoded once rather than once for the threshold and again fortokenCount.ToolsExecutorLiveresolved a client viaclientSingleton.get()(disk reads) and then discarded it, since every caller passes one in. Resolved lazily now.ts-morphis removed from the CLI's dependencies. Nothing imports it after change 1, and leaving it listed made it easy to bring its TypeScript copy back.What changes in behavior:
Tiktoken.encode(), so the<|endoftext|>crash stops happening for them. Larger responses still hit it. fix(cli): stop tiktoken special-token literals from failing execute #4464 is the real fix.composio runno longer build the tokenizer for responses over 10KB. Their output was always printed inline, so the count was discarded.COMPOSIO_LOG_LEVEL=Debug, ProjectContext's "resolved from" lines no longer print on the remote execute path.On change 3: an earlier version of this description said it saved ~500ms per execute, measured in isolation under a different Bun. Inside the compiled binary,
new Tiktoken(o200k)costs ~390ms andencode()of 7.5KB about 4ms. Measured end to end on a realHACKERNEWS_GET_ITEM_WITH_IDexecute withCOMPOSIO_PERF_DEBUG=1, the time fromexecute.tool_call.endto exit drops from ~300ms to ~15ms for responses under 10KB, and stays ~350ms above it.Type of change
It removes a crash, but by accident, so it is not marked as a bug fix.
How Has This Been Tested?
Bun 1.4.1+4661e494f, Node 24.17.0, pnpm 11.8.0, linux-x64.
cd ts/packages/cli && pnpm run typecheck && pnpm run validate:boundariespnpm exec vitest run test/src/commands/tools/tools.execute.cmd.test.ts test/src/commands/run.cmd.test.ts: 90 passed. A new case covers a ~18KB response that encodes to ~4k tokens, past the byte check but under the threshold, and asserts it stays inline.pnpm build:binary && time ./dist/composio --versionChecked but not committed: the three parse helpers give identical output under both compilers across 20 sources (TSX, decorators,
using,satisfies, import attributes, unicode). Lite tiktoken gives identical token id streams on 8 samples including CJK, RTL, emoji and control characters. Max tokens per byte was 0.846, under the 1.0 the byte check needs.Screenshots (if applicable)
Not applicable.
Checklist
No docs describe the bundle contents or the token threshold. The byte pre-filter boundary has a test. The compiler and tokenizer comparisons above are still not committed and should become a suite.
@composio/cliisprivate: true, so no changeset.Additional context
Bun 1.4.2 gives no gain over 1.4.1 (three rounds of best of 7: 723/732/756ms vs 744/713/747ms). Keep the pin.
Not touched: ~1.1s of execute preflight (5 to 7 serial round trips;
project/resolvehas no cache and can fire three times), the two-round-trip session create plus execute, and--skip-checks, which currently skips nothing measurable.🤖 Generated with Claude Code
https://claude.ai/code/session_01EzaE7oGVgziJ5nRvBhcci2