Skip to content

Commit cd55d40

Browse files
committed
chore: plug 3rd party tools benchmarks
1 parent e8c27d9 commit cd55d40

File tree

3 files changed

+47
-1
lines changed

3 files changed

+47
-1
lines changed

benchmark/README.md

Lines changed: 36 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,39 +35,73 @@ Until Claude Code adopts a persistent-process approach, this benchmark measures
3535
- [**git-aware-statusline.sh**](https://docs.claude.com/en/docs/claude-code/statusline#git-aware-status-line) (@Anthropic) - Bash, jq, Git
3636
- [**python-example.py**](https://docs.claude.com/en/docs/claude-code/statusline#python-example) (@Anthropic) - Python
3737
- [**nodejs-example.js**](https://docs.claude.com/en/docs/claude-code/statusline#node-js-example) (@Anthropic) - Node.js
38+
- [**ccusage**](https://github.com/ryoppippi/ccusage) (@ryoppippi) - Node.js
39+
- [**ccstatusline**](https://github.com/sirmalloc/ccstatusline) (@sirmalloc) - Node.js
40+
41+
## ⚠️ Warning
42+
Unlike the reference implementations that only process JSON from stdin, `ccusage` and `ccstatusline` also read configuration files from `$HOME/.claude` and may make API calls. The performance measurements for these tools reflect their full behavior and may vary depending on your system configuration, network latency, and Claude API response times.
43+
44+
Measuring the performance of these tools is out of scope for this benchmark; technically one could point `$CLAUDE_CONFIG_DIR` to a synthetic directory and have a file like `../docs/actual_stdin.json` point to it. This would lead to reproducible results.
3845

3946
## Execution Time
4047

4148
| Implementation | Author | Stack | Avg | Min | Max | vs Baseline (Avg) |
4249
|----------------|--------|-------|-----|-----|-----|-------------------|
4350
| [mini-ccstatus](https://github.com/meeghele/mini-ccstatus) | meeghele | C, cJSON | 688.3 µs | 362.8 µs | 1.2 ms | **baseline** |
51+
| **Examples** | | | | | | | |
4452
| [simple-statusline.sh](https://docs.claude.com/en/docs/claude-code/statusline#simple-status-line) | Anthropic | Bash, jq | 8.7 ms | 6.3 ms | 14.6 ms | **12.7x** |
4553
| [helper-function-statusline.sh](https://docs.claude.com/en/docs/claude-code/statusline#helper-function-approach) | Anthropic | Bash, jq | 11.3 ms | 6.6 ms | 25.0 ms | **16.4x** |
4654
| [git-aware-statusline.sh](https://docs.claude.com/en/docs/claude-code/statusline#git-aware-status-line) | Anthropic | Bash, jq, Git | 16.6 ms | 9.4 ms | 32.2 ms | **24.2x** |
4755
| [python-example.py](https://docs.claude.com/en/docs/claude-code/statusline#python-example) | Anthropic | Python | 19.3 ms | 13.8 ms | 38.2 ms | **28.0x** |
4856
| [nodejs-example.js](https://docs.claude.com/en/docs/claude-code/statusline#node-js-example) | Anthropic | Node.js | 23.6 ms | 17.3 ms | 40.0 ms | **34.3x** |
57+
| **Third-party tools** | | | | | | |
58+
| [ccstatusline (direct)](https://github.com/sirmalloc/ccstatusline) | sirmalloc | Node.js | 193.4 ms | 169.9 ms | 217.7 ms | **281.0x** |
59+
| [ccusage statusline --offline (direct)](https://github.com/ryoppippi/ccusage) | ryoppippi | Node.js | 205.5 ms | 51.9 ms | 3.0 s | **298.6x** |
60+
| [ccusage statusline (direct)](https://github.com/ryoppippi/ccusage) | ryoppippi | Node.js | 252.8 ms | 51.8 ms | 3.1 s | **367.3x** |
61+
| **Third-party tools with `bunx`/`npx`** | | | | | | |
62+
| [ccstatusline (bunx)](https://github.com/sirmalloc/ccstatusline) | sirmalloc | Bun, bunx | 669.8 ms | 527.0 ms | 1.6 s | **973.1x** |
63+
| [ccusage statusline --offline (npx)](https://github.com/ryoppippi/ccusage) | ryoppippi | Node.js, npx | 873.6 ms | 210.9 ms | 3.1 s | **1269.2x** |
64+
| [ccusage statusline (npx)](https://github.com/ryoppippi/ccusage) | ryoppippi | Node.js, npx | 704.5 ms | 208.9 ms | 3.2 s | **1023.5x** |
4965

5066
## CPU Cycles
5167

5268
| Implementation | Author | Stack | Cycles | Instructions | IPC | Cache Miss Rate | vs Baseline (Cycles) |
5369
|----------------|--------|-------|--------|--------------|-----|-----------------|----------------------|
5470
| [mini-ccstatus](https://github.com/meeghele/mini-ccstatus) | meeghele | C, cJSON | 1,276,362 | 1,741,398 | 1.36 | 1.39% | **baseline** |
71+
| **Examples** | | | | | | | |
5572
| [simple-statusline.sh](https://docs.claude.com/en/docs/claude-code/statusline#simple-status-line) | Anthropic | Bash, jq | 13,520,602 | 26,933,264 | 1.99 | 7.80% | **10.6x** |
5673
| [helper-function-statusline.sh](https://docs.claude.com/en/docs/claude-code/statusline#helper-function-approach) | Anthropic | Bash, jq | 13,933,142 | 28,540,923 | 2.05 | 5.63% | **10.9x** |
5774
| [git-aware-statusline.sh](https://docs.claude.com/en/docs/claude-code/statusline#git-aware-status-line) | Anthropic | Bash, jq, Git | 16,830,378 | 32,010,846 | 1.90 | 7.93% | **13.2x** |
5875
| [python-example.py](https://docs.claude.com/en/docs/claude-code/statusline#python-example) | Anthropic | Python | 53,595,859 | 74,758,019 | 1.39 | 6.82% | **42.0x** |
5976
| [nodejs-example.js](https://docs.claude.com/en/docs/claude-code/statusline#node-js-example) | Anthropic | Node.js | 58,988,313 | 122,894,612 | 2.08 | 29.16% | **46.2x** |
77+
| **Third-party tools** | | | | | | | |
78+
| [ccstatusline (direct)](https://github.com/sirmalloc/ccstatusline) | sirmalloc | Node.js | 557,996,963 | 1,263,271,861 | 2.26 | 18.11% | **437.2x** |
79+
| [ccusage statusline --offline (direct)](https://github.com/ryoppippi/ccusage) | ryoppippi | Node.js | 743,941,587 | 2,192,976,337 | 2.95 | 24.74% | **582.9x** |
80+
| [ccusage statusline (direct)](https://github.com/ryoppippi/ccusage) | ryoppippi | Node.js | 780,058,704 | 2,311,189,217 | 2.96 | 24.76% | **611.2x** |
81+
| **Third-party tools with `bunx`/`npx`** | | | | | | | |
82+
| [ccstatusline (bunx)](https://github.com/sirmalloc/ccstatusline) | sirmalloc | Bun, bunx | 599,324,634 | 1,357,631,857 | 2.27 | 18.25% | **469.6x** |
83+
| [ccusage statusline --offline (npx)](https://github.com/ryoppippi/ccusage) | ryoppippi | Node.js, npx | 5,607,044,575 | 16,703,413,501 | 2.98 | 24.99% | **4393.0x** |
84+
| [ccusage statusline (npx)](https://github.com/ryoppippi/ccusage) | ryoppippi | Node.js, npx | 5,607,945,761 | 16,713,890,173 | 2.98 | 24.92% | **4393.7x** |
6085

6186
## Memory Usage
6287

6388
| Implementation | Author | Stack | Avg KB | Avg MB | vs Baseline |
6489
|----------------|--------|-------|--------|--------|-------------|
6590
| [mini-ccstatus](https://github.com/meeghele/mini-ccstatus) | meeghele | C, cJSON | 1669 | 1 | **baseline** |
91+
| **Examples** | | | | | | | |
6692
| [simple-statusline.sh](https://docs.claude.com/en/docs/claude-code/statusline#simple-status-line) | Anthropic | Bash, jq | 3572 | 3 | **2.1x** |
6793
| [helper-function-statusline.sh](https://docs.claude.com/en/docs/claude-code/statusline#helper-function-approach) | Anthropic | Bash, jq | 3562 | 3 | **2.1x** |
6894
| [git-aware-statusline.sh](https://docs.claude.com/en/docs/claude-code/statusline#git-aware-status-line) | Anthropic | Bash, jq, Git | 4632 | 4 | **2.8x** |
6995
| [python-example.py](https://docs.claude.com/en/docs/claude-code/statusline#python-example) | Anthropic | Python | 13297 | 12 | **8.0x** |
7096
| [nodejs-example.js](https://docs.claude.com/en/docs/claude-code/statusline#node-js-example) | Anthropic | Node.js | 46711 | 45 | **28.0x** |
97+
| **Third-party tools** | | | | | |
98+
| [ccstatusline (direct)](https://github.com/sirmalloc/ccstatusline) | sirmalloc | Node.js | 86376 | 84 | **51.8x** |
99+
| [ccusage statusline --offline (direct)](https://github.com/ryoppippi/ccusage) | ryoppippi | Node.js | 108568 | 106 | **65.0x** |
100+
| [ccusage statusline (direct)](https://github.com/ryoppippi/ccusage) | ryoppippi | Node.js | 116568 | 113 | **69.8x** |
101+
| **Third-party tools with `bunx`/`npx`** | | | | | |
102+
| [ccstatusline (bunx)](https://github.com/sirmalloc/ccstatusline) | sirmalloc | Bun, bunx | 86329 | 84 | **51.7x** |
103+
| [ccusage statusline --offline (npx)](https://github.com/ryoppippi/ccusage) | ryoppippi | Node.js, npx | 249868 | 244 | **149.7x** |
104+
| [ccusage statusline (npx)](https://github.com/ryoppippi/ccusage) | ryoppippi | Node.js, npx | 245819 | 240 | **147.3x** |
71105

72106
## Configuration
73107

@@ -91,6 +125,7 @@ Until Claude Code adopts a persistent-process approach, this benchmark measures
91125

92126
```bash
93127
sudo dnf install hyperfine time jq python3 nodejs perf
128+
npm install -g ccusage ccstatusline
94129
```
95130

96131
#### Note
@@ -119,7 +154,7 @@ Show peak RSS (Resident Set Size)
119154

120155
#### Test data
121156

122-
All tests use the same input data from `docs/actual_stdin.json`
157+
All tests use the same input data from `../docs/actual_stdin.json`
123158

124159
### No Shell Overhead
125160

benchmark/data/commands.txt

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,13 @@ helper-function-statusline.sh|Anthropic|Bash, jq|docs.claude.com/en/docs/claude-
1111
git-aware-statusline.sh|Anthropic|Bash, jq, Git|docs.claude.com/en/docs/claude-code/statusline#git-aware-status-line|examples/git-aware-statusline.sh|direct
1212
python-example.py|Anthropic|Python|docs.claude.com/en/docs/claude-code/statusline#python-example|examples/python-example.py|python
1313
nodejs-example.js|Anthropic|Node.js|docs.claude.com/en/docs/claude-code/statusline#node-js-example|examples/nodejs-example.js|node
14+
15+
# Community implementations: Direct command invocations (require global install via npm install -g)
16+
ccstatusline (direct)|sirmalloc|Node.js|github.com/sirmalloc/ccstatusline|ccstatusline statusline|system
17+
ccusage statusline (direct)|ryoppippi|Node.js|github.com/ryoppippi/ccusage|ccusage statusline|system
18+
ccusage statusline --offline (direct)|ryoppippi|Node.js|github.com/ryoppippi/ccusage|ccusage statusline --offline|system
19+
20+
# Community implementations: Indirect command invocations (require global install via npm install -g)
21+
ccstatusline (bunx)|sirmalloc|Bun, bunx|github.com/sirmalloc/ccstatusline|bunx ccstatusline@latest|system
22+
ccusage statusline (npx)|ryoppippi|Node.js, npx|github.com/ryoppippi/ccusage|npx ccusage statusline|system
23+
ccusage statusline --offline (npx)|ryoppippi|Node.js, npx|github.com/ryoppippi/ccusage|npx ccusage statusline --offline|system

benchmark/data/notes.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ Required tools (Fedora):
66

77
```bash
88
sudo dnf install hyperfine time jq python3 nodejs perf
9+
npm install -g ccusage ccstatusline
910
```
1011

1112
#### Note

0 commit comments

Comments
 (0)