Skip to content

Commit 6db7c72

Browse files
committed
feat: huge perf updates
1 parent dbff4aa commit 6db7c72

20 files changed

+2571
-310
lines changed

β€Ž.npmignoreβ€Ž

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,13 @@
11
# Tests
2+
test.ts
23
test.js
34
test/
5+
test-workers/
46
*.test.js
7+
*.test.ts
58
*.spec.js
9+
*.spec.ts
10+
tsconfig.test.json
611

712
# Docs (README is included automatically)
813
DOCS.md
@@ -13,6 +18,9 @@ DOCS.md
1318
.eslintrc*
1419
.prettierrc*
1520
tsconfig.json
21+
biome.json
22+
bun.lock
23+
bun.lockb
1624

1725
# IDE
1826
.vscode/
@@ -26,3 +34,16 @@ coverage/
2634
.git/
2735
.github/
2836

37+
# Benchmarks (includes Docker files)
38+
benchmarks/
39+
40+
# Source TypeScript files (dist/ is published, keep .d.ts)
41+
src/
42+
*.ts
43+
!*.d.ts
44+
45+
# Misc
46+
*.log
47+
.DS_Store
48+
Thumbs.db
49+

β€ŽDOCS.mdβ€Ž

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1127,7 +1127,26 @@ const user = await beeThreads.worker<typeof findUser>('./workers/find-user')(123
11271127
| Real-time event handling | `turbo()` |
11281128
| Cron jobs | `max()` |
11291129

1130-
### 11. Why security by default?
1130+
### 11. Why AutoPack for object serialization?
1131+
1132+
**Decision:** Use AutoPack to convert arrays of objects into TypedArrays before postMessage transfer, providing 1.5-10x faster serialization than structuredClone.
1133+
1134+
**Performance benchmarks (1M objects, Node.js):**
1135+
1136+
| Object Type | structuredClone | AutoPack | Speedup |
1137+
|-------------|-----------------|----------|---------|
1138+
| Numeric (5 fields) | 2275ms | 698ms | **3.3x** |
1139+
| String (2 fields) | 2150ms | 1283ms | **1.7x** |
1140+
| Mixed (num+str+bool) | 2114ms | 863ms | **2.4x** |
1141+
| Nested (2 levels) | 5010ms | 1709ms | **2.9x** |
1142+
1143+
**When AutoPack is used:**
1144+
1145+
- `autoPack: 'auto'` (default): Enabled when array has 50+ objects
1146+
- `autoPack: true`: Always use, throws if data not compatible
1147+
- `autoPack: false`: Never use, fallback to structuredClone
1148+
1149+
### 12. Why security by default?
11311150

11321151
**Decision:** Enable security protections by default with opt-out config.
11331152

β€ŽREADME.mdβ€Ž

Lines changed: 10 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -251,60 +251,16 @@ if (result.status === 'fulfilled') console.log(result.value)
251251

252252
---
253253

254-
## Benchmarks
254+
## When to Use
255255

256-
```bash
257-
bun benchmarks.js # Bun
258-
node benchmarks.js # Node
259-
```
260-
261-
### Results (1M items, heavy computation, 12 CPUs, 10 runs average)
262-
263-
**Windows**
264-
265-
| Runtime | Mode | Time (Β±std) | vs Main | Main Thread |
266-
| ------- | ---------- | ------------- | --------- | ----------- |
267-
| Bun | main | 285 ± 5ms | 1.00x | ❌ blocked |
268-
| Bun | bee | 1138 Β± 51ms | 0.25x | βœ… free |
269-
| Bun | turbo(8) | 180 Β± 8ms | 1.58x | βœ… free |
270-
| Bun | turbo(12) | **156 Β± 12ms**| **1.83x** | βœ… free |
271-
| Node | main | 368 ± 13ms | 1.00x | ❌ blocked |
272-
| Node | bee | 5569 Β± 203ms | 0.07x | βœ… free |
273-
| Node | turbo(8) | 1052 Β± 22ms | 0.35x | βœ… free |
274-
| Node | turbo(12) | 1017 Β± 57ms | 0.36x | βœ… free |
275-
276-
**Linux (Docker)**
277-
278-
| Runtime | Mode | Time (Β±std) | vs Main | Main Thread |
279-
| ------- | ---------- | ------------- | --------- | ----------- |
280-
| Bun | main | 338 ± 8ms | 1.00x | ❌ blocked |
281-
| Bun | bee | 1882 Β± 64ms | 0.18x | βœ… free |
282-
| Bun | turbo(8) | 226 Β± 7ms | 1.50x | βœ… free |
283-
| Bun | turbo(12) | **213 Β± 20ms**| **1.59x** | βœ… free |
284-
| Node | main | 522 ± 54ms | 1.00x | ❌ blocked |
285-
| Node | bee | 5520 Β± 163ms | 0.09x | βœ… free |
286-
| Node | turbo(8) | 953 Β± 44ms | 0.55x | βœ… free |
287-
| Node | turbo(12) | **861 Β± 64ms**| **0.61x** | βœ… free |
288-
289-
### Key Insights
290-
291-
| Insight | Explanation |
292-
|---------|-------------|
293-
| **Bun + turbo = real speedup** | 1.6-1.8x faster than main thread |
294-
| **Node + turbo = non-blocking** | Main thread free for HTTP/events |
295-
| **Linux > Windows** | Node performs ~40% better on Linux |
296-
| **turbo >> bee for arrays** | 7x faster for large array processing |
297-
| **Default workers** | `os.cpus() - 1` is safe for all systems |
298-
299-
### When to Use
300-
301-
| Scenario | Recommendation |
302-
| -------------------------- | ----------------------------------- |
303-
| Bun + heavy computation | `turbo(cpus)` β†’ real parallelism |
304-
| Node + HTTP server | `turbo()` β†’ non-blocking I/O |
305-
| Light function (`x * x`) | Main thread β†’ overhead not worth it |
306-
| CLI/batch processing | `turbo(cpus + 4)` β†’ max throughput |
307-
| Database + large arrays | `worker().turbo()` β†’ best of both |
256+
| Your Scenario | Best Choice | Why |
257+
|---------------|-------------|-----|
258+
| **Arrays** | `turbo()` | SharedArrayBuffer = zero-copy |
259+
| **Single heavy function** | `bee()` | Keeps main thread free |
260+
| **Light function** (`x * 2`) | Main thread | Overhead > benefit |
261+
| **Need `require()`/DB** | `worker()` | Full Node.js access |
262+
| **Batch processing with full import features** | `worker().turbo()` | Parallel + DB access |
263+
| **HTTP server** | `turbo()` | Non-blocking for requests |
308264

309265
---
310266

@@ -317,6 +273,7 @@ node benchmarks.js # Node
317273
- **Worker affinity** - V8 JIT optimization
318274
- **Request coalescing** - Deduplicates identical calls
319275
- **Turbo mode** - Parallel array processing
276+
- **AutoPack** - 2-3x faster object serialization
320277
- **File workers** - External files with `require()` + turbo
321278
- **Full TypeScript** - Complete type inference
322279

β€Žbenchmarks/Dockerfileβ€Ž

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# bee-threads Benchmark Dockerfile
2+
# Runs benchmarks on both Node.js and Bun
3+
4+
FROM node:22-slim
5+
WORKDIR /app
6+
7+
# Install Bun
8+
RUN apt-get update && apt-get install -y curl unzip && \
9+
curl -fsSL https://bun.sh/install | bash && \
10+
apt-get clean && rm -rf /var/lib/apt/lists/*
11+
ENV PATH="/root/.bun/bin:$PATH"
12+
13+
# Copy package files from parent directory
14+
COPY package.json package-lock.json* ./
15+
RUN npm install
16+
17+
# Copy source files
18+
COPY src ./src
19+
COPY tsconfig.json ./
20+
21+
# Copy benchmark files
22+
COPY benchmarks ./benchmarks
23+
24+
# Build TypeScript
25+
RUN npm run build
26+
27+
# Make benchmark script executable (convert CRLF to LF)
28+
COPY benchmarks/run-benchmarks.sh ./run-benchmarks.sh
29+
RUN sed -i 's/\r$//' run-benchmarks.sh && chmod +x run-benchmarks.sh
30+
31+
CMD ["./run-benchmarks.sh"]
Lines changed: 198 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,198 @@
1+
/**
2+
* Benchmark: AutoPack para chamadas ÚNICAS (bee normal, não turbo)
3+
*
4+
* Pergunta: Vale a pena usar AutoPack no bee() normal?
5+
*
6+
* CenΓ‘rios:
7+
* 1. Objeto pequeno (10 campos)
8+
* 2. Objeto mΓ©dio (100 campos)
9+
* 3. Objeto grande (1000 campos)
10+
* 4. Array pequeno (100 objetos)
11+
* 5. Array mΓ©dio (1000 objetos)
12+
* 6. Array grande (10000 objetos)
13+
*/
14+
15+
import { autoPack, autoUnpack, clearAutoPackCaches } from '../src/autopack';
16+
17+
// ============ DATA GENERATORS ============
18+
19+
function generateObject(fieldCount: number): Record<string, unknown> {
20+
const obj: Record<string, unknown> = {};
21+
for (let i = 0; i < fieldCount; i++) {
22+
if (i % 3 === 0) obj[`num_${i}`] = Math.random() * 1000;
23+
else if (i % 3 === 1) obj[`str_${i}`] = `value_${i}_${Math.random().toString(36).slice(2, 10)}`;
24+
else obj[`bool_${i}`] = i % 2 === 0;
25+
}
26+
return obj;
27+
}
28+
29+
function generateArray(size: number, fieldsPerObject: number): Record<string, unknown>[] {
30+
const arr: Record<string, unknown>[] = new Array(size);
31+
for (let i = 0; i < size; i++) {
32+
arr[i] = generateObject(fieldsPerObject);
33+
}
34+
return arr;
35+
}
36+
37+
// ============ BENCHMARK ============
38+
39+
interface BenchResult {
40+
scenario: string;
41+
dataSize: string;
42+
packUnpackTime: number;
43+
structuredCloneTime: number;
44+
ratio: number;
45+
verdict: string;
46+
}
47+
48+
function benchmarkSingle(
49+
name: string,
50+
data: Record<string, unknown> | Record<string, unknown>[],
51+
iterations: number = 100
52+
): BenchResult {
53+
clearAutoPackCaches();
54+
55+
const dataArray = Array.isArray(data) ? data : [data];
56+
57+
// Warmup
58+
for (let i = 0; i < 5; i++) {
59+
const packed = autoPack(dataArray);
60+
autoUnpack(packed);
61+
structuredClone(data);
62+
}
63+
64+
// Benchmark AutoPack (pack + unpack = roundtrip simulando postMessage)
65+
const packTimes: number[] = [];
66+
for (let i = 0; i < iterations; i++) {
67+
const start = performance.now();
68+
const packed = autoPack(dataArray);
69+
autoUnpack(packed);
70+
packTimes.push(performance.now() - start);
71+
}
72+
73+
// Benchmark structuredClone (simula o que postMessage faz)
74+
const cloneTimes: number[] = [];
75+
for (let i = 0; i < iterations; i++) {
76+
const start = performance.now();
77+
structuredClone(data);
78+
cloneTimes.push(performance.now() - start);
79+
}
80+
81+
// Mediana
82+
packTimes.sort((a, b) => a - b);
83+
cloneTimes.sort((a, b) => a - b);
84+
85+
const packTime = packTimes[Math.floor(iterations / 2)];
86+
const cloneTime = cloneTimes[Math.floor(iterations / 2)];
87+
const ratio = cloneTime / packTime;
88+
89+
// Calcular tamanho aproximado
90+
const jsonSize = JSON.stringify(data).length;
91+
const sizeStr = jsonSize < 1024
92+
? `${jsonSize} B`
93+
: jsonSize < 1024 * 1024
94+
? `${(jsonSize / 1024).toFixed(1)} KB`
95+
: `${(jsonSize / 1024 / 1024).toFixed(1)} MB`;
96+
97+
const verdict = ratio >= 1.2
98+
? 'βœ… USAR AutoPack'
99+
: ratio >= 0.8
100+
? '⚠️ Similar'
101+
: '❌ NΓƒO usar';
102+
103+
return {
104+
scenario: name,
105+
dataSize: sizeStr,
106+
packUnpackTime: packTime,
107+
structuredCloneTime: cloneTime,
108+
ratio,
109+
verdict
110+
};
111+
}
112+
113+
async function main() {
114+
console.log('╔══════════════════════════════════════════════════════════════════════════════╗');
115+
console.log('β•‘ AUTOPACK PARA CHAMADAS ÚNICAS (bee normal) - VALE A PENA? β•‘');
116+
console.log('β•‘ β•‘');
117+
console.log('β•‘ Comparando: autoPack + autoUnpack vs structuredClone β•‘');
118+
console.log('β•‘ (structuredClone Γ© o que postMessage usa internamente) β•‘');
119+
console.log('β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•\n');
120+
121+
const results: BenchResult[] = [];
122+
123+
// ============ OBJETOS ÚNICOS ============
124+
console.log('πŸ“¦ OBJETOS ÚNICOS (1 objeto, N campos)\n');
125+
126+
results.push(benchmarkSingle('1 objeto, 10 campos', generateObject(10)));
127+
results.push(benchmarkSingle('1 objeto, 50 campos', generateObject(50)));
128+
results.push(benchmarkSingle('1 objeto, 100 campos', generateObject(100)));
129+
results.push(benchmarkSingle('1 objeto, 500 campos', generateObject(500)));
130+
results.push(benchmarkSingle('1 objeto, 1000 campos', generateObject(1000)));
131+
132+
console.log('β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”');
133+
console.log('β”‚ CenΓ‘rio β”‚ Tamanho β”‚ AutoPack β”‚ structClone β”‚ Ratio β”‚ Veredicto β”‚');
134+
console.log('β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€');
135+
136+
for (const r of results.slice(0, 5)) {
137+
const scenario = r.scenario.padEnd(23);
138+
const size = r.dataSize.padStart(8);
139+
const pack = `${r.packUnpackTime.toFixed(3)}ms`.padStart(9);
140+
const clone = `${r.structuredCloneTime.toFixed(3)}ms`.padStart(13);
141+
const ratio = `${r.ratio.toFixed(2)}x`.padStart(7);
142+
const verdict = r.verdict.padEnd(16);
143+
console.log(`β”‚ ${scenario} β”‚ ${size} β”‚ ${pack} β”‚ ${clone} β”‚ ${ratio} β”‚ ${verdict} β”‚`);
144+
}
145+
console.log('β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜\n');
146+
147+
// ============ ARRAYS DE OBJETOS ============
148+
console.log('πŸ“¦ ARRAYS DE OBJETOS (N objetos, 10 campos cada)\n');
149+
150+
results.push(benchmarkSingle('10 objetos', generateArray(10, 10)));
151+
results.push(benchmarkSingle('50 objetos', generateArray(50, 10)));
152+
results.push(benchmarkSingle('100 objetos', generateArray(100, 10)));
153+
results.push(benchmarkSingle('500 objetos', generateArray(500, 10)));
154+
results.push(benchmarkSingle('1000 objetos', generateArray(1000, 10)));
155+
results.push(benchmarkSingle('5000 objetos', generateArray(5000, 10)));
156+
results.push(benchmarkSingle('10000 objetos', generateArray(10000, 10)));
157+
158+
console.log('β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”');
159+
console.log('β”‚ CenΓ‘rio β”‚ Tamanho β”‚ AutoPack β”‚ structClone β”‚ Ratio β”‚ Veredicto β”‚');
160+
console.log('β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€');
161+
162+
for (const r of results.slice(5)) {
163+
const scenario = r.scenario.padEnd(23);
164+
const size = r.dataSize.padStart(8);
165+
const pack = `${r.packUnpackTime.toFixed(3)}ms`.padStart(9);
166+
const clone = `${r.structuredCloneTime.toFixed(3)}ms`.padStart(13);
167+
const ratio = `${r.ratio.toFixed(2)}x`.padStart(7);
168+
const verdict = r.verdict.padEnd(16);
169+
console.log(`β”‚ ${scenario} β”‚ ${size} β”‚ ${pack} β”‚ ${clone} β”‚ ${ratio} β”‚ ${verdict} β”‚`);
170+
}
171+
console.log('β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜\n');
172+
173+
// ============ CONCLUSÃO ============
174+
console.log('╔══════════════════════════════════════════════════════════════════════════════╗');
175+
console.log('║ CONCLUSÃO ║');
176+
console.log('╠══════════════════════════════════════════════════════════════════════════════╣');
177+
console.log('β•‘ β•‘');
178+
console.log('β•‘ Para bee() NORMAL (chamadas ΓΊnicas): β•‘');
179+
console.log('β•‘ β•‘');
180+
console.log('β•‘ ❌ NΓƒO usar AutoPack para: β•‘');
181+
console.log('β•‘ - Objetos pequenos (< 100 campos) β•‘');
182+
console.log('β•‘ - Arrays pequenos (< 100 objetos) β•‘');
183+
console.log('β•‘ - Qualquer dado < 10KB β•‘');
184+
console.log('β•‘ β•‘');
185+
console.log('β•‘ βœ… USAR AutoPack para: β•‘');
186+
console.log('β•‘ - Arrays com 500+ objetos β•‘');
187+
console.log('β•‘ - Objetos com 500+ campos β•‘');
188+
console.log('β•‘ - Qualquer dado > 50KB β•‘');
189+
console.log('β•‘ β•‘');
190+
console.log('β•‘ πŸ“Œ RECOMENDAÇÃO: β•‘');
191+
console.log('║ - bee() normal: NÃO usar AutoPack (overhead não compensa) ║');
192+
console.log('β•‘ - turbo(): USAR AutoPack (arrays grandes, ganho significativo) β•‘');
193+
console.log('β•‘ β•‘');
194+
console.log('β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•');
195+
}
196+
197+
main().catch(console.error);
198+

0 commit comments

Comments
Β (0)