Skip to content

Commit 753dc36

Browse files
committed
adding hedging requests feature and update for the benchmarks + docs
1 parent 4ca1ef7 commit 753dc36

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

53 files changed

+4504
-907
lines changed

.github/workflows/deploy.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,10 @@ jobs:
5555
mkdir -p "$TARGET/dist"
5656
rsync -a dist/ "$TARGET/dist"/
5757
58+
echo "Copying benchmark/browser/ into release/benchmark/"
59+
mkdir -p "$TARGET/benchmark"
60+
rsync -a benchmark/browser/ "$TARGET/benchmark"/
61+
5862
# (No npm install here, release is pure static assets)
5963
6064
echo "Pointing 'current' symlink to new release"

README.md

Lines changed: 238 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,13 @@ Its domain-driven architecture and type-safe foundation make it ideal for enterp
1111
[![npm](https://img.shields.io/npm/v/luminara?style=flat-square&logo=npm)](https://www.npmjs.com/package/luminara)
1212
[![License](https://img.shields.io/badge/License-MIT-green?style=flat-square)](./LICENSE)
1313

14+
## 🔗 Links
15+
16+
- 🎨 **Interactive Sandbox And Documentation Website**: [luminara.website](https://luminara.website/)
17+
- 📦 **npm Package**: [npmjs.com/package/luminara](https://www.npmjs.com/package/luminara)
18+
- 🐙 **GitHub Repository**: [github.com/miller-28/luminara](https://github.com/miller-28/luminara)
19+
-**Performance Benchmarks**: [luminara.website/benchmark](https://luminara.website/benchmark/)
20+
1421
## ✨ Features
1522

1623
### Core Architecture
@@ -37,6 +44,7 @@ Its domain-driven architecture and type-safe foundation make it ideal for enterp
3744
### In-Flight Features (Request Execution - Phase 2)
3845
- ⏱️ **Configurable timeouts** - Request timeouts and abort controller support
3946
- 🔄 **Comprehensive retry system** - 6 backoff strategies (exponential, fibonacci, jitter, etc.)
47+
- 🏎️ **Request hedging** - Race and cancel-and-retry policies for latency optimization
4048

4149
### Post-Flight Features (Response Handlers - Phase 3)
4250
- 🎯 **Response type handling** - JSON, text, form data, binary support
@@ -47,6 +55,29 @@ Its domain-driven architecture and type-safe foundation make it ideal for enterp
4755

4856
---
4957

58+
## ✅ Battle-Tested Reliability
59+
60+
Luminara is validated by a **comprehensive test suite** covering all features and edge cases:
61+
62+
-**234 tests** across **16 test suites** (100% passing)
63+
- 🎯 **Programmatic validation** - Tests actual behavior, not just API contracts
64+
- 🧪 **Framework simulation** - React, Vue, Angular usage patterns
65+
- ⏱️ **Timing accuracy** - Backoff strategies validated to millisecond precision
66+
- 🛡️ **Error scenarios** - Comprehensive failure case coverage
67+
- 🔄 **Integration testing** - Feature combinations (retry + timeout + hedging)
68+
- 📊 **Real package testing** - Tests built distribution, not source files
69+
70+
**Test Categories:**
71+
- Basic HTTP Operations (8) • Retry Logic (23) • Backoff Strategies (17)
72+
- **Request Hedging (24)** • Interceptors (12) • Stats System (23)
73+
- Rate Limiting (7) • Debouncing (16) • Deduplication (17)
74+
- Error Handling (21) • Timeouts (11) • Response Types (7)
75+
- Custom Drivers (10) • Edge Cases (15) • Framework Patterns (8)
76+
77+
📋 **[View Test Documentation](./test-cli/README.md)****[Run Tests Locally](./test-cli/)**
78+
79+
---
80+
5081
## 📦 Installation
5182

5283
### NPM/Yarn (All Frameworks)
@@ -419,6 +450,213 @@ const api = createLuminara({
419450

420451
---
421452

453+
## 🏎️ Request Hedging
454+
455+
Request hedging sends multiple concurrent or sequential requests to reduce latency by racing against slow responses. This is particularly effective for high-latency scenarios where P99 tail latencies impact user experience.
456+
457+
### When to Use Hedging
458+
459+
**Use hedging when:**
460+
- High P99 latencies are impacting user experience
461+
- Idempotent read operations (GET, HEAD, OPTIONS)
462+
- Server-side variability is high (cloud, microservices)
463+
- Cost of duplicate requests is acceptable
464+
465+
**Don't use hedging when:**
466+
- Non-idempotent operations (POST, PUT, DELETE)
467+
- Bandwidth is severely constrained
468+
- Server capacity is limited
469+
- Operations have side effects
470+
471+
### Basic Race Policy
472+
473+
Race policy sends multiple concurrent requests and uses the first successful response:
474+
475+
```js
476+
const api = createLuminara({
477+
hedging: {
478+
policy: 'race',
479+
hedgeDelay: 1000, // Wait 1s before sending hedge
480+
maxHedges: 2 // Up to 2 hedge requests
481+
}
482+
});
483+
484+
// Timeline:
485+
// T+0ms: Primary request sent
486+
// T+1000ms: Hedge #1 sent (if primary not complete)
487+
// T+2000ms: Hedge #2 sent (if neither complete)
488+
// First successful response wins, others cancelled
489+
await api.get('/api/data');
490+
```
491+
492+
### Cancel-and-Retry Policy
493+
494+
Cancel-and-retry policy cancels slow requests and retries sequentially:
495+
496+
```js
497+
const api = createLuminara({
498+
hedging: {
499+
policy: 'cancel-and-retry',
500+
hedgeDelay: 1500, // Wait 1.5s before cancelling
501+
maxHedges: 2 // Up to 2 hedge attempts
502+
}
503+
});
504+
505+
// Timeline:
506+
// T+0ms: Primary request sent
507+
// T+1500ms: Cancel primary, send hedge #1
508+
// T+3000ms: Cancel hedge #1, send hedge #2
509+
// Only one request active at a time
510+
await api.get('/api/data');
511+
```
512+
513+
### Exponential Backoff & Jitter
514+
515+
Increase hedge delays exponentially with randomization to prevent thundering herd:
516+
517+
```js
518+
const api = createLuminara({
519+
hedging: {
520+
policy: 'race',
521+
hedgeDelay: 500, // Base delay: 500ms
522+
maxHedges: 3,
523+
exponentialBackoff: true, // Enable exponential backoff
524+
backoffMultiplier: 2, // 2x each time
525+
jitter: true, // Add randomness
526+
jitterRange: 0.3 // ±30% jitter
527+
}
528+
});
529+
530+
// Hedge timing with backoff:
531+
// - Primary: 0ms
532+
// - Hedge 1: ~500ms (500 ±30%)
533+
// - Hedge 2: ~1000ms (1000 ±30%)
534+
// - Hedge 3: ~2000ms (2000 ±30%)
535+
```
536+
537+
### HTTP Method Whitelist
538+
539+
By default, only idempotent methods are hedged:
540+
541+
```js
542+
// Default whitelist: ['GET', 'HEAD', 'OPTIONS']
543+
544+
const api = createLuminara({
545+
hedging: {
546+
policy: 'race',
547+
hedgeDelay: 1000,
548+
maxHedges: 2,
549+
includeHttpMethods: ['GET', 'HEAD', 'OPTIONS', 'POST'] // Custom whitelist
550+
}
551+
});
552+
553+
await api.get('/users'); // ✅ Hedged (in whitelist)
554+
await api.post('/data', {}); // ✅ Hedged (added to whitelist)
555+
await api.put('/users/1', {}); // ❌ Not hedged (not in whitelist)
556+
```
557+
558+
### Per-Request Configuration (Bidirectional Override)
559+
560+
Override hedging settings for specific requests:
561+
562+
```js
563+
// Scenario 1: Global enabled → disable per-request
564+
const client = createLuminara({
565+
hedging: { policy: 'race', hedgeDelay: 1000 }
566+
});
567+
568+
await client.get('/critical', {
569+
hedging: { enabled: false } // Disable for this request
570+
});
571+
572+
// Scenario 2: Global disabled → enable per-request
573+
const client2 = createLuminara({ /* no hedging */ });
574+
575+
await client2.get('/slow-endpoint', {
576+
hedging: { // Enable for this request
577+
policy: 'race',
578+
hedgeDelay: 500,
579+
maxHedges: 1
580+
}
581+
});
582+
```
583+
584+
### Server Rotation
585+
586+
Distribute hedge requests across multiple servers:
587+
588+
```js
589+
const api = createLuminara({
590+
hedging: {
591+
policy: 'race',
592+
hedgeDelay: 1000,
593+
maxHedges: 2,
594+
servers: [
595+
'https://api1.example.com',
596+
'https://api2.example.com',
597+
'https://api3.example.com'
598+
]
599+
}
600+
});
601+
602+
// Each hedge uses a different server:
603+
// - Primary: api1.example.com/data
604+
// - Hedge 1: api2.example.com/data
605+
// - Hedge 2: api3.example.com/data
606+
```
607+
608+
### Hedging vs Retry
609+
610+
**Hedging** and **Retry** serve different purposes and can be used together:
611+
612+
| Feature | Hedging | Retry |
613+
|---------|---------|-------|
614+
| **Purpose** | Reduce latency | Handle failures |
615+
| **Trigger** | Slow response | Error response |
616+
| **Requests** | Concurrent/Sequential proactive | Sequential reactive |
617+
| **Cost** | Higher (multiple requests) | Lower (on error only) |
618+
| **Use Case** | P99 optimization | Reliability |
619+
620+
```js
621+
// Combined: Hedging for latency + Retry for reliability
622+
const api = createLuminara({
623+
retry: false, // Disable retry (can use false or 0)
624+
hedging: {
625+
policy: 'race',
626+
hedgeDelay: 1000,
627+
maxHedges: 1
628+
}
629+
});
630+
```
631+
632+
### Configuration Options
633+
634+
| Option | Type | Default | Description |
635+
|--------|------|---------|-------------|
636+
| `enabled` | `boolean` | implicit | Explicit enable/disable (implicit if config present) |
637+
| `policy` | `string` | `'race'` | `'race'` or `'cancel-and-retry'` |
638+
| `hedgeDelay` | `number` | - | Base delay before hedge (ms) |
639+
| `maxHedges` | `number` | `1` | Maximum hedge requests |
640+
| `exponentialBackoff` | `boolean` | `false` | Enable exponential backoff |
641+
| `backoffMultiplier` | `number` | `2` | Backoff multiplier |
642+
| `jitter` | `boolean` | `false` | Add randomness to delays |
643+
| `jitterRange` | `number` | `0.3` | Jitter range (±30%) |
644+
| `includeHttpMethods` | `string[]` | `['GET', 'HEAD', 'OPTIONS']` | Hedged HTTP methods |
645+
| `servers` | `string[]` | `[]` | Server rotation URLs |
646+
647+
### Performance Implications
648+
649+
**Bandwidth:** Hedging increases bandwidth usage by sending multiple requests. For a `maxHedges: 2` configuration:
650+
- Best case: 1 request (primary succeeds quickly)
651+
- Worst case: 3 requests (primary + 2 hedges)
652+
- Average: ~1.5-2 requests depending on latency
653+
654+
**Latency Reduction:** Typical P99 improvements:
655+
- Race policy: 30-60% reduction in tail latencies
656+
- Cancel-and-retry: 20-40% reduction with lower bandwidth cost
657+
658+
---
659+
422660
## 🚦 Rate Limiting
423661

424662
Luminara's rate limiting system uses a **token bucket algorithm** with flexible scoping to control request flow and prevent API abuse.

benchmark/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ Opens beautiful HTML report with:
9494
- Stats system queries
9595
- Request deduplication
9696
- Debouncing
97+
- Request hedging (race policy, cancel-and-retry, exponential backoff, overhead)
9798

9899
### Integrated Scenarios
99100
- Bare minimum GET request

benchmark/browser/index.html

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,16 @@
55
<meta name="viewport" content="width=device-width, initial-scale=1.0">
66
<title>Luminara Browser Benchmarks</title>
77
<link rel="stylesheet" href="styles.css">
8+
9+
<!-- Google tag (gtag.js) -->
10+
<script async src="https://www.googletagmanager.com/gtag/js?id=G-G8T5Q4JYTX"></script>
11+
<script>
12+
window.dataLayer = window.dataLayer || [];
13+
function gtag(){dataLayer.push(arguments);}
14+
gtag('js', new Date());
15+
gtag('config', 'G-G8T5Q4JYTX');
16+
</script>
17+
818
</head>
919
<body>
1020
<div class="container">
@@ -58,6 +68,7 @@ <h3>⏱️ Mean Execution Time</h3>
5868

5969
<!-- Import Tinybench and Luminara, then load runner -->
6070
<script type="module">
71+
6172
// Import dependencies
6273
import { Bench } from 'https://esm.sh/tinybench@2.9.0';
6374

benchmark/browser/runner.js

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ class BrowserBenchmarkRunner {
8080

8181
for (const benchmark of benchmarks) {
8282
if (!benchmark || !benchmark.fn) {
83-
console.warn(`Skipping benchmark: invalid structure`, benchmark);
83+
console.warn('Skipping benchmark: invalid structure', benchmark);
8484
continue;
8585
}
8686
bench.add(benchmark.name, benchmark.fn);
@@ -147,7 +147,9 @@ class BrowserBenchmarkRunner {
147147
const tableContainer = document.getElementById('resultsTable');
148148
const summaryContainer = document.getElementById('summary');
149149

150-
if (this.results.length === 0) return;
150+
if (this.results.length === 0) {
151+
return;
152+
}
151153

152154
// Show summary
153155
summaryContainer.innerHTML = `
@@ -278,15 +280,25 @@ class BrowserBenchmarkRunner {
278280
}
279281

280282
formatTime(ms) {
281-
if (ms < 0.001) return `${(ms * 1000000).toFixed(2)} ns`;
282-
if (ms < 1) return `${(ms * 1000).toFixed(2)} μs`;
283-
if (ms < 1000) return `${ms.toFixed(2)} ms`;
283+
if (ms < 0.001) {
284+
return `${(ms * 1000000).toFixed(2)} ns`;
285+
}
286+
if (ms < 1) {
287+
return `${(ms * 1000).toFixed(2)} μs`;
288+
}
289+
if (ms < 1000) {
290+
return `${ms.toFixed(2)} ms`;
291+
}
284292
return `${(ms / 1000).toFixed(2)} s`;
285293
}
286294

287295
formatOps(ops) {
288-
if (ops >= 1000000) return `${(ops / 1000000).toFixed(2)}M ops/s`;
289-
if (ops >= 1000) return `${(ops / 1000).toFixed(2)}K ops/s`;
296+
if (ops >= 1000000) {
297+
return `${(ops / 1000000).toFixed(2)}M ops/s`;
298+
}
299+
if (ops >= 1000) {
300+
return `${(ops / 1000).toFixed(2)}K ops/s`;
301+
}
290302
return `${ops.toFixed(0)} ops/s`;
291303
}
292304
}

0 commit comments

Comments
 (0)