Skip to content

Commit 525b866

Browse files
committed
fix 1.77.7 stable
1 parent 4e88e21 commit 525b866

File tree

3 files changed

+5
-1
lines changed

3 files changed

+5
-1
lines changed
1.25 MB
Loading

docs/my-website/release_notes/v1.77.5-stable/index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,8 @@ pip install litellm==1.77.5
5959

6060
### Performance Improvements - 54% RPS Improvement
6161

62+
<Image img={require('../../img/release_notes/perf_77_5.png')} style={{ width: '800px', height: 'auto' }} />
63+
6264
Throughput increased by 54% (1,040 → 1,602 RPS, aggregated) per instance while maintaining a 40 ms median overhead. The improvement comes from fixing major O(n²) inefficiencies in the router, primarily caused by repeated use of in statements inside loops over large arrays. Tests were run with a database-only setup (no cache hits). As a result, p95 latency improved by 30% (2,700 → 1,900 ms), enhancing overall stability and scalability under heavy load.
6365

6466
#### Test Setup

docs/my-website/release_notes/v1.77.7-stable/index.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,9 @@ pip install litellm==1.77.7.rc.1
6767

6868
### 2.9x Lower Median Latency
6969

70-
<Image img={require('../../img/perf_77_7.png')} style={{ width: '800px', height: 'auto' }} />
70+
<Image img={require('../../img/release_notes/perf_77_7.png')} style={{ width: '800px', height: 'auto' }} />
71+
72+
<br/>
7173

7274
This update removes LiteLLM router inefficiencies, reducing complexity from O(M×N) to O(1). Previously, it built a new array and ran repeated checks like data["model"] in llm_router.get_model_ids(). Now, a direct ID-to-deployment map eliminates redundant allocations and scans.
7375

0 commit comments

Comments
 (0)