|
2 | 2 |
|
3 | 3 | A minimalist but robust LLM integration layer designed to ensure reliable, seamless interactions across multiple LLM providers by intelligently handling failures and rate limits. |
4 | 4 |
|
5 | | -## Why Use ResilentLLM |
| 5 | +--- |
6 | 6 |
|
7 | | -This library solves challenges in building production-ready AI Agents due to |
| 7 | +This library solves challenges in building production-ready AI Agents due to: |
8 | 8 |
|
9 | | -- Unstable network conditions |
10 | | -- Inconsistent error handling |
11 | | -- Unpredictable LLM API rate limit errors |
| 9 | +- ❌ Unstable network conditions |
| 10 | +- ⚠️ Inconsistent error handling |
| 11 | +- ⏳ Unpredictable LLM API rate limit errors |
12 | 12 |
|
13 | 13 | ### Key Features |
14 | 14 |
|
15 | | -- **Rate limiting**: You don’t need to calculate tokens or manage rate limits yourself |
16 | | -- **Token estimation**: The number of LLM tokens is estimated for each request and enforced. |
| 15 | +- **Token estimation**: You don’t need to calculate LLM tokens, they are estimated for each request |
| 16 | +- **Rate limiting**: You don't need to manage the token bucket rate algorithm yourself to follow the rate limits by LLM service providers, it is done for you automatically |
17 | 17 | - **Retries, backoff, and circuit breaker**: All are handled internally by the `ResilientOperation`. |
18 | 18 |
|
19 | 19 | ## Installation |
@@ -55,23 +55,6 @@ const conversationHistory = [ |
55 | 55 | })(); |
56 | 56 | ``` |
57 | 57 |
|
58 | | -### Advanced Options |
59 | | - |
60 | | -```js |
61 | | -const response = await llm.chat( |
62 | | - [ |
63 | | - { role: 'user', content: 'Summarize the plot of Inception.' } |
64 | | - ], |
65 | | - { |
66 | | - maxTokens: 512, |
67 | | - temperature: 0.5, |
68 | | - aiService: 'anthropic', // override default |
69 | | - model: 'claude-3-5-sonnet-20240620' |
70 | | - } |
71 | | -); |
72 | | -``` |
73 | | - |
74 | | - |
75 | 58 | ## Motivation |
76 | 59 |
|
77 | 60 | ResilientLLM is a resilient, unified LLM interface featuring circuit breaker, token bucket rate limiting, caching, and adaptive retry with dynamic backoff support. |
|
0 commit comments