Skip to content

Commit 348f21a

Browse files
committed
Refactor HttpClient and LLMClient structure; enhance logging and error handling
- Updated HttpClient to improve request handling and integrate logging. - Removed unnecessary parameters and streamlined request methods. - Refactored LLMClient to support multiple providers and fallback mechanisms. - Enhanced error handling with detailed logging for better debugging. - Updated provider factory to support dynamic client creation based on provider type. - Improved retry strategy with configurable options and logging. - Removed deprecated methods and cleaned up code for better maintainability. - Added logging utility for consistent logging across the application. - Updated types to reflect changes in response structures and configurations.
1 parent 33de1f0 commit 348f21a

File tree

15 files changed

+644
-637
lines changed

15 files changed

+644
-637
lines changed

.gitignore

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,4 +25,13 @@ coverage/
2525

2626
# Temporary files
2727
*.log
28-
*.tmp
28+
*.tmp
29+
*.tgz
30+
# Miscellaneous
31+
*.bak
32+
*.orig
33+
# Operating system files
34+
Thumbs.db
35+
# Logs
36+
logs/
37+
*/log

README.md

Lines changed: 330 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,344 @@
1-
# llmforge
1+
# LLMForge
22

33
A unified, pluggable AI runtime to run prompts across OpenAI, Gemini, Ollama, and custom models — all with a single line of code.
44

5+
**🚀 Lightweight & Fast** - Only 42.8 kB package size, 381.1 kB unpacked
6+
7+
## Features
8+
9+
- **Unified Interface**: Single API for multiple AI providers
10+
- **Lightweight**: Only 42.8 kB package size for minimal bundle impact
11+
- **Intelligent Fallback**: Automatic failover between providers
12+
- **Configurable Retry Logic**: Built-in retry mechanisms with customizable delays
13+
- **Token Usage Tracking**: Detailed usage statistics for cost monitoring
14+
- **TypeScript Support**: Full type safety and IntelliSense
15+
- **Flexible Configuration**: Per-provider settings and generation parameters
16+
517
## Installation
618

719
```bash
820
npm install llmforge
21+
# or
22+
npm install @nginh/llmforge@1.0.0
923
```
1024

11-
<!--
12-
## Usage
25+
## Quick Start
26+
27+
```typescript
28+
import { RunnerClient } from 'llmforge';
29+
30+
const config = {
31+
llmConfig: {
32+
apiKey: process.env.OPENAI_API_KEY,
33+
provider: 'openai',
34+
model: 'gpt-3.5-turbo',
35+
generationConfig: {
36+
temperature: 0.7,
37+
maxOutputTokens: 150,
38+
},
39+
},
40+
enableFallback: false,
41+
};
1342

14-
```js
15-
import { yourFunction } from 'llmforge';
16-
// Example usage
43+
const client = await RunnerClient.create(config);
44+
const response = await client.run([
45+
{
46+
role: 'user',
47+
parts: [{ text: 'Hello! Can you tell me a joke?' }],
48+
},
49+
]);
50+
51+
console.log(response.output);
1752
```
1853

19-
## Features
20-
- Unified API for multiple LLM providers
21-
- Pluggable architecture
22-
- Easy to use
54+
## Supported Providers
55+
56+
| Provider | Status | Models |
57+
| ------------- | -------------- | --------------------------------------------------------------- |
58+
| OpenAI | ✅ Supported | All text models (e.g., gpt-3.5-turbo, gpt-4, gpt-4-turbo, etc.) |
59+
| Google Gemini | ✅ Supported | All text models (e.g., gemini-1.5-flash, gemini-1.5-pro, etc.) |
60+
| Ollama | 🚧 Coming Soon | Local models |
61+
| Custom Models | 🚧 Coming Soon | User-defined endpoints |
62+
63+
_Current version (v1.0.1) supports OpenAI and Google Gemini_
64+
65+
## Configuration
66+
67+
### Single Provider
68+
69+
```typescript
70+
const config = {
71+
llmConfig: {
72+
apiKey: 'your-api-key',
73+
provider: 'openai', // or 'google'
74+
model: 'gpt-3.5-turbo',
75+
generationConfig: {
76+
temperature: 0.7,
77+
maxOutputTokens: 150,
78+
},
79+
retryConfig: {
80+
maxRetries: 3,
81+
retryDelay: 1000,
82+
},
83+
},
84+
enableFallback: false,
85+
};
86+
```
87+
88+
### Multiple Providers with Fallback
89+
90+
```typescript
91+
const config = {
92+
llmConfig: [
93+
{
94+
apiKey: process.env.OPENAI_API_KEY,
95+
provider: 'openai',
96+
model: 'gpt-3.5-turbo',
97+
priority: 1, // Primary provider
98+
generationConfig: {
99+
temperature: 0.7,
100+
maxOutputTokens: 150,
101+
},
102+
retryConfig: {
103+
maxRetries: 3,
104+
retryDelay: 1000,
105+
},
106+
},
107+
{
108+
apiKey: process.env.GOOGLE_API_KEY,
109+
provider: 'google',
110+
model: 'gemini-1.5-flash',
111+
priority: 2, // Fallback provider
112+
generationConfig: {
113+
temperature: 0.7,
114+
maxOutputTokens: 150,
115+
},
116+
retryConfig: {
117+
maxRetries: 3,
118+
retryDelay: 1000,
119+
},
120+
},
121+
],
122+
enableFallback: true,
123+
};
124+
```
125+
126+
## Response Format
127+
128+
LLMForge returns a standardized response format across all providers:
129+
130+
```typescript
131+
{
132+
"resp_id": "unique-response-id",
133+
"output": "Generated text response",
134+
"status": "success",
135+
"created_at": 1750283611,
136+
"model": "gpt-3.5-turbo-0125",
137+
"usage": {
138+
"input_tokens": 17,
139+
"output_tokens": 24,
140+
"total_tokens": 41
141+
},
142+
"fallback": {
143+
"isUsed": false,
144+
"reason": ""
145+
}
146+
}
147+
```
148+
149+
## Message Format
150+
151+
LLMForge uses a unified message format compatible with multiple providers:
152+
153+
```typescript
154+
const messages = [
155+
{
156+
role: 'user',
157+
parts: [{ text: 'Your prompt here' }],
158+
},
159+
{
160+
role: 'assistant',
161+
parts: [{ text: 'Assistant response' }],
162+
},
163+
];
164+
```
165+
166+
## Error Handling
167+
168+
LLMForge provides comprehensive error handling with automatic fallback:
169+
170+
```typescript
171+
try {
172+
const response = await client.run(messages);
173+
console.log(response.output);
174+
} catch (error) {
175+
console.error('Error:', error.message);
176+
177+
// Check if fallback was attempted
178+
if (response?.fallback?.isUsed) {
179+
console.log('Fallback reason:', response.fallback.reason);
180+
}
181+
}
182+
```
183+
184+
## Environment Variables
185+
186+
Create a `.env` file in your project root:
187+
188+
```env
189+
OPENAI_API_KEY=your-openai-api-key
190+
GOOGLE_API_KEY=your-google-api-key
191+
```
192+
193+
## Examples
194+
195+
### Basic OpenAI Usage
196+
197+
```typescript
198+
import { RunnerClient } from 'llmforge';
199+
200+
const client = await RunnerClient.create({
201+
llmConfig: {
202+
apiKey: process.env.OPENAI_API_KEY,
203+
provider: 'openai',
204+
model: 'gpt-3.5-turbo',
205+
},
206+
});
207+
208+
const response = await client.run([{ role: 'user', parts: [{ text: 'Explain quantum computing' }] }]);
209+
```
210+
211+
### Basic Gemini Usage
212+
213+
```typescript
214+
import { RunnerClient } from 'llmforge';
215+
216+
const client = await RunnerClient.create({
217+
llmConfig: {
218+
apiKey: process.env.GOOGLE_API_KEY,
219+
provider: 'google',
220+
model: 'gemini-1.5-flash',
221+
},
222+
});
223+
224+
const response = await client.run([{ role: 'user', parts: [{ text: 'Write a haiku about technology' }] }]);
225+
```
226+
227+
### Fallback Configuration
228+
229+
```typescript
230+
const client = await RunnerClient.create({
231+
llmConfig: [
232+
{
233+
apiKey: process.env.OPENAI_API_KEY,
234+
provider: 'openai',
235+
model: 'gpt-4',
236+
priority: 1,
237+
},
238+
{
239+
apiKey: process.env.GOOGLE_API_KEY,
240+
provider: 'google',
241+
model: 'gemini-1.5-pro',
242+
priority: 2,
243+
},
244+
],
245+
enableFallback: true,
246+
});
247+
```
248+
249+
## API Reference
250+
251+
### RunnerClient
252+
253+
#### `RunnerClient.create(config)`
254+
255+
Creates a new LLMForge client instance.
256+
257+
**Parameters:**
258+
259+
- `config`: Configuration object containing LLM settings and fallback options
260+
261+
**Returns:** Promise<RunnerClient>
262+
263+
#### `client.run(messages)`
264+
265+
Executes a prompt against the configured LLM provider(s).
266+
267+
**Parameters:**
268+
269+
- `messages`: Array of message objects in the unified format
270+
271+
**Returns:** Promise<Response>
272+
273+
### Configuration Options
274+
275+
#### `generationConfig`
276+
277+
- `temperature`: Controls randomness (0.0 - 1.0)
278+
- `maxOutputTokens`: Maximum tokens in response
279+
- `topP`: Nucleus sampling parameter
280+
- `topK`: Top-k sampling parameter
281+
282+
#### `retryConfig`
283+
284+
- `maxRetries`: Maximum retry attempts
285+
- `retryDelay`: Delay between retries (ms)
286+
287+
## Roadmap
288+
289+
- [✔️] OpenAI support
290+
- [✔️] Google Gemini support
291+
- [✔️] Basic error handling
292+
- [✔️] Fallback mechanism
293+
- [✔️] TypeScript support
294+
- [✔️] Unified message format
295+
- [ ] Unified thinking and reasoning
296+
- [ ] Ollama support for local models
297+
- [ ] Custom model endpoint support
298+
- [ ] Streaming responses
299+
- [ ] Response caching
300+
- [ ] Anthropic Claude support
301+
- [ ] Azure OpenAI support
302+
303+
## Contributing
304+
305+
We welcome contributions!
306+
307+
To contribute:
308+
309+
1. **Create a Provider Interface:**
310+
Follow the existing patterns in the codebase to add new providers. Ensure your implementation is modular and consistent with current interfaces.
311+
312+
2. **Error Handling:**
313+
Handle errors gracefully using proper error types. Provide clear error messages and ensure fallback mechanisms work as expected.
314+
315+
3. **Code Formatting:**
316+
Use [Prettier](https://prettier.io/) for code formatting. Run `npm run format .` before submitting your changes.
317+
318+
4. **Testing:**
319+
Add or update tests to cover your changes. Ensure all tests pass before submitting a PR.
320+
321+
5. **Pull Request Guidelines:**
322+
323+
- Create a Pull Request on GitHub with a clear title and description.
324+
- Reference any related issues (e.g., "Closes #123").
325+
- Include screenshots or examples if applicable.
326+
- Provide testing instructions so reviewers can verify your changes.
327+
- Paste the rewritten markdown content (if documentation is updated).
328+
329+
6. **Review Process:**
330+
Your PR will be reviewed for code quality, consistency, and adherence to project guidelines.
331+
332+
Thank you for helping improve LLMForge!
333+
334+
MIT License - see [LICENSE](LICENSE) file for details.
335+
336+
## Support
337+
338+
- 📧 Email: <harshanand.cloud@gmail.com>
339+
- 🐛 Issues: [GitHub Issues](https://github.com/nginH/llmforge/issues)
23340

24-
## Documentation
25-
See [GitHub](https://github.com/nginH/llmforge) for full documentation and examples.
341+
---
26342

27-
## License
28-
MIT -->
343+
Built with ❤️ for the AI developer community
344+
by [nginH](https://github.com/nginH)

0 commit comments

Comments
 (0)