Skip to content

Commit c1d4d2b

Browse files
committed
feat(logger): add structured logging module with rotation and redaction
Add new logger module with: - Base logger with level control and timestamps - Log rotation with size-based and time-based policies - Sensitive data redaction for security - Correlation IDs for request tracing - Error aggregation for pattern detection - Configurable formatters (JSON, text)
1 parent 7da136e commit c1d4d2b

File tree

12 files changed

+3304
-0
lines changed

12 files changed

+3304
-0
lines changed

package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@
2828
"./platform/*": "./platform/*.js",
2929
"./run/*": "./run/*.js",
3030
"./core/*": "./core/*.js",
31+
"./core/logger": "./core/logger/index.js",
3132
"./workers": "./workers/index.js",
3233
"./types/*": "./types/*.js"
3334
},

src/core/logger/README.md

Lines changed: 304 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,304 @@
1+
# Script Kit Logger
2+
3+
Unified structured logging for the Script Kit ecosystem.
4+
5+
## Installation
6+
7+
The logger is part of the SDK and can be imported directly:
8+
9+
```typescript
10+
import { createSDKLogger, quickLogger } from '@johnlindquist/kit/core/logger';
11+
```
12+
13+
## Quick Start
14+
15+
### For Scripts
16+
17+
```typescript
18+
import { createScriptLogger } from '@johnlindquist/kit/core/logger';
19+
20+
const log = createScriptLogger();
21+
22+
log.info('Script started');
23+
log.debug('Processing item', { itemId: 123 });
24+
log.error('Something failed', new Error('oops'), { context: 'cleanup' });
25+
```
26+
27+
### For App Components
28+
29+
```typescript
30+
import {
31+
initializeDomainLoggers,
32+
getCoreLogger
33+
} from './logger';
34+
35+
// Initialize once at app startup
36+
const loggers = initializeDomainLoggers();
37+
38+
// Get domain-specific loggers
39+
const log = getCoreLogger();
40+
log.info('App initialized');
41+
```
42+
43+
### For Website/API
44+
45+
```typescript
46+
import { createApiLogger } from '@/lib/logger';
47+
48+
const log = createApiLogger('scripts');
49+
log.info('Script created', { scriptId: '123' });
50+
```
51+
52+
## Configuration
53+
54+
### Environment Variables
55+
56+
| Variable | Description | Default |
57+
|----------|-------------|---------|
58+
| `LOG_LEVEL` | Minimum log level | `info` (prod), `debug` (dev) |
59+
| `LOG_FORMAT` | Output format: `json`, `pretty`, `minimal` | `pretty` (dev), `json` (prod) |
60+
| `LOG_COLORS` | Enable ANSI colors | `true` (TTY), `false` (non-TTY) |
61+
| `LOG_FILE` | File path for file logging | (none) |
62+
| `LOG_REDACT` | Enable sensitive data redaction | `true` |
63+
| `LOG_REDACT_PATHS` | Additional paths to redact (comma-separated) | (none) |
64+
| `NODE_ENV` | Environment (`production`, `development`, `test`) | `development` |
65+
| `DEBUG` | Debug filter pattern | (none) |
66+
| `VERBOSE` | Enable verbose (debug) logging | `false` |
67+
68+
### App-Specific Variables
69+
70+
| Variable | Description | Default |
71+
|----------|-------------|---------|
72+
| `KIT_LOG_LEVEL` | Log level for Script Kit | (uses LOG_LEVEL) |
73+
| `KIT_LOG_TO_FILE` | Enable file logging | `true` |
74+
| `KIT_HEALTH_CHECK_INTERVAL` | Health check interval in seconds | `120` |
75+
| `KIT_PROCESS_SCAN_INTERVAL` | Process scan interval in minutes | `5` |
76+
77+
## Log Levels
78+
79+
From most to least severe:
80+
81+
| Level | Description | Use Case |
82+
|-------|-------------|----------|
83+
| `fatal` | System is unusable | Unrecoverable errors, crash scenarios |
84+
| `error` | Error conditions | Exceptions, failed operations |
85+
| `warn` | Warning conditions | Deprecations, recoverable issues |
86+
| `info` | Informational | Normal operations, user actions |
87+
| `debug` | Debug information | Detailed flow, variable values |
88+
| `trace` | Trace/verbose | Very detailed, high-frequency events |
89+
90+
## Features
91+
92+
### Structured Logging
93+
94+
All logs are structured with consistent fields:
95+
96+
```typescript
97+
log.info('User logged in', {
98+
userId: 'u123',
99+
method: 'github',
100+
responseTime: 150,
101+
});
102+
// Output (JSON format):
103+
// {"level":"info","time":"2025-01-15T10:30:00Z","msg":"User logged in","userId":"u123","method":"github","responseTime":150}
104+
```
105+
106+
### Correlation IDs
107+
108+
Track requests across async operations:
109+
110+
```typescript
111+
import { withCorrelationAsync, getCorrelationId } from '@johnlindquist/kit/core/logger';
112+
113+
await withCorrelationAsync(async () => {
114+
log.info('Starting operation'); // Includes correlationId
115+
await doSomething();
116+
log.info('Operation complete'); // Same correlationId
117+
});
118+
```
119+
120+
### Sensitive Data Redaction
121+
122+
Automatically redacts sensitive fields:
123+
124+
```typescript
125+
log.info('User data', {
126+
username: 'john',
127+
password: 'secret123', // Becomes "[REDACTED]"
128+
apiKey: 'sk-12345', // Becomes "[REDACTED]"
129+
});
130+
```
131+
132+
Redacted paths include:
133+
- `password`, `secret`, `token`, `apiKey`, `api_key`
134+
- `authorization`, `cookie`, `accessToken`, `refreshToken`
135+
- `creditCard`, `ssn`, `email`
136+
137+
### Child Loggers
138+
139+
Create contextual child loggers:
140+
141+
```typescript
142+
const log = createSDKLogger({ name: 'scripts' });
143+
const scriptLog = log.child({ scriptId: 'abc123' });
144+
145+
scriptLog.info('Processing'); // Includes scriptId in all logs
146+
```
147+
148+
### Performance Timing
149+
150+
```typescript
151+
const endTimer = log.startTimer('database-query');
152+
await db.query('SELECT ...');
153+
const duration = endTimer();
154+
// Logs: "database-query completed" with duration
155+
```
156+
157+
### Error Aggregation
158+
159+
Track and summarize errors:
160+
161+
```typescript
162+
import { getErrorAggregator } from '@johnlindquist/kit/core/logger';
163+
164+
const aggregator = getErrorAggregator();
165+
166+
try {
167+
await riskyOperation();
168+
} catch (error) {
169+
aggregator.record(error, { operation: 'risky' });
170+
}
171+
172+
// Get summary
173+
const summary = aggregator.getSummary();
174+
console.log(`Total errors: ${summary.totalErrors}`);
175+
console.log(`Top error: ${summary.topErrors[0]?.message}`);
176+
```
177+
178+
## Domain Loggers (App)
179+
180+
The app consolidates 35+ category loggers into 8 domains:
181+
182+
| Domain | Categories |
183+
|--------|------------|
184+
| `core` | main, kit, system, health |
185+
| `window` | window, prompt, widget, theme |
186+
| `process` | process, script, child, spawn |
187+
| `input` | input, keyboard, actions, shortcut |
188+
| `communication` | ipc, channel, messages, sync |
189+
| `scheduling` | schedule, cron, watch, clipboard |
190+
| `terminal` | terminal, pty |
191+
| `diagnostic` | debug, log, performance |
192+
193+
Usage:
194+
195+
```typescript
196+
import { getCoreLogger, getProcessLogger } from './logger';
197+
198+
getCoreLogger().info('System started');
199+
getProcessLogger().debug('Script spawned', { pid: 1234 });
200+
```
201+
202+
## Log Rotation
203+
204+
File logs are automatically rotated:
205+
206+
- Max size: 10MB per file
207+
- Max files: 5 (oldest deleted)
208+
- Compression: gzip for rotated files
209+
210+
Configuration:
211+
212+
```typescript
213+
import { createRotatingStream } from '@johnlindquist/kit/core/logger';
214+
215+
const stream = createRotatingStream('/path/to/app.log', {
216+
maxSize: 10 * 1024 * 1024, // 10MB
217+
maxFiles: 5,
218+
compress: true,
219+
});
220+
```
221+
222+
## Formatters
223+
224+
### JSON Format
225+
Best for production, log aggregation:
226+
```json
227+
{"level":"info","time":"2025-01-15T10:30:00Z","name":"api","msg":"Request completed","path":"/api/scripts","responseTime":45}
228+
```
229+
230+
### Pretty Format
231+
Best for development:
232+
```
233+
10:30:00.123 INFO [api] Request completed path=/api/scripts responseTime=45
234+
```
235+
236+
### Minimal Format
237+
Best for tests:
238+
```
239+
[INFO] Request completed
240+
```
241+
242+
## Migration Guide
243+
244+
### From Old Category Loggers
245+
246+
```typescript
247+
// Before
248+
import { scriptLog, processLog } from './logs';
249+
scriptLog.info('...');
250+
processLog.debug('...');
251+
252+
// After
253+
import { getProcessLogger } from './logger';
254+
const log = getProcessLogger();
255+
log.info('...'); // Covers both script and process
256+
```
257+
258+
### From console.log
259+
260+
```typescript
261+
// Before
262+
console.log('User logged in:', userId);
263+
console.error('Error:', error);
264+
265+
// After
266+
import { createSDKLogger } from '@johnlindquist/kit/core/logger';
267+
const log = createSDKLogger({ name: 'auth' });
268+
log.info('User logged in', { userId });
269+
log.error('Error', error);
270+
```
271+
272+
## Best Practices
273+
274+
1. **Use structured data**: Pass objects, not string interpolation
275+
```typescript
276+
// Good
277+
log.info('User created', { userId, email });
278+
279+
// Avoid
280+
log.info(`User ${userId} created with email ${email}`);
281+
```
282+
283+
2. **Use appropriate levels**: Debug for development, info for production
284+
3. **Include context**: Add relevant IDs and metadata
285+
4. **Use child loggers**: For request/operation-scoped logging
286+
5. **Don't log sensitive data**: Use redaction for safety
287+
6. **Use correlation IDs**: For distributed tracing
288+
289+
## Troubleshooting
290+
291+
### Logs not appearing
292+
- Check `LOG_LEVEL` is set correctly
293+
- Verify TTY for colored output
294+
- Check if redaction is hiding content
295+
296+
### Log files not rotating
297+
- Check file permissions
298+
- Verify disk space
299+
- Check `LOG_FILE` path is writable
300+
301+
### Performance issues
302+
- Use appropriate log levels
303+
- Avoid logging in hot paths
304+
- Consider sampling for high-frequency events

0 commit comments

Comments
 (0)