Performance Monitoring & Optimization

Comprehensive guide to performance monitoring, metrics collection, and optimization strategies in the Qwe Framework.

Overview
Performance Monitoring
Metrics Collection
System Monitoring
Load Testing
Optimization Strategies
Production Monitoring
Best Practices

Overview

Qwe Framework provides built-in performance monitoring and metrics collection capabilities designed to help you understand, measure, and optimize your application's performance. The monitoring system includes real-time metrics, system resource tracking, and comprehensive performance analysis tools.

Key Features

📊 Real-time Metrics: Counter, gauge, and histogram metrics
🖥️ System Monitoring: CPU, memory, disk, and network tracking
⏱️ Performance Profiling: Request timing and bottleneck identification
🚀 Load Testing: Built-in load testing utilities
💡 Health Checks: Automated health monitoring and alerts
📈 Analytics: Performance trend analysis and reporting
🔔 Alerting: Threshold-based alerts and notifications

Performance Monitoring

Basic Performance Monitor Setup

import { createPerformanceMonitor } from 'qwe-framework';

// Create performance monitor
const monitor = createPerformanceMonitor({
  enabled: true,
  collectInterval: 5000,    // Collect metrics every 5 seconds
  retentionPeriod: 3600000, // Keep data for 1 hour
  
  thresholds: {
    responseTime: 1000,     // Alert if response time > 1s
    errorRate: 0.05,        // Alert if error rate > 5%
    memoryUsage: 0.9        // Alert if memory usage > 90%
  }
});

// Start monitoring
monitor.start();

// Get current performance stats
const stats = monitor.getStats();
console.log('Performance Stats:', stats);

Request Performance Tracking

// Automatic request monitoring middleware
app.use(async (qwe, next) => {
  const startTime = Date.now();
  
  try {
    await next();
    
    // Record successful request
    monitor.recordRequest({
      duration: Date.now() - startTime,
      method: qwe.method,
      path: qwe.url,
      statusCode: qwe._getStatusCode(),
      success: true
    });
  } catch (error) {
    // Record failed request
    monitor.recordRequest({
      duration: Date.now() - startTime,
      method: qwe.method,
      path: qwe.url,
      statusCode: qwe._getStatusCode(),
      success: false,
      error: error.message
    });
    
    throw error;
  }
});

// Monitor specific operations
app.get('/expensive-operation', async (qwe) => {
  const timer = monitor.startTimer('expensive_operation');
  
  try {
    // Perform expensive operation
    const result = await performExpensiveOperation();
    
    timer.end({ success: true });
    return qwe.success('Operation completed', result);
  } catch (error) {
    timer.end({ success: false, error: error.message });
    return qwe.internalServerError('Operation failed');
  }
});

Performance Analytics

// Get detailed performance analytics
app.get('/admin/performance', (qwe) => {
  const analytics = monitor.getAnalytics();
  
  return qwe.success('Performance analytics', {
    summary: {
      totalRequests: analytics.totalRequests,
      averageResponseTime: analytics.averageResponseTime,
      requestsPerSecond: analytics.requestsPerSecond,
      errorRate: analytics.errorRate
    },
    
    percentiles: {
      p50: analytics.getPercentile(50),
      p90: analytics.getPercentile(90),
      p95: analytics.getPercentile(95),
      p99: analytics.getPercentile(99)
    },
    
    slowestEndpoints: analytics.getSlowestEndpoints(10),
    errorsByEndpoint: analytics.getErrorsByEndpoint(),
    
    trends: {
      hourly: analytics.getTrends('hour'),
      daily: analytics.getTrends('day')
    }
  });
});

Metrics Collection

Creating and Using Metrics

import { createMetrics } from 'qwe-framework';

// Initialize metrics system
const metrics = createMetrics({
  collectInterval: 10000,     // Collect every 10 seconds
  maxMetricValues: 1000,      // Keep max 1000 values per metric
  retentionPeriod: 86400000,  // 24 hours retention
  enableSystemMetrics: true   // Collect system metrics
});

// Create different metric types
const httpRequests = metrics.counter('http_requests_total', {
  help: 'Total number of HTTP requests',
  labelNames: ['method', 'path', 'status_code']
});

const activeConnections = metrics.gauge('active_connections', {
  help: 'Number of active connections'
});

const requestDuration = metrics.histogram('http_request_duration_seconds', {
  help: 'HTTP request duration in seconds',
  buckets: [0.1, 0.5, 1, 2, 5, 10] // Response time buckets
});

Counter Metrics

// Increment counters
app.use(async (qwe, next) => {
  await next();
  
  // Increment request counter with labels
  httpRequests.inc({
    method: qwe.method,
    path: qwe.url,
    status_code: qwe._getStatusCode().toString()
  });
});

// Business metrics
const userRegistrations = metrics.counter('user_registrations_total', {
  help: 'Total user registrations'
});

app.post('/auth/register', async (qwe) => {
  const user = await registerUser(qwe.body);
  
  // Increment business metric
  userRegistrations.inc();
  
  return qwe.created('User registered', user);
});

Gauge Metrics

// Track current values
const cpuUsage = metrics.gauge('cpu_usage_percent', {
  help: 'Current CPU usage percentage'
});

const memoryUsage = metrics.gauge('memory_usage_bytes', {
  help: 'Current memory usage in bytes'
});

// Update gauges periodically
setInterval(() => {
  const usage = process.cpuUsage();
  cpuUsage.set(usage.user + usage.system);
  
  const memory = process.memoryUsage();
  memoryUsage.set(memory.heapUsed);
}, 5000);

// Business gauges
const activeUsers = metrics.gauge('active_users', {
  help: 'Number of currently active users'
});

// Update on user activity
app.use('/api', jwtMiddleware(), (qwe, next) => {
  activeUsers.inc(); // User became active
  
  // Track user session
  qwe.on('disconnect', () => {
    activeUsers.dec(); // User became inactive
  });
  
  next();
});

Histogram Metrics

// Measure distributions and timing
app.use(async (qwe, next) => {
  const startTime = Date.now();
  
  await next();
  
  const duration = (Date.now() - startTime) / 1000;
  requestDuration.observe(duration, {
    method: qwe.method,
    path: qwe.url
  });
});

// Database query timing
const queryDuration = metrics.histogram('db_query_duration_seconds', {
  help: 'Database query duration',
  buckets: [0.001, 0.01, 0.1, 0.5, 1, 2, 5]
});

async function executeQuery(sql: string, params: any[]) {
  const startTime = Date.now();
  
  try {
    const result = await database.query(sql, params);
    
    const duration = (Date.now() - startTime) / 1000;
    queryDuration.observe(duration, { 
      operation: sql.split(' ')[0].toLowerCase() // SELECT, INSERT, etc.
    });
    
    return result;
  } catch (error) {
    const duration = (Date.now() - startTime) / 1000;
    queryDuration.observe(duration, { 
      operation: sql.split(' ')[0].toLowerCase(),
      error: 'true'
    });
    
    throw error;
  }
}

System Monitoring

System Resource Tracking

// Enable comprehensive system monitoring
const systemMonitor = createMetrics({
  enableSystemMetrics: true,
  systemMetricsInterval: 5000,
  
  systemMetrics: {
    cpu: true,      // CPU usage and load
    memory: true,   // Memory usage statistics
    disk: true,     // Disk I/O and usage
    network: true   // Network I/O statistics
  }
});

// Get system metrics
app.get('/admin/system', (qwe) => {
  const systemStats = systemMonitor.getSystemMetrics();
  
  return qwe.success('System metrics', {
    cpu: {
      usage: systemStats.cpu.usage,
      cores: systemStats.cpu.cores,
      loadAverage: systemStats.cpu.loadAverage
    },
    
    memory: {
      total: systemStats.memory.total,
      used: systemStats.memory.used,
      free: systemStats.memory.free,
      usagePercent: systemStats.memory.usagePercent
    },
    
    process: {
      heapUsed: process.memoryUsage().heapUsed,
      heapTotal: process.memoryUsage().heapTotal,
      external: process.memoryUsage().external,
      uptime: process.uptime()
    }
  });
});

Health Checks

// Register health checks
metrics.registerHealthCheck('database', async () => {
  try {
    await database.query('SELECT 1');
    return {
      status: 'healthy',
      message: 'Database connection OK',
      responseTime: Date.now() - start
    };
  } catch (error) {
    return {
      status: 'unhealthy',
      message: 'Database connection failed',
      error: error.message
    };
  }
});

metrics.registerHealthCheck('redis', async () => {
  try {
    await redis.ping();
    return {
      status: 'healthy',
      message: 'Redis connection OK'
    };
  } catch (error) {
    return {
      status: 'unhealthy',
      message: 'Redis connection failed',
      error: error.message
    };
  }
});

// Health check endpoint
app.get('/health', async (qwe) => {
  const healthChecks = await metrics.runHealthChecks();
  const allHealthy = healthChecks.every(check => check.status === 'healthy');
  
  return qwe.status(allHealthy ? 200 : 503).json({
    status: allHealthy ? 'healthy' : 'unhealthy',
    timestamp: new Date().toISOString(),
    checks: healthChecks
  });
});

Load Testing

Built-in Load Testing

import { createLoadTester } from 'qwe-framework';

// Create load tester
const loadTester = createLoadTester({
  baseURL: 'http://localhost:3000',
  concurrency: 10,        // 10 concurrent users
  duration: 60000,        // Test for 60 seconds
  rampUp: 5000,          // Ramp up over 5 seconds
  
  requests: [
    {
      method: 'GET',
      path: '/api/users',
      weight: 0.6  // 60% of requests
    },
    {
      method: 'POST',
      path: '/api/users',
      weight: 0.3,  // 30% of requests
      body: {
        name: 'Test User',
        email: 'test@example.com'
      }
    },
    {
      method: 'GET',
      path: '/api/posts',
      weight: 0.1   // 10% of requests
    }
  ]
});

// Run load test
const results = await loadTester.run();

console.log('Load Test Results:', {
  totalRequests: results.totalRequests,
  requestsPerSecond: results.requestsPerSecond,
  averageLatency: results.averageLatency,
  errorRate: results.errorRate,
  
  latencyPercentiles: {
    p50: results.latencyPercentiles.p50,
    p95: results.latencyPercentiles.p95,
    p99: results.latencyPercentiles.p99
  }
});

Performance Benchmarking

// Benchmark different framework configurations
async function benchmarkFramework() {
  const scenarios = [
    { name: 'No Middleware', middlewares: [] },
    { name: 'With CORS', middlewares: [cors()] },
    { name: 'Full Stack', middlewares: [cors(), logger(), rateLimiter()] }
  ];
  
  const results = [];
  
  for (const scenario of scenarios) {
    const app = createApp();
    scenario.middlewares.forEach(mw => app.plugin(mw));
    
    app.get('/test', (qwe) => qwe.success('OK'));
    
    const server = app.listen(0); // Random port
    const port = server.address().port;
    
    const loadTester = createLoadTester({
      baseURL: `http://localhost:${port}`,
      concurrency: 50,
      duration: 30000,
      requests: [{ method: 'GET', path: '/test' }]
    });
    
    const result = await loadTester.run();
    results.push({
      scenario: scenario.name,
      requestsPerSecond: result.requestsPerSecond,
      averageLatency: result.averageLatency,
      errorRate: result.errorRate
    });
    
    server.close();
  }
  
  return results;
}

Optimization Strategies

Response Time Optimization

// 1. Implement caching
const cache = createCache('memory', { maxSize: 1000 });

app.get('/api/expensive-data', async (qwe) => {
  const cacheKey = `expensive-data:${qwe.query.id}`;
  
  // Try cache first
  let data = await cache.get(cacheKey);
  if (data) {
    qwe.header('X-Cache', 'HIT');
    return qwe.success('Data retrieved', data);
  }
  
  // Compute data
  data = await computeExpensiveData(qwe.query.id);
  
  // Cache for 5 minutes
  await cache.set(cacheKey, data, 300);
  
  qwe.header('X-Cache', 'MISS');
  return qwe.success('Data retrieved', data);
});

// 2. Database query optimization
app.get('/api/users', async (qwe) => {
  const page = parseInt(qwe.query.page || '1');
  const limit = Math.min(parseInt(qwe.query.limit || '10'), 100);
  
  // Use pagination to limit data
  const users = await db.users.findMany({
    take: limit,
    skip: (page - 1) * limit,
    orderBy: { createdAt: 'desc' }
  });
  
  return qwe.success('Users retrieved', users);
});

// 3. Async processing for heavy operations
const jobQueue = createJobQueue();

app.post('/api/process-file', async (qwe) => {
  const fileId = qwe.generateId();
  
  // Queue processing job instead of blocking
  await jobQueue.add('process-file', {
    fileId,
    filePath: qwe.body.filePath
  });
  
  return qwe.accepted('File queued for processing', { fileId });
});

Memory Optimization

// Monitor memory usage
const memoryMonitor = setInterval(() => {
  const usage = process.memoryUsage();
  const usagePercent = (usage.heapUsed / usage.heapTotal) * 100;
  
  if (usagePercent > 80) {
    console.warn(`High memory usage: ${usagePercent.toFixed(2)}%`);
    
    // Trigger garbage collection if available
    if (global.gc) {
      global.gc();
    }
  }
}, 30000);

// Implement object pooling for frequently created objects
class ObjectPool<T> {
  private pool: T[] = [];
  
  constructor(private factory: () => T, private reset: (obj: T) => void) {}
  
  acquire(): T {
    return this.pool.pop() || this.factory();
  }
  
  release(obj: T): void {
    this.reset(obj);
    this.pool.push(obj);
  }
}

// Use streaming for large responses
app.get('/api/large-dataset', async (qwe) => {
  const stream = database.stream('SELECT * FROM large_table');
  
  qwe.header('Content-Type', 'application/json');
  qwe.text('[');
  
  let first = true;
  stream.on('data', (row) => {
    if (!first) qwe.text(',');
    qwe.text(JSON.stringify(row));
    first = false;
  });
  
  stream.on('end', () => {
    qwe.text(']');
  });
});

Production Monitoring

Metrics Export

// Prometheus-compatible metrics endpoint
app.get('/metrics', (qwe) => {
  const prometheusMetrics = metrics.getPrometheusFormat();
  
  qwe.header('Content-Type', 'text/plain; version=0.0.4; charset=utf-8');
  return qwe.text(prometheusMetrics);
});

// JSON metrics for custom monitoring
app.get('/metrics/json', (qwe) => {
  const metricsData = metrics.getMetricsJSON();
  
  return qwe.success('Metrics data', metricsData);
});

// Performance dashboard data
app.get('/admin/dashboard', (qwe) => {
  const dashboardData = {
    overview: monitor.getOverview(),
    recentAlerts: monitor.getRecentAlerts(),
    topEndpoints: monitor.getTopEndpoints(),
    systemHealth: metrics.getSystemHealth(),
    performanceTrends: monitor.getPerformanceTrends()
  };
  
  return qwe.success('Dashboard data', dashboardData);
});

Alert Configuration

// Configure performance alerts
monitor.addAlert({
  name: 'high_response_time',
  condition: (stats) => stats.averageResponseTime > 1000,
  message: 'Average response time exceeded 1 second',
  cooldown: 300000, // 5 minutes
  
  actions: [
    {
      type: 'log',
      level: 'warn'
    },
    {
      type: 'webhook',
      url: 'https://alerts.example.com/webhook',
      payload: (stats) => ({
        alert: 'High Response Time',
        value: stats.averageResponseTime,
        threshold: 1000
      })
    }
  ]
});

monitor.addAlert({
  name: 'high_error_rate',
  condition: (stats) => stats.errorRate > 0.05,
  message: 'Error rate exceeded 5%',
  
  actions: [
    {
      type: 'email',
      to: 'alerts@example.com',
      subject: 'High Error Rate Alert'
    }
  ]
});

Best Practices

1. Metric Naming and Labels

// ✅ Good: Consistent naming convention
const httpRequests = metrics.counter('http_requests_total', {
  help: 'Total HTTP requests',
  labelNames: ['method', 'path', 'status_code']
});

const dbQueries = metrics.histogram('db_query_duration_seconds', {
  help: 'Database query duration',
  labelNames: ['operation', 'table']
});

// ❌ Avoid: High-cardinality labels
// Don't use user IDs, timestamps, or unique identifiers as labels
httpRequests.inc({ userId: 'user-123' }); // Bad
httpRequests.inc({ method: 'GET', status_code: '200' }); // Good

2. Performance Monitoring Strategy

// ✅ Good: Comprehensive monitoring
class ApplicationMonitor {
  constructor() {
    this.setupRequestMonitoring();
    this.setupBusinessMetrics();
    this.setupSystemMonitoring();
    this.setupAlerts();
  }
  
  private setupRequestMonitoring() {
    // Monitor all HTTP requests
    app.use(this.requestMiddleware);
  }
  
  private setupBusinessMetrics() {
    // Track business-specific metrics
    this.userRegistrations = metrics.counter('user_registrations_total');
    this.orderValue = metrics.histogram('order_value_dollars');
  }
  
  private setupSystemMonitoring() {
    // Monitor system resources
    setInterval(() => {
      this.collectSystemMetrics();
    }, 10000);
  }
  
  private setupAlerts() {
    // Configure meaningful alerts
    this.addPerformanceAlerts();
    this.addBusinessAlerts();
  }
}

3. Load Testing Strategy

// ✅ Good: Realistic load testing
const loadTestScenarios = [
  {
    name: 'Normal Load',
    concurrency: 10,
    duration: 300000, // 5 minutes
    rampUp: 30000     // 30 seconds ramp-up
  },
  {
    name: 'Peak Load',
    concurrency: 50,
    duration: 180000, // 3 minutes
    rampUp: 60000     // 1 minute ramp-up
  },
  {
    name: 'Stress Test',
    concurrency: 100,
    duration: 120000, // 2 minutes
    rampUp: 30000     // 30 seconds ramp-up
  }
];

// Run tests with realistic user behavior
const userBehavior = [
  { path: '/api/login', method: 'POST', weight: 0.1 },
  { path: '/api/dashboard', method: 'GET', weight: 0.3 },
  { path: '/api/search', method: 'GET', weight: 0.4 },
  { path: '/api/profile', method: 'GET', weight: 0.2 }
];

Conclusion

Qwe Framework's performance monitoring and optimization tools provide comprehensive insights into your application's behavior and performance characteristics. The built-in metrics collection, system monitoring, and load testing capabilities enable you to maintain high-performance applications in production.

Key Benefits

Real-time Insights: Immediate visibility into application performance
Zero Dependencies: Complete monitoring solution built-in
Production Ready: Enterprise-grade monitoring and alerting
Developer Friendly: Easy setup and intuitive APIs
Comprehensive Coverage: From system metrics to business KPIs

Next Steps

🚀 Read Deployment for production monitoring setup
📚 Check API Reference for complete monitoring APIs
🧪 Review Testing for performance testing strategies

Need help? Check the API Reference for complete performance monitoring documentation.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Performance Monitoring & Optimization

Table of Contents

Overview

Key Features

Performance Monitoring

Basic Performance Monitor Setup

Request Performance Tracking

Performance Analytics

Metrics Collection

Creating and Using Metrics

Counter Metrics

Gauge Metrics

Histogram Metrics

System Monitoring

System Resource Tracking

Health Checks

Load Testing

Built-in Load Testing

Performance Benchmarking

Optimization Strategies

Response Time Optimization

Memory Optimization

Production Monitoring

Metrics Export

Alert Configuration

Best Practices

1. Metric Naming and Labels

2. Performance Monitoring Strategy

3. Load Testing Strategy

Conclusion

Key Benefits

Next Steps

FilesExpand file tree

PERFORMANCE.md

Latest commit

History

PERFORMANCE.md

File metadata and controls

Performance Monitoring & Optimization

Table of Contents

Overview

Key Features

Performance Monitoring

Basic Performance Monitor Setup

Request Performance Tracking

Performance Analytics

Metrics Collection

Creating and Using Metrics

Counter Metrics

Gauge Metrics

Histogram Metrics

System Monitoring

System Resource Tracking

Health Checks

Load Testing

Built-in Load Testing

Performance Benchmarking

Optimization Strategies

Response Time Optimization

Memory Optimization

Production Monitoring

Metrics Export

Alert Configuration

Best Practices

1. Metric Naming and Labels

2. Performance Monitoring Strategy

3. Load Testing Strategy

Conclusion

Key Benefits

Next Steps