Structured logging + HikariCP tuning + observability

[groldan]

Observability and Database Connection Pool Improvements

Description

Enhance production observability and prevent connection pool bottlenecks under load.

Changes

Structured JSON Logging

File: src/artifacts/api/src/main/resources/application.yml

logging:
  structured:
    format:
      console: ecs  # Elastic Common Schema

HikariCP Tuning

File: src/artifacts/api/src/main/resources/application.yml

geoserver:
  acl:
    datasource:
      hikari:
        minimum-idle: 10  # increased from 1
        maximum-pool-size: 30  # increased from 20
        connection-timeout: 3000
        idle-timeout: 600000
        max-lifetime: 1800000
        leak-detection-threshold: 60000

Rationale: Current max pool size of 20 could bottleneck under load since Tomcat allows 64 concurrent request threads. Rule of thumb: connection pool = threads/2 to threads for I/O-bound apps.

Enhanced Metrics

File: src/artifacts/api/src/main/resources/application.yml

management:
  observations:
    annotations:
      enabled: true
  metrics:
    distribution:
      percentiles-histogram:
        http.server.requests: true

Expected Benefits

• Machine-readable logs for ELK/Splunk/CloudWatch integration
• Prevent connection starvation under load
• Better monitoring of ACL authorization performance and cache hit ratios
• Improved production debugging with structured log parsing

Testing

• Deploy to staging environment
• Run load tests to verify connection pool behavior
• Validate structured logs are properly ingested by logging infrastructure
• Monitor actuator metrics for connection pool health