Metrics in HttpService Options

[coolwednesday]

Fix Circuit Breaker Concurrency, Error Handling, and Metrics

• Closes metrics support for the circuit breaker #2813

Description

This PR addresses critical issues in the Circuit Breaker implementation regarding concurrency safety, error handling logic, and metric reporting.

Key Changes

1. Concurrency Safety & Race Condition Fixes

• pkg/gofr/service/circuit_breaker.go: Added necessary locking in startHealthChecks and tryCircuitRecovery. Previously, resetCircuit was being called without holding the lock, leading to potential race conditions on state and failureCount.
• pkg/gofr/service/metrics_helper.go: Implemented a global lock (metricsMu) for metric registration to prevent "concurrent map writes" when multiple services initialize metrics simultaneously.

2. Robust Edge Case Handling

• Thundering Herd Fix: In tryCircuitRecovery, added a check for cb.state == ClosedState immediately after acquiring the lock. This prevents queued requests from re-running the health check if another request has already recovered the circuit.
• Busy Loop Prevention: Updated lastChecked timestamp before attempting a health check in tryCircuitRecovery. This ensures that if a health check fails, subsequent requests respect the interval wait time instead of immediately retrying in a tight loop.

3. Functional Correctness

• HTTP 500 Handling: Updated executeWithCircuitBreaker to treat HTTP 500+ status codes as failures. Previously, only network errors (err != nil) triggered the circuit breaker, causing it to remain closed during server-side outages.

Checklist:

• I have formatted my code using goimport and golangci-lint.
• All new code is covered by unit tests.
• This PR does not decrease the overall code coverage.
• I have reviewed the code comments and documentation for clarity.