Skip to content

Conversation

mag123c
Copy link

@mag123c mag123c commented Aug 7, 2025

PR Checklist

Please check if your PR fulfills the following requirements:

PR Type

  • Bugfix
  • Feature
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • CI related changes
  • Other... Please describe:

What is the current behavior?

Currently, the graceful shutdown mechanism doesn't provide proper support for production environments where readiness and liveness probes need to be handled separately. When a pod receives SIGTERM, there's no way to mark the readiness probe as unhealthy while keeping the liveness probe healthy, which can lead to:

  • Load balancers continuing to route traffic to terminating pods
  • Requests being lost during pod restarts
  • No proper time for load balancer to detect and stop routing traffic

Issue Number: #2569

What is the new behavior?

This PR introduces an enhanced graceful shutdown mechanism with multi-stage shutdown process:

  1. New configuration options:
    - enableEnhancedShutdown: Enables the enhanced shutdown sequence
    - beforeShutdownDelayMs: Time to wait after marking unhealthy (default: 15000ms)
  2. Multi-stage shutdown sequence:
    - Stage 1: Mark readiness probe as unhealthy via isApplicationShuttingDown()
    - Stage 2: Wait for load balancer to stop routing traffic (configurable delay)
    - Stage 3: Process remaining requests for gracefulShutdownTimeoutMs
    - Stage 4: Terminate application
  3. Public API additions:
    - Exported GracefulShutdownService for integration with health checks
    - Added isApplicationShuttingDown() method to check shutdown status

Example usage:

// Module configuration
TerminusModule.forRoot({
  enableEnhancedShutdown: true,
  beforeShutdownDelayMs: 15000,
  gracefulShutdownTimeoutMs: 30000,
})

// Health check integration
if (this.gracefulShutdownService.isApplicationShuttingDown()) {
  // Return unhealthy for readiness probe
}

Does this PR introduce a breaking change?

  • Yes
  • No

@mag123c
Copy link
Author

mag123c commented Aug 27, 2025

ci e2e fail reason: #2672

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant