From 54cd540eeaa41e514ccb9c2f07887655c81a6a41 Mon Sep 17 00:00:00 2001 From: Rick Anderson <3605364+Rick-Anderson@users.noreply.github.com> Date: Mon, 20 Mar 2023 14:45:28 -1000 Subject: [PATCH 1/5] Kestrel graceful shutdown /1 --- aspnetcore/fundamentals/servers/kestrel.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/aspnetcore/fundamentals/servers/kestrel.md b/aspnetcore/fundamentals/servers/kestrel.md index 8c90903db86e..36a4564e4bb5 100644 --- a/aspnetcore/fundamentals/servers/kestrel.md +++ b/aspnetcore/fundamentals/servers/kestrel.md @@ -49,6 +49,22 @@ The following timeouts and rate limits aren't enforced when a debugger is attach * * +## Graceful shutdown + +- The Host receives a shutdown signal (e.g. CTL+C, StopAsync, etc.) +- IHostApplicationLifetime.ApplicationStopping is signaled to notify the application. Long running operations should subscribe to this event. +- The Host calls IServer.StopAsync with a configurable shutdown timeout (default 30s) +- Kestrel (and Http.Sys) close their port bindings and stop accepting new connections. They also signal current connections to stop processing new requests. For HTTP/2 and HTTP/3 this involves sending a preliminary GoAway message to the client. For HTTP/1.1 requests are processed in order so it stops that connection loop. + - IIS is a little different, it rejects new requests with a 503. +- Active requests are given until the shutdown timeout to complete. If everything completes before the timeout then the server returns control to the host sooner. If the timeout expires then pending connections & requests are forcibly aborted which can cause errors to be reported in the logs and to the clients. + +When working with a load balancer several steps can ensure a smooth transition of clients to a new destination. +- Bring up the new instance and start balancing traffic to it (you may already be running several instances for scale purposes) +- Disable or remove the old instance in the load balancer config so it stops receiving new traffic. +- Signal the old instance to shut down +- Wait for it to drain or timeout +(need specific examples for different environments) + ## Additional resources From e234d3fb0d978e6fa406c152fd5903d8961b345a Mon Sep 17 00:00:00 2001 From: Rick Anderson <3605364+Rick-Anderson@users.noreply.github.com> Date: Mon, 20 Mar 2023 15:48:32 -1000 Subject: [PATCH 2/5] Kestrel graceful shutdown /1 --- aspnetcore/fundamentals/servers/kestrel.md | 38 ++++++++++++++-------- 1 file changed, 24 insertions(+), 14 deletions(-) diff --git a/aspnetcore/fundamentals/servers/kestrel.md b/aspnetcore/fundamentals/servers/kestrel.md index 36a4564e4bb5..8f8a869233f8 100644 --- a/aspnetcore/fundamentals/servers/kestrel.md +++ b/aspnetcore/fundamentals/servers/kestrel.md @@ -49,21 +49,31 @@ The following timeouts and rate limits aren't enforced when a debugger is attach * * -## Graceful shutdown - -- The Host receives a shutdown signal (e.g. CTL+C, StopAsync, etc.) -- IHostApplicationLifetime.ApplicationStopping is signaled to notify the application. Long running operations should subscribe to this event. -- The Host calls IServer.StopAsync with a configurable shutdown timeout (default 30s) -- Kestrel (and Http.Sys) close their port bindings and stop accepting new connections. They also signal current connections to stop processing new requests. For HTTP/2 and HTTP/3 this involves sending a preliminary GoAway message to the client. For HTTP/1.1 requests are processed in order so it stops that connection loop. - - IIS is a little different, it rejects new requests with a 503. -- Active requests are given until the shutdown timeout to complete. If everything completes before the timeout then the server returns control to the host sooner. If the timeout expires then pending connections & requests are forcibly aborted which can cause errors to be reported in the logs and to the clients. - -When working with a load balancer several steps can ensure a smooth transition of clients to a new destination. -- Bring up the new instance and start balancing traffic to it (you may already be running several instances for scale purposes) -- Disable or remove the old instance in the load balancer config so it stops receiving new traffic. -- Signal the old instance to shut down -- Wait for it to drain or timeout +## Shutdown + +The following list details server shutdown: + +* The Host receives a shutdown signal, for example, from `CTL+C`, [StopAsync](/aspnet/core/fundamentals/host/hosted-services#stopasync), etc. +* [IHostApplicationLifetime.ApplicationStopping](xref:Microsoft.Extensions.Hosting.IHostApplicationLifetime.ApplicationStopping) is signaled to notify the app. Long running operations should subscribe to this event. +* The Host calls [IServer.StopAsync](xref:Microsoft.AspNetCore.Hosting.Server.IServer.StopAsync%2A) with a configurable shutdown timeout. The default shutdown is 30 seconds. +* Kestrel (and [Http.Sys](/aspnet/core/fundamentals/servers/#kestrel-vs-httpsys)): + * Close their port bindings and stop accepting new connections. + * Signal current connections to stop processing new requests. For HTTP/2 and HTTP/3 this involves sending a preliminary GoAway message to the client. For HTTP/1.1, requests are processed in order so it stops that connection loop. + * IIS is a little different, it rejects new requests with a [503](https://developer.mozilla.org/docs/Web/HTTP/Status/503) status. +* Active requests are given until the shutdown timeout to complete. If everything completes before the timeout then the server returns control to the host sooner. If the timeout expires then pending connections and requests are forcibly aborted. Aborting connections and requests can cause errors to be reported in the logs and to the clients. + +### Using a load balancer for graceful shutdown + +When working with a load balancer several steps can ensure a smooth transition of clients to a new destination: + +* If other instances aren't running and able to meet demand, start a new server instance. +* Start balancing traffic to the other instances. Many apps run multiple instances and [auto scaling](/azure/azure-monitor/autoscale/autoscale-get-started?toc=%2Fazure%2Fapp-service%2Ftoc.json) to meet demand. +* Disable or remove the old instance in the load balancer configuration so it stops receiving new traffic. +* Signal the old instance to shut down. +* Wait for the shutting down server to drain or timeout. + ## Additional resources From eb3c050679d3bf2195c2a05b28b39c9fae4e6f6a Mon Sep 17 00:00:00 2001 From: Rick Anderson <3605364+Rick-Anderson@users.noreply.github.com> Date: Tue, 21 Mar 2023 12:02:30 -1000 Subject: [PATCH 3/5] Kestrel graceful shutdown /1 --- aspnetcore/fundamentals/servers/kestrel.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/aspnetcore/fundamentals/servers/kestrel.md b/aspnetcore/fundamentals/servers/kestrel.md index 8f8a869233f8..e79c637be6c2 100644 --- a/aspnetcore/fundamentals/servers/kestrel.md +++ b/aspnetcore/fundamentals/servers/kestrel.md @@ -75,6 +75,8 @@ When working with a load balancer several steps can ensure a smooth transition o (need specific examples for different environments) --> +For detailed information on Generic host shutdown, see [Host shutdown](/dotnet/core/extensions/generic-host#host-shutdown). Although the [minimal hosting model has some differences with the generic host](/aspnet/core/migration/50-to-60#faq), the generic host underpins the minimal hosting model and behaves similarly on shutdown. + ## Additional resources From a8628658d57dafd14b47ed9636b49bde2190aeeb Mon Sep 17 00:00:00 2001 From: Rick Anderson <3605364+Rick-Anderson@users.noreply.github.com> Date: Mon, 27 Mar 2023 13:54:17 -1000 Subject: [PATCH 4/5] work --- aspnetcore/fundamentals/servers/kestrel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aspnetcore/fundamentals/servers/kestrel.md b/aspnetcore/fundamentals/servers/kestrel.md index e79c637be6c2..1d13c6aee369 100644 --- a/aspnetcore/fundamentals/servers/kestrel.md +++ b/aspnetcore/fundamentals/servers/kestrel.md @@ -51,7 +51,7 @@ The following timeouts and rate limits aren't enforced when a debugger is attach ## Shutdown -The following list details server shutdown: +When an ASP.NET Core app receives a shutdown signal, it goes through a series of steps to gracefully shut down the server: * The Host receives a shutdown signal, for example, from `CTL+C`, [StopAsync](/aspnet/core/fundamentals/host/hosted-services#stopasync), etc. * [IHostApplicationLifetime.ApplicationStopping](xref:Microsoft.Extensions.Hosting.IHostApplicationLifetime.ApplicationStopping) is signaled to notify the app. Long running operations should subscribe to this event. From 9b5820f04a9f48881eb74d6bd4df3a6d9db3a87d Mon Sep 17 00:00:00 2001 From: Rick Anderson <3605364+Rick-Anderson@users.noreply.github.com> Date: Tue, 4 Apr 2023 12:35:59 -1000 Subject: [PATCH 5/5] react to feedback --- aspnetcore/fundamentals/servers/kestrel.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/aspnetcore/fundamentals/servers/kestrel.md b/aspnetcore/fundamentals/servers/kestrel.md index 1d13c6aee369..4d5f760d78ba 100644 --- a/aspnetcore/fundamentals/servers/kestrel.md +++ b/aspnetcore/fundamentals/servers/kestrel.md @@ -53,9 +53,9 @@ The following timeouts and rate limits aren't enforced when a debugger is attach When an ASP.NET Core app receives a shutdown signal, it goes through a series of steps to gracefully shut down the server: -* The Host receives a shutdown signal, for example, from `CTL+C`, [StopAsync](/aspnet/core/fundamentals/host/hosted-services#stopasync), etc. +* The Host receives a shutdown signal, for example, from `CTL+C`, [StopAsync](/aspnet/core/fundamentals/host/generic-host#manage-the-host-lifetime), etc. * [IHostApplicationLifetime.ApplicationStopping](xref:Microsoft.Extensions.Hosting.IHostApplicationLifetime.ApplicationStopping) is signaled to notify the app. Long running operations should subscribe to this event. -* The Host calls [IServer.StopAsync](xref:Microsoft.AspNetCore.Hosting.Server.IServer.StopAsync%2A) with a configurable shutdown timeout. The default shutdown is 30 seconds. +* The Host calls [ShutdownTimeout](/aspnet/core/fundamentals/host/generic-host#shutdowntimeout) with a configurable shutdown timeout. The default shutdown is 30 seconds. * Kestrel (and [Http.Sys](/aspnet/core/fundamentals/servers/#kestrel-vs-httpsys)): * Close their port bindings and stop accepting new connections. * Signal current connections to stop processing new requests. For HTTP/2 and HTTP/3 this involves sending a preliminary GoAway message to the client. For HTTP/1.1, requests are processed in order so it stops that connection loop.