Quarkus container shuts down immediately after deployment & startup

[jehrenzweig-leagueapps]

I have a weird one that I was hoping ya'll might know something about. For some reason, our Quarkus/Kotlin application -- which is deployed as a container to GCP -- is shutting down immediately after successfully deploying.

Here are the most recent logs, with timestamps removed to improve readability:

__ ____ __ _____ ___ __ ____ ______
--/ __ \/ / / / _ | / _ \/ //_/ / / / __/
-/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
INFO [com.lea.ser.bra.app.ApplicationStartup] (main) [ApplicationStartup.main()] Entered method call.
INFO [io.qua.sma.ope.run.OpenApiRecorder] (main) CORS filtering is disabled and cross-origin resource sharing is allowed without restriction, which is not recommended in production. Please configure the CORS filter through 'quarkus.http.cors.*' properties. For more information, see Quarkus HTTP CORS documentation
INFO [io.quarkus] (main) widget-service e6dc04c on JVM (powered by Quarkus 2.16.3.Final) started in 4.392s. Listening on: http://0.0.0.0:5678
INFO [io.quarkus] (main) Profile dev activated.
INFO [io.quarkus] (main) Installed features: [agroal, cdi, config-yaml, flyway, google-cloud-secret-manager, jdbc-mysql, kotlin, micrometer, narayana-jta, resteasy-reactive, resteasy-reactive-jackson, smallrye-context-propagation, smallrye-health, smallrye-openapi, swagger-ui, vertx]
INFO [com.lea.ser.bra.app.ApplicationStartup] (main) [ApplicationInitializer.run()] Entered method call.
INFO [com.lea.ser.bra.app.ApplicationStartup] (main) [ApplicationInitializer.run()] Exiting method call.
INFO [io.quarkus] (Shutdown thread) widget-service stopped in 0.090s

Our application has a very basic entry-point class that's decorated with the @QuarkusMain annotation:

ApplicationStartup.kt (entry-point class)

import io.quarkus.runtime.Quarkus
import io.quarkus.runtime.QuarkusApplication
import io.quarkus.runtime.annotations.QuarkusMain
import org.slf4j.LoggerFactory

/**
 * More info @ "[Application Initialization and Termination](https://quarkus.io/guides/lifecycle)" guide
 */
@QuarkusMain
class ApplicationStartup {
	companion object {
		private val logger = LoggerFactory.getLogger(ApplicationStartup::class.java)

		@JvmStatic
		fun main(args: Array<String>) {
			logger.info("[ApplicationStartup.main()] Entered method call.")
			Quarkus.run(ApplicationInitializer::class.java, *args)
			logger.info("[ApplicationStartup.main()] Exiting method call.")
		}
	}

	class ApplicationInitializer : QuarkusApplication {
		override fun run(vararg args: String): Int {
			logger.info("[ApplicationInitializer.run()] Entered method call.")
			Quarkus.waitForExit()
			logger.info("[ApplicationInitializer.run()] Exiting method call.")
			return 0
		}
	}
}

There are no reasons given about why Quarkus was shutdown. But it does appear that Quarkus.waitForExit() is getting the command to shutdown from somewhere, somehow.

Any idea how to start debugging this issue, or where to look? Thanks in advance!

[quarkus-bot[bot]]

/cc @evanchooly (kotlin), @geoand (kotlin)

[maxandersen]

Try running with debug or trace enabled. That might give a better hint.

[jehrenzweig-leagueapps]

I'm getting more log info now, but nothing that points at where the shutdown request was initiated:

GCP logs with quarkus.log.level=TRACE

DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-1) requestRouted null HttpRequestMetric [initialPath=/q/health/ready, currentRoutePath=null, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@a5d17d8]
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-1) requestRouted /q/ HttpRequestMetric [initialPath=/q/health/ready, currentRoutePath=null, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@a5d17d8]
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-0) requestRouted null HttpRequestMetric [initialPath=/q/health/live, currentRoutePath=null, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@16299a14]
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-0) requestRouted null HttpRequestMetric [initialPath=/q/health/live, currentRoutePath=null, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@16299a14]
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-0) requestRouted /q/ HttpRequestMetric [initialPath=/q/health/live, currentRoutePath=null, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@16299a14]
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-0) requestRouted /health/live HttpRequestMetric [initialPath=/q/health/live, currentRoutePath=/q/, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@16299a14]
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-1) requestRouted /health/ready HttpRequestMetric [initialPath=/q/health/ready, currentRoutePath=/q/, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@a5d17d8]
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-0) responseEnd io.vertx.core.http.impl.Http1xServerResponse@1fd9b2f6, HttpRequestMetric [initialPath=/q/health/live, currentRoutePath=/health/live, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@16299a14]
DEBUG [io.qua.mic.run.bin.RequestMetricInfo] (vert.x-eventloop-thread-0) Path /q/health/live ignored; matches pattern /q/.*
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-1) responseEnd io.vertx.core.http.impl.Http1xServerResponse@749d5f7c, HttpRequestMetric [initialPath=/q/health/ready, currentRoutePath=/health/ready, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@a5d17d8]
DEBUG [io.qua.mic.run.bin.RequestMetricInfo] (vert.x-eventloop-thread-1) Path /q/health/ready ignored; matches pattern /q/.*
INFO  [com.lea.ser.bra.app.ApplicationStartup] (main) [ApplicationInitializer.run()] Exiting method call.
DEBUG [io.qua.run.Application] (main) Stopping application
DEBUG [io.qua.run.shu.ShutdownRecorder] (main) Attempting to gracefully shutdown.
DEBUG [io.qua.run.Application] (Shutdown thread) Stopping application
DEBUG [io.qua.arc.impl] (main) ArC DI container shut down
DEBUG [io.net.buf.PoolThreadCache] (vert.x-eventloop-thread-0) Freed 3 thread-local buffer(s) from thread: vert.x-eventloop-thread-0
DEBUG [io.net.buf.PoolThreadCache] (vert.x-eventloop-thread-1) Freed 11 thread-local buffer(s) from thread: vert.x-eventloop-thread-1
DEBUG [io.qua.thread-pool] (main) loop: 1, remaining: 60000000000, intervalRemaining: 5000000000, interruptRemaining: 10000000000
INFO  [io.quarkus] (main) widget-service stopped in 0.088s
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-1) requestRouted null HttpRequestMetric [initialPath=/q/metrics, currentRoutePath=null, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@70d623f3]
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-1) requestRouted null HttpRequestMetric [initialPath=/q/metrics, currentRoutePath=null, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@70d623f3]
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-1) requestRouted /q/ HttpRequestMetric [initialPath=/q/metrics, currentRoutePath=null, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@70d623f3]
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-1) requestRouted /metrics HttpRequestMetric [initialPath=/q/metrics, currentRoutePath=/q/, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@70d623f3]
DEBUG [io.qua.mic.run.bin.ver.VertxHttpServerMetrics] (vert.x-eventloop-thread-1) responseEnd io.vertx.core.http.impl.Http1xServerResponse@1f5e8d79, HttpRequestMetric [initialPath=/q/metrics, currentRoutePath=/metrics, templatePath=null, request=io.vertx.core.http.impl.Http1xServerRequest@70d623f3]
DEBUG [io.qua.mic.run.bin.RequestMetricInfo] (vert.x-eventloop-thread-1) Path /q/metrics ignored; matches pattern /q/.*

Do the health check logs look right? I wasn't sure if those routing messages were part of normal behavior.

[geoand]

I don't see any health check logs in there, they all seem to be metrics related.

[jehrenzweig-leagueapps]

Whoops, you're right -- I said health check, I meant metrics.

[geoand]

The metrics logs do look correct to me

[jehrenzweig-leagueapps]

Great, just wanted a sanity check -- thank you!

[jehrenzweig-leagueapps]

Closing this discussion (for now, anyway). If there's no obvious way of discerning why Quarkus is shutting down, then I'll probably try deploying a more boilerplate API and see if that works. If even a basic Quarkus API won't work when deployed, it's probably an issue with our deployment process; we have at least one other Quarkus app deployed, so they're obviously doing it right. And if a basic Quarkus API does work, I'll start laying more of my actual API on top of it, to see if there's a specific point at which it starts misbehaving.

Thanks ya'll for taking a peek!