Health check on port 8080 has failed. Your app is not responding properly.

[alxvallejo]

Not sure what changed, other than a failed and then fixed migration in the deployment, but if I inspect the VM via flyctl vm status {recentBuildVmId} then I see this:

Instance
  ID            = 7dc27308
  Process       = app
  Version       = 18
  Region        = mia
  Desired       = run
  Status        = failed
  Health Checks = 2 total, 2 critical
  Restarts      = 2
  Created       = 10h40m ago

Events
TIMESTAMP           	TYPE           	MESSAGE
2023-05-10T02:41:01Z	Received       	Task received by client
2023-05-10T02:41:01Z	Task Setup     	Building Task Directory
2023-05-10T02:41:05Z	Started        	Task started by client
2023-05-10T02:41:17Z	Terminated     	Exit Code: 1
2023-05-10T02:41:17Z	Restarting     	Task restarting in 1.109847978s
2023-05-10T02:41:23Z	Started        	Task started by client
2023-05-10T02:41:37Z	Terminated     	Exit Code: 1
2023-05-10T02:41:37Z	Restarting     	Task restarting in 1.11169659s
2023-05-10T02:41:42Z	Started        	Task started by client
2023-05-10T02:41:54Z	Terminated     	Exit Code: 137
2023-05-10T02:41:54Z	Not Restarting 	Exceeded allowed attempts 2 in interval 5m0s and mode is "fail"
2023-05-10T02:41:54Z	Alloc Unhealthy	Unhealthy because of failed task
2023-05-10T02:41:54Z	Killing        	Sent interrupt. Waiting 5s before force killing

Checks
ID                              	SERVICE 	STATE   	OUTPUT
01ec42b539ad1e4fc12b4b6dd7e3ac93	tcp-8080	critical	Get "http://172.19.68.58:8080/": dial tcp 172.19.68.58:8080: connect: connection refused
3df2415693844068640885b45074b954	tcp-8080	critical	dial tcp 172.19.68.58:8080: connect: connection refused

And from the fly.io deployment logs:

2023-05-10T02:41:42.267 app[7dc27308] mia [info] Starting init (commit: 127ec32)...

2023-05-10T02:41:42.286 app[7dc27308] mia [info] Mounting /dev/vdc at /data w/ uid: 0, gid: 0 and chmod 0755

2023-05-10T02:41:42.288 app[7dc27308] mia [info] Preparing to run: `sh start.sh` as root

2023-05-10T02:41:42.305 app[7dc27308] mia [info] 2023/05/10 02:41:42 listening on [fdaa:1:33ca:a7b:9f:4:7ad0:2]:22 (DNS: [fdaa::3]:53)

2023-05-10T02:41:42.308 app[7dc27308] mia [info] + npx prisma migrate deploy

2023-05-10T02:41:45.089 app[7dc27308] mia [info] npm WARN exec The following package was not found and will be installed: [email protected]

2023-05-10T02:41:46.943 health[7dc27308] mia [error] Health check on port 8080 has failed. Your app is not responding properly. Services exposed on ports [80, 443] will have intermittent failures until the health check passes.

2023-05-10T02:41:48.941 health[7dc27308] mia [error] Health check on port 8080 has failed. Your app is not responding properly. Services exposed on ports [80, 443] will have intermittent failures until the health check passes.

2023-05-10T02:41:52.425 app[7dc27308] mia [info] [ 10.229295] Out of memory: Killed process 567 (node) total-vm:11300528kB, anon-rss:122332kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:1420kB oom_score_adj:0

2023-05-10T02:41:52.489 app[7dc27308] mia [info] Killed

2023-05-10T02:41:52.504 app[7dc27308] mia [info] npm notice

2023-05-10T02:41:52.504 app[7dc27308] mia [info] npm notice New major version of npm available! 8.19.4 -> 9.6.6

2023-05-10T02:41:52.505 app[7dc27308] mia [info] npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.6.6>

2023-05-10T02:41:52.505 app[7dc27308] mia [info] npm notice Run `npm install -g [email protected]` to update!

2023-05-10T02:41:52.506 app[7dc27308] mia [info] npm notice

2023-05-10T02:41:53.337 app[7dc27308] mia [info] Starting clean up.

2023-05-10T02:41:53.337 app[7dc27308] mia [info] Umounting /dev/vdc from /data

So my first question is - why is there a healthcheck on port 8080 when the public port is 80/443?

Also, under fly.toml, why is the path for the healthcheck /healthcheck, when that path doesn't exist in my application? I tried setting it to the root path but same error.

The other thing I noticed, and I didn't change it, is that the internal port in fly.toml was originally set to 8_080. Weird. So I changed it to 8080 but same error.

Not sure what other information would be required here, but it does run locally. Maybe the migrations screwed something up? Not sure how to debug from here though.

[courtsimas]

Literally in the same boat. Wondering the same thing.

[remorses]

If you use the [http_service] field you have to pass checks this way instead of [[services.http_checks]]:

[[http_service.checks]]
interval = "5s"
grace_period = "4s"
method = "post"
path = "/health"
protocol = "http"
timeout = "5s"
tls_skip_verify = false


[http_service]
internal_port = 3000
force_https = true
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
processes = ["app"]