EKS 1.23 RunnerDeployment "The connection to the server localhost:8080 was refused - did you specify the right host or port?"

[williamsun-hha]

The GitHub Actions workflow deployment works 100% time when the runner is on VM.

When the runner is on EKS.

The 1st time when the code is pushed, then I got the following error

Run kubectl apply -f xyz/ --namespace ***
The connection to the server localhost:8080 was refused - did you specify the right host or port?
Error: Process completed with exit code 1.

Then I re-run the failed job from github, the job can be completed successfully.

After the job finished, runner container will be deleted and a new runner container started, then the above process repeats itself. Can you please point me to the right direction? How can I prevent the container to be deleted after the job run?

Thank you very much!

[kevholmes]

Without seeing your workflows it's hard to really give you a ton of input here @williamsun-hha. I'm pretty sure you have an issue with whatever's providing your kube context not being picked up, which is why you are having that The connection to the server localhost:8080 was refused - did you specify the right host or port? error regarding localhost:8080... (you aren't connecting to a local k8s cluster in the runner right?) Are you providing that kube context in another step/job that does succeed but isn't correctly depended on by a downstream step using its outputs? This is me just thinking out loud but that would explain why the re-run failed steps approach is working?

You can make the runner pods stay alive after a job completes but then you are somewhat defeating the purpose of ARC? One of the nicest parts about this project is that you can easily get started with ephemeral runners and Actions using something like k8s. If that's really what you want it's possible to do by changing the RunnerDeployment/RunnerSet resources.

[williamsun-hha]

Thank you very much for your help, Kevin! The action runner is good. I found out a bug in my code caused the pod crash. Really appreciate your help and have a nice day!