Trying to remote debug while also remote-dev

[edeandrea]

I used the openshift extension to deploy my app to OpenShift via an S2I binary build. The app is up and running and quarkus:remote-dev connects to it fine and any local changes are pushed/reflected as expected.

My challenge is I can't seem to connect a remote debugger to it. I can't seem to get the pod to run in debug.

I've tried lots of combinations of environment variables/arguments, but nothing works. Here's what I've tried:

• set env var JAVA_DEBUG=true
• set property quarkus.openshift.jvm-arguments=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005
• set property quarkus.openshift.jvm-arguments=-Ddebug=true

The Quarkus OpenShift config says that the base jvm image is registry.access.redhat.com/ubi8/openjdk-11. When looking at the documentation for the image I should just be able to set JAVA_DEBUG=true and it should work, but it doesn't seem to be.

Is this a Quarkus issue? An issue with the openjdk-11 image?

[quarkus-bot[bot]]

/cc @geoand, @iocanel

[geoand]

I would suggest opening a terminal in the pod and see what env vars have been set. Theoretically having JAVA_DEBUG set to true should be enough.
Also, you need to make sure that port 5005 (the default debugging port) is open and accessible from outside the cluster.

[edeandrea]

I tried the openjdk-17 image too just to see and it has the same behavior. These are the environment variables that are set in the pod:

AB_PROMETHEUS_JMX_EXPORTER_CONFIG=/opt/jboss/container/prometheus/etc/jmx-exporter-config.yaml
EVENT_STATISTICS_PORT=tcp://172.30.219.252:80
EVENT_STATISTICS_PORT_80_TCP=tcp://172.30.219.252:80
EVENT_STATISTICS_PORT_80_TCP_ADDR=172.30.219.252
EVENT_STATISTICS_PORT_80_TCP_PORT=80
EVENT_STATISTICS_PORT_80_TCP_PROTO=tcp
EVENT_STATISTICS_SERVICE_HOST=172.30.219.252
EVENT_STATISTICS_SERVICE_PORT=80
EVENT_STATISTICS_SERVICE_PORT_HTTP=80
FIGHTS_DB_PORT=tcp://172.30.235.3:5432
FIGHTS_DB_PORT_5432_TCP=tcp://172.30.235.3:5432
FIGHTS_DB_PORT_5432_TCP_ADDR=172.30.235.3
FIGHTS_DB_PORT_5432_TCP_PORT=5432
FIGHTS_DB_PORT_5432_TCP_PROTO=tcp
FIGHTS_DB_SERVICE_HOST=172.30.235.3
FIGHTS_DB_SERVICE_PORT=5432
FIGHTS_KAFKA_PORT=tcp://172.30.99.245:9092
FIGHTS_KAFKA_PORT_9092_TCP=tcp://172.30.99.245:9092
FIGHTS_KAFKA_PORT_9092_TCP_ADDR=172.30.99.245
FIGHTS_KAFKA_PORT_9092_TCP_PORT=9092
FIGHTS_KAFKA_PORT_9092_TCP_PROTO=tcp
FIGHTS_KAFKA_SERVICE_HOST=172.30.99.245
FIGHTS_KAFKA_SERVICE_PORT=9092
HEROES_DB_PORT=tcp://172.30.139.171:5432
HEROES_DB_PORT_5432_TCP=tcp://172.30.139.171:5432
HEROES_DB_PORT_5432_TCP_ADDR=172.30.139.171
HEROES_DB_PORT_5432_TCP_PORT=5432
HEROES_DB_PORT_5432_TCP_PROTO=tcp
HEROES_DB_SERVICE_HOST=172.30.139.171
HEROES_DB_SERVICE_PORT=5432
HOME=/home/jboss
HOSTNAME=rest-fights-3-8tjs4
JAVA_APP_JAR=/deployments/quarkus-run.jar
JAVA_DATA_DIR=/deployments/data
JAVA_DEBUG=true
JAVA_HOME=/usr/lib/jvm/java-17
JAVA_VENDOR=openjdk
JAVA_VERSION=17
JBOSS_CONTAINER_JAVA_JVM_MODULE=/opt/jboss/container/java/jvm
JBOSS_CONTAINER_JAVA_PROXY_MODULE=/opt/jboss/container/java/proxy
JBOSS_CONTAINER_JAVA_RUN_MODULE=/opt/jboss/container/java/run
JBOSS_CONTAINER_JAVA_S2I_MODULE=/opt/jboss/container/java/s2i
JBOSS_CONTAINER_MAVEN_36_MODULE=/opt/jboss/container/maven/36/
JBOSS_CONTAINER_MAVEN_DEFAULT_MODULE=/opt/jboss/container/maven/default/
JBOSS_CONTAINER_MAVEN_S2I_MODULE=/opt/jboss/container/maven/s2i
JBOSS_CONTAINER_OPENJDK_JDK_MODULE=/opt/jboss/container/openjdk/jdk
JBOSS_CONTAINER_PROMETHEUS_MODULE=/opt/jboss/container/prometheus
JBOSS_CONTAINER_S2I_CORE_MODULE=/opt/jboss/container/s2i/core/
JBOSS_CONTAINER_UTIL_LOGGING_MODULE=/opt/jboss/container/util/logging/
JBOSS_IMAGE_NAME=ubi8/openjdk-17
JBOSS_IMAGE_VERSION=1.11
KUBERNETES_NAMESPACE=eric-deandrea-dev
KUBERNETES_PORT=tcp://172.30.0.1:443
KUBERNETES_PORT_443_TCP=tcp://172.30.0.1:443
KUBERNETES_PORT_443_TCP_ADDR=172.30.0.1
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_SERVICE_HOST=172.30.0.1
KUBERNETES_SERVICE_PORT=443
KUBERNETES_SERVICE_PORT_HTTPS=443
LANG=C.utf8
MAVEN_VERSION=3.6
NSS_SDB_USE_CACHE=no
OPENSHIFT_BUILD_NAME=rest-fights-1
OPENSHIFT_BUILD_NAMESPACE=eric-deandrea-dev
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/s2i
PWD=/home/jboss
QUARKUS_DATASOURCE_PASSWORD=superfight
QUARKUS_DATASOURCE_USERNAME=superfight
QUARKUS_LAUNCH_DEVMODE=true
REST_FIGHTS_PORT=tcp://172.30.87.238:80
REST_FIGHTS_PORT_80_TCP=tcp://172.30.87.238:80
REST_FIGHTS_PORT_80_TCP_ADDR=172.30.87.238
REST_FIGHTS_PORT_80_TCP_PORT=80
REST_FIGHTS_PORT_80_TCP_PROTO=tcp
REST_FIGHTS_SERVICE_HOST=172.30.87.238
REST_FIGHTS_SERVICE_PORT=80
REST_FIGHTS_SERVICE_PORT_HTTP=80
REST_HEROES_PORT=tcp://172.30.136.232:80
REST_HEROES_PORT_80_TCP=tcp://172.30.136.232:80
REST_HEROES_PORT_80_TCP_ADDR=172.30.136.232
REST_HEROES_PORT_80_TCP_PORT=80
REST_HEROES_PORT_80_TCP_PROTO=tcp
REST_HEROES_SERVICE_HOST=172.30.136.232
REST_HEROES_SERVICE_PORT=80
REST_HEROES_SERVICE_PORT_HTTP=80
REST_VILLAINS_PORT=tcp://172.30.68.163:80
REST_VILLAINS_PORT_80_TCP=tcp://172.30.68.163:80
REST_VILLAINS_PORT_80_TCP_ADDR=172.30.68.163
REST_VILLAINS_PORT_80_TCP_PORT=80
REST_VILLAINS_PORT_80_TCP_PROTO=tcp
REST_VILLAINS_SERVICE_HOST=172.30.68.163
REST_VILLAINS_SERVICE_PORT=80
REST_VILLAINS_SERVICE_PORT_HTTP=80
S2I_SOURCE_DEPLOYMENTS_FILTER=*.jar
SHLVL=2
TERM=xterm
UI_SUPER_HEROES_PORT=tcp://172.30.169.200:80
UI_SUPER_HEROES_PORT_80_TCP=tcp://172.30.169.200:80
UI_SUPER_HEROES_PORT_80_TCP_ADDR=172.30.169.200
UI_SUPER_HEROES_PORT_80_TCP_PORT=80
UI_SUPER_HEROES_PORT_80_TCP_PROTO=tcp
UI_SUPER_HEROES_SERVICE_HOST=172.30.169.200
UI_SUPER_HEROES_SERVICE_PORT=80
UI_SUPER_HEROES_SERVICE_PORT_HTTP=80
VILLAINS_DB_PORT=tcp://172.30.38.103:5432
VILLAINS_DB_PORT_5432_TCP=tcp://172.30.38.103:5432
VILLAINS_DB_PORT_5432_TCP_ADDR=172.30.38.103
VILLAINS_DB_PORT_5432_TCP_PORT=5432
VILLAINS_DB_PORT_5432_TCP_PROTO=tcp
VILLAINS_DB_SERVICE_HOST=172.30.38.103
VILLAINS_DB_SERVICE_PORT=5432
_=/usr/bin/env
container=oci
fight.villain.client-base-url=http://rest-villains
kafka.bootstrap.servers=PLAINTEXT://fights-kafka:9092
quarkus.datasource.reactive.url=postgresql://fights-db:5432/fights_database
quarkus.hibernate-orm.database.generation=none
quarkus.hibernate-orm.sql-load-script=no-file
quarkus.rest-client.hero-client.url=http://rest-heroes

This is the startup logs from the pod:

Press [h] for more options>
13:03:54 WARN  [io.qu.ku.de.KubernetesProcessor] (build-40) No project was detected, skipping generation of kubernetes manifests!
13:03:58 WARN  [io.qu.mi.de.bi.mp.MicroprofileMetricsProcessor] (build-27) This application uses the MP Metrics API. The micrometer extension currently provides a compatibility layer that supports the MP Metrics API, but metric names and recorded values will be different. Note that the MP Metrics compatibility layer will move to a different extension in the future.
13:03:58 INFO  [io.qu.sm.re.ka.de.SmallRyeReactiveMessagingKafkaProcessor] (build-7) Generating Jackson serializer for type io.quarkus.sample.superheroes.fight.Fight
__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
13:04:04 INFO  [or.hi.re.pr.im.ReactiveIntegrator] (JPA Startup Thread: default-reactive) HR000001: Hibernate Reactive
13:04:05 INFO  [io.sm.re.me.kafka] (Quarkus Main Thread) SRMSG18258: Kafka producer kafka-producer-fights, connected to Kafka brokers 'PLAINTEXT://fights-kafka:9092', is configured to write records to 'fights'
13:04:05 INFO  [io.quarkus] (Quarkus Main Thread) rest-fights 1.0 on JVM (powered by Quarkus 2.6.3.Final) started in 15.558s. Listening on: http://0.0.0.0:8082
13:04:05 INFO  [io.quarkus] (Quarkus Main Thread) Profile dev activated. Live Coding activated.
13:04:05 INFO  [io.quarkus] (Quarkus Main Thread) Installed features: [cdi, hibernate-orm, hibernate-reactive, hibernate-reactive-panache, hibernate-validator, jaxrs-client-reactive, kafka-client, kubernetes, micrometer, reactive-pg-client, rest-client-reactive, rest-client-reactive-jackson, resteasy-reactive, resteasy-reactive-jackson, smallrye-context-propagation, smallrye-fault-tolerance, smallrye-health, smallrye-openapi, smallrye-reactive-messaging, smallrye-reactive-messaging-kafka, swagger-ui, vertx]

I'm doing an oc port-forward of port 5005 from the pod to localhost and then trying to attach the debugger to localhost:5005.

 oc port-forward rest-fights-3-8tjs4 5005:5005
Forwarding from 127.0.0.1:5005 -> 5005
Forwarding from [::1]:5005 -> 5005
Handling connection for 5005
E0128 08:05:58.515254    9128 portforward.go:400] an error occurred forwarding 5005 -> 5005: error forwarding port 5005 to pod a84b23061855fdc25dcb8eede15f901562e7908c1b9df3e546299b5b0446acf5, uid : port forward into network namespace "/var/run/netns/484c6f8b-21cc-473a-8912-dd89ad001405": failed to connect to localhost:5005 inside namespace a84b23061855fdc25dcb8eede15f901562e7908c1b9df3e546299b5b0446acf5: dial tcp [::1]:5005: connect: connection refused

[edeandrea]

It also doesn't seem to matter whether I am trying to do remote dev or not. Even just a "regular" app I add the JAVA_DEBUG=true env variable (on both the registry.access.redhat.com/ubi8/openjdk-11-runtime:1.10 & registry.access.redhat.com/ubi8/openjdk-17-runtime:1.10 images) and I still do not get the JVM to run debug.

[maxandersen]

This is s2i issue - but This should work assuming the Java process is started with run-Java

[edeandrea]

I agree it should work. But it doesn't. I don't know what happens behind the scenes.

[edeandrea]

@maxandersen Should I open an issue on the Quarkus issue tracker for this?

[edeandrea]

This looks like its been resolved now (I closed #23765).

The DeploymentConfig no longer specified the command.

The latest code.quarkus.io also generates a different Dockerfile - one that's based on the OpenJDK image and does not override the ENTRYPOINT.