JBoss starting time too long

[palonsoro]

I have been testing this buildpack for some time and I have not been able to deploy even a simple hello world JSP. The problem is that the JBoss starting process takes too much time, so the health check fails even when set to 180 seconds.

We are working on a private Cloud Foundry instance, which did not have any performance issue with the normal Java buildpack even with much larger applications. The application has been given 1GB RAM, so that should also not be the problem.

Would it be possible to shorten that time in any way, so that the application can start normally?

Thank you in advance

[palonsoro]

We have left the application instance untouched for some hours after having tried to start it. It seems now to be started, but it uses a 400% CPU (100% of 4 cores).

Through the administration interface, we could guess the IP of the VM where JBoss is running and we could start a SSH session to it. What we found there is that the JBoss process is using the whole CPU and it has been doing it since it started. If we examine it in more detail, we find that only some of the threads are running and all of them try to use as mach CPU as possible, although not all of them seem to do the same kind of task.

Here, we attach an extract from the thread dump containing only running threads:

Full thread dump OpenJDK 64-Bit Server VM (25.31-b07 mixed mode):

...

"http--0.0.0.0-61706-Acceptor-0" #81 daemon prio=5 os_prio=0 tid=0x00007ff8fc19f800 nid=0xa8 runnable [0x00007ff8ef51e000]
   java.lang.Thread.State: RUNNABLE
    at java.net.PlainSocketImpl.socketAccept(Native Method)
    at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404)
    at java.net.ServerSocket.implAccept(ServerSocket.java:545)
    at java.net.ServerSocket.accept(ServerSocket.java:513)
    at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
    at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:321)
    at java.lang.Thread.run(Thread.java:745)

...

"Remoting "1889bsii09p" write-1" #74 prio=5 os_prio=0 tid=0x000000000209d000 nid=0xa2 runnable [0x00007ff908224000]
   java.lang.Thread.State: RUNNABLE
    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
    - locked <0x00000000f1031d90> (a sun.nio.ch.Util$2)
    - locked <0x00000000f1031d80> (a java.util.Collections$UnmodifiableSet)
    - locked <0x00000000f1031c68> (a sun.nio.ch.EPollSelectorImpl)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
    at org.xnio.nio.WorkerThread.run(WorkerThread.java:153)
"Remoting "1889bsii09p" read-1" #73 prio=5 os_prio=0 tid=0x0000000001fa6800 nid=0xa1 runnable [0x00007ff908327000]
   java.lang.Thread.State: RUNNABLE
    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
    - locked <0x00000000f102d200> (a sun.nio.ch.Util$2)
    - locked <0x00000000f102d1f0> (a java.util.Collections$UnmodifiableSet)
    - locked <0x00000000f102d0d8> (a sun.nio.ch.EPollSelectorImpl)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
    at org.xnio.nio.WorkerThread.run(WorkerThread.java:153)
"ServerService Thread Pool -- 49" #72 prio=5 os_prio=0 tid=0x00007ff8f8077000 nid=0xa0 runnable [0x00007ff908427000]
   java.lang.Thread.State: RUNNABLE
    at java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:685)
    at java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:823)
    at java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1962)
    at java.util.concurrent.ConcurrentSkipListSet.add(ConcurrentSkipListSet.java:241)
    at org.jboss.as.server.DeployerChainAddHandler.addDeploymentProcessor(DeployerChainAddHandler.java:55)
    at org.jboss.as.server.AbstractDeploymentChainStep$1.addDeploymentProcessor(AbstractDeploymentChainStep.java:40)
    at org.jboss.as.weld.WeldSubsystemAdd$1.execute(WeldSubsystemAdd.java:63)
    at org.jboss.as.server.AbstractDeploymentChainStep.execute(AbstractDeploymentChainStep.java:45)
    at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:385)
    at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:272)
    at org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:200)
    at org.jboss.as.controller.ParallelBootOperationStepHandler$ParallelBootTask.run(ParallelBootOperationStepHandler.java:311)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:122)
"ServerService Thread Pool -- 48" #71 prio=5 os_prio=0 tid=0x00007ff8f8075800 nid=0x9f runnable [0x00007ff908528000]
   java.lang.Thread.State: RUNNABLE
    at java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:685)
    at java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:823)
    at java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1962)
    at java.util.concurrent.ConcurrentSkipListSet.add(ConcurrentSkipListSet.java:241)
    at org.jboss.as.server.DeployerChainAddHandler.addDeploymentProcessor(DeployerChainAddHandler.java:55)
    at org.jboss.as.server.AbstractDeploymentChainStep$1.addDeploymentProcessor(AbstractDeploymentChainStep.java:40)
    at org.jboss.as.webservices.dmr.WSDeploymentActivator.activate(WSDeploymentActivator.java:69)
    at org.jboss.as.webservices.dmr.WSSubsystemAdd$1.execute(WSSubsystemAdd.java:118)
    at org.jboss.as.server.AbstractDeploymentChainStep.execute(AbstractDeploymentChainStep.java:45)
    at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:385)
    at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:272)
    at org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:200)
    at org.jboss.as.controller.ParallelBootOperationStepHandler$ParallelBootTask.run(ParallelBootOperationStepHandler.java:311)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:122)
"ServerService Thread Pool -- 47" #70 prio=5 os_prio=0 tid=0x00007ff8f8074800 nid=0x9e runnable [0x00007ff908629000]
   java.lang.Thread.State: RUNNABLE
    at java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:685)
    at java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:823)
    at java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1962)
    at java.util.concurrent.ConcurrentSkipListSet.add(ConcurrentSkipListSet.java:241)
    at org.jboss.as.server.DeployerChainAddHandler.addDeploymentProcessor(DeployerChainAddHandler.java:55)
    at org.jboss.as.server.AbstractDeploymentChainStep$1.addDeploymentProcessor(AbstractDeploymentChainStep.java:40)
    at org.jboss.as.web.WebSubsystemAdd$1.execute(WebSubsystemAdd.java:108)
    at org.jboss.as.server.AbstractDeploymentChainStep.execute(AbstractDeploymentChainStep.java:45)
    at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:385)
    at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:272)
    at org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:200)
    at org.jboss.as.controller.ParallelBootOperationStepHandler$ParallelBootTask.run(ParallelBootOperationStepHandler.java:311)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:122)
"ServerService Thread Pool -- 46" #69 prio=5 os_prio=0 tid=0x00007ff8f805d000 nid=0x9d runnable [0x00007ff90872a000]
   java.lang.Thread.State: RUNNABLE
    at java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:685)
    at java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:823)
    at java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1962)
    at java.util.concurrent.ConcurrentSkipListSet.add(ConcurrentSkipListSet.java:241)
    at org.jboss.as.server.DeployerChainAddHandler.addDeploymentProcessor(DeployerChainAddHandler.java:55)
    at org.jboss.as.server.AbstractDeploymentChainStep$1.addDeploymentProcessor(AbstractDeploymentChainStep.java:40)
    at org.jboss.as.txn.subsystem.TransactionSubsystemAdd$1.execute(TransactionSubsystemAdd.java:188)
    at org.jboss.as.server.AbstractDeploymentChainStep.execute(AbstractDeploymentChainStep.java:45)
    at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:385)
    at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:272)
    at org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:200)
    at org.jboss.as.controller.ParallelBootOperationStepHandler$ParallelBootTask.run(ParallelBootOperationStepHandler.java:311)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:122)
...

Would this information be useful?
Thank you in advance.

[dwu-pivotal]

Similar issue.. I've put -t 600 while pushing, no luck to start it up. Have you made any progress on that?

[pivotal-choonghyun-oh]

JBoss 7.1.1 is not working with JDK 1.8.x version and you should add the following env into your manifest.yml and it will work well.

env:
JBP_CONFIG_OPEN_JDK_JRE: '{jre: { version: 1.7.0_+ }}'

---
applications:
- name: jayden-jboss
  buildpack: https://github.com/cloudfoundry-community/jboss-buildpack.git
  path: /Users/coh/jboss/cf/sample/sample.war
  memory: 512M
  timeout: 180
  env:
   JBP_CONFIG_OPEN_JDK_JRE: '{jre: { version: 1.7.0_+ }}'