High CPU usage after switching from openj9 21.0.8+9 to 21.0.9+10

[klim-ibm]

$ java -version
openjdk version "21.0.9" 2025-10-21 LTS
IBM Semeru Runtime Open Edition 21.0.9.0 (build 21.0.9+10-LTS)
Eclipse OpenJ9 VM 21.0.9.0 (build 21.0.9+10-openj9-0.56.0, JRE 21 Linux amd64-64-Bit Compressed References 20251021_630 (JIT enabled, AOT enabled)
OpenJ9   - 14b3b2de26
OMR      - d4c7e3040
JCL      - 7a630fb0f55 based on jdk-21.0.9+10)

After switching to the above openj9 version, the java process of our server almost immediately goes to 200% CPU usage:

top - 12:59:26 up 10 days, 15:48,  0 users,  load average: 2.00, 2.00, 1.73
Tasks:  11 total,   1 running,  10 sleeping,   0 stopped,   0 zombie
%Cpu(s):  50.2/0.5    51[                                                                                                    ]
MiB Mem :  15353.2 total,    354.5 free,   3043.7 used,  12304.3 buff/cache
MiB Swap:  16380.0 total,  16379.2 free,      0.8 used.  12309.4 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
    148 sagadmin  20   0 3959736 330052  30592 S 199.7   2.1  53:13.06 java
    124 sagadmin  20   0   86888   3712   2688 S   0.3   0.0   0:01.87 wrapper-3.5.53
      1 sagadmin  20   0    4200   3200   2944 S   0.0   0.0   0:00.02 umstart.sh
      9 sagadmin  20   0    4944   2432   2304 S   0.0   0.0   0:00.00 tail
     10 sagadmin  20   0    3652   2048   1920 S   0.0   0.0   0:00.00 sed
     11 sagadmin  20   0    4944   2688   2560 S   0.0   0.0   0:00.04 tail
     12 sagadmin  20   0    3652   2048   1920 S   0.0   0.0   0:00.03 sed
     44 sagadmin  20   0    4200   3200   2944 S   0.0   0.0   0:00.00 nserver
     58 sagadmin  20   0    5388   4352   2944 S   0.0   0.0   0:00.00 nserverdaemon
   1102 sagadmin  20   0    4464   3712   3200 S   0.0   0.0   0:00.02 bash
   1114 sagadmin  20   0    7308   3712   3072 R   0.0   0.0   0:00.02 top

The issue seems to be caused by calling LockSupport.parkNanos(Constants.LOCK_WAIT) with Constants.LOCK_WAIT=500000. It's possible that values below 1ms are not honored and the thread that was supposed to be parked is instead occupying all available CPU cycles.

The issue has also been reproduced with the milestone releases https://github.com/ibmruntimes/semeru21-binaries/releases/tag/jdk-21.0.9%2B7_openj9-0.56.0-m1 and https://github.com/ibmruntimes/semeru21-binaries/releases/tag/jdk-21.0.9%2B9_openj9-0.56.0-m2.

The issue is not affecting our product in the previous OpenJ9 21.0.8+9 release.

[pshipton]

Any ideas what I can do to reproduce this problem? I tried something simple but it doesn't take any CPU.

import java.util.concurrent.locks.*;

public class TestPark {
public static void main(String[] args) throws Exception {
        new TestPark().run();
}
public void run() throws Exception {
        while(true) {
                LockSupport.park(500000);
        }
}
}

[pshipton]

Is there some particular OS or container where this occurs?

[pshipton]

@tajila any ideas for this one?

[tajila]

@thallium Please take a look at this.

[klim-ibm]

One of my co-workers provided this zip file with a small sample project that should reproduce the issue. Let me know if it helps.

parknanos.zip

[pshipton]

That should be a great help. I can reproduce the CPU issue on Linux, which means we can track down what change caused it.

I see the cpu usage on 25.0.1, but not 25.0, so the change causing it should be in
https://github.com/eclipse-openj9/openj9/releases/tag/openj9-0.56.0
https://github.com/eclipse-openj9/openj9-omr/releases/tag/openj9-0.56.0

In fact, after reverting the following two changes, the problem is gone.
eclipse-openj9/openj9-omr#245
#22569

[pshipton]

Creating a jdk21 xlinux and windows 0.56 build with those changes reverted.
https://openj9-jenkins.osuosl.org/job/Pipeline-Build-Test-Personal/785/

[pshipton]

The open source builds are available for testing, these don't contains any OpenJCEPlus or FIPS support.
https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Build_JDK21_x86-64_linux_Personal/920/OpenJ9-JDK21-x86-64_linux-20251219-173459.tar.gz
https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Build_JDK21_x86-64_windows_Personal/664/OpenJ9-JDK21-x86-64_windows-20251219-123610.tar.gz

[thallium]

Can confirm that eclipse-omr/omr#8000 fixes this issue.

[pshipton]

The fix is included in 0.57 (for the upcoming January release), the milestone drivers are available for testing.
https://github.com/ibmruntimes/semeru21-binaries/releases/tag/jdk-21.0.10%2B5_openj9-0.57.0-m2