Increase memory to avoid DaemonDisappearedException on GitLab CI#10177
Increase memory to avoid DaemonDisappearedException on GitLab CI#10177
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 8 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~00495f3e1c, baseline=1.57.0-SNAPSHOT~1ba60ee63f
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.083 s) : 0, 1083370
Total [baseline] (8.816 s) : 0, 8816442
Agent [candidate] (1.092 s) : 0, 1091753
Total [candidate] (8.832 s) : 0, 8832176
section iast
Agent [baseline] (1.224 s) : 0, 1224054
Total [baseline] (9.398 s) : 0, 9397698
Agent [candidate] (1.221 s) : 0, 1220990
Total [candidate] (9.395 s) : 0, 9395041
gantt
title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~00495f3e1c, baseline=1.57.0-SNAPSHOT~1ba60ee63f
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.204 ms) : 0, 1204
crashtracking [candidate] (1.196 ms) : 0, 1196
BytebuddyAgent [baseline] (650.345 ms) : 0, 650345
BytebuddyAgent [candidate] (655.613 ms) : 0, 655613
GlobalTracer [baseline] (282.533 ms) : 0, 282533
GlobalTracer [candidate] (284.644 ms) : 0, 284644
AppSec [baseline] (32.466 ms) : 0, 32466
AppSec [candidate] (32.814 ms) : 0, 32814
Debugger [baseline] (67.724 ms) : 0, 67724
Debugger [candidate] (68.147 ms) : 0, 68147
Remote Config [baseline] (653.692 µs) : 0, 654
Remote Config [candidate] (660.321 µs) : 0, 660
Telemetry [baseline] (9.001 ms) : 0, 9001
Telemetry [candidate] (9.075 ms) : 0, 9075
Flare Poller [baseline] (3.761 ms) : 0, 3761
Flare Poller [candidate] (3.762 ms) : 0, 3762
section iast
crashtracking [baseline] (1.199 ms) : 0, 1199
crashtracking [candidate] (1.191 ms) : 0, 1191
BytebuddyAgent [baseline] (791.891 ms) : 0, 791891
BytebuddyAgent [candidate] (789.964 ms) : 0, 789964
GlobalTracer [baseline] (256.315 ms) : 0, 256315
GlobalTracer [candidate] (255.673 ms) : 0, 255673
AppSec [baseline] (33.545 ms) : 0, 33545
AppSec [candidate] (34.602 ms) : 0, 34602
Debugger [baseline] (66.049 ms) : 0, 66049
Debugger [candidate] (64.73 ms) : 0, 64730
Remote Config [baseline] (568.6 µs) : 0, 569
Remote Config [candidate] (546.002 µs) : 0, 546
Telemetry [baseline] (8.444 ms) : 0, 8444
Telemetry [candidate] (8.403 ms) : 0, 8403
Flare Poller [baseline] (3.536 ms) : 0, 3536
Flare Poller [candidate] (3.482 ms) : 0, 3482
IAST [baseline] (26.914 ms) : 0, 26914
IAST [candidate] (26.949 ms) : 0, 26949
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~00495f3e1c, baseline=1.57.0-SNAPSHOT~1ba60ee63f
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.097 s) : 0, 1097434
Total [baseline] (11.0 s) : 0, 11000442
Agent [candidate] (1.09 s) : 0, 1090012
Total [candidate] (10.915 s) : 0, 10914686
section appsec
Agent [baseline] (1.27 s) : 0, 1270267
Total [baseline] (11.138 s) : 0, 11138176
Agent [candidate] (1.264 s) : 0, 1264364
Total [candidate] (10.989 s) : 0, 10988726
section iast
Agent [baseline] (1.233 s) : 0, 1233089
Total [baseline] (11.31 s) : 0, 11309720
Agent [candidate] (1.223 s) : 0, 1222602
Total [candidate] (11.307 s) : 0, 11307441
section profiling
Agent [baseline] (1.206 s) : 0, 1206383
Total [baseline] (10.933 s) : 0, 10933402
Agent [candidate] (1.21 s) : 0, 1210133
Total [candidate] (11.046 s) : 0, 11045661
gantt
title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~00495f3e1c, baseline=1.57.0-SNAPSHOT~1ba60ee63f
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.198 ms) : 0, 1198
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (659.043 ms) : 0, 659043
BytebuddyAgent [candidate] (654.874 ms) : 0, 654874
GlobalTracer [baseline] (285.849 ms) : 0, 285849
GlobalTracer [candidate] (283.62 ms) : 0, 283620
AppSec [baseline] (33.074 ms) : 0, 33074
AppSec [candidate] (32.715 ms) : 0, 32715
Debugger [baseline] (68.819 ms) : 0, 68819
Debugger [candidate] (68.38 ms) : 0, 68380
Remote Config [baseline] (656.264 µs) : 0, 656
Remote Config [candidate] (639.713 µs) : 0, 640
Telemetry [baseline] (9.25 ms) : 0, 9250
Telemetry [candidate] (9.018 ms) : 0, 9018
Flare Poller [baseline] (3.815 ms) : 0, 3815
Flare Poller [candidate] (3.716 ms) : 0, 3716
section appsec
crashtracking [baseline] (1.199 ms) : 0, 1199
crashtracking [candidate] (1.189 ms) : 0, 1189
BytebuddyAgent [baseline] (691.419 ms) : 0, 691419
BytebuddyAgent [candidate] (688.742 ms) : 0, 688742
GlobalTracer [baseline] (260.166 ms) : 0, 260166
GlobalTracer [candidate] (258.37 ms) : 0, 258370
AppSec [baseline] (175.875 ms) : 0, 175875
AppSec [candidate] (174.83 ms) : 0, 174830
Debugger [baseline] (67.468 ms) : 0, 67468
Debugger [candidate] (67.387 ms) : 0, 67387
Remote Config [baseline] (749.338 µs) : 0, 749
Remote Config [candidate] (670.176 µs) : 0, 670
Telemetry [baseline] (9.197 ms) : 0, 9197
Telemetry [candidate] (9.2 ms) : 0, 9200
Flare Poller [baseline] (3.96 ms) : 0, 3960
Flare Poller [candidate] (3.943 ms) : 0, 3943
IAST [baseline] (24.523 ms) : 0, 24523
IAST [candidate] (24.515 ms) : 0, 24515
section iast
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.189 ms) : 0, 1189
BytebuddyAgent [baseline] (797.763 ms) : 0, 797763
BytebuddyAgent [candidate] (791.119 ms) : 0, 791119
GlobalTracer [baseline] (257.322 ms) : 0, 257322
GlobalTracer [candidate] (256.087 ms) : 0, 256087
AppSec [baseline] (35.477 ms) : 0, 35477
AppSec [candidate] (35.003 ms) : 0, 35003
Debugger [baseline] (65.973 ms) : 0, 65973
Debugger [candidate] (64.473 ms) : 0, 64473
Remote Config [baseline] (561.335 µs) : 0, 561
Remote Config [candidate] (533.991 µs) : 0, 534
Telemetry [baseline] (8.513 ms) : 0, 8513
Telemetry [candidate] (8.357 ms) : 0, 8357
Flare Poller [baseline] (3.551 ms) : 0, 3551
Flare Poller [candidate] (3.462 ms) : 0, 3462
IAST [baseline] (27.061 ms) : 0, 27061
IAST [candidate] (26.825 ms) : 0, 26825
section profiling
crashtracking [baseline] (1.196 ms) : 0, 1196
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (702.445 ms) : 0, 702445
BytebuddyAgent [candidate] (704.771 ms) : 0, 704771
GlobalTracer [baseline] (221.13 ms) : 0, 221130
GlobalTracer [candidate] (221.818 ms) : 0, 221818
AppSec [baseline] (32.17 ms) : 0, 32170
AppSec [candidate] (32.217 ms) : 0, 32217
Debugger [baseline] (69.161 ms) : 0, 69161
Debugger [candidate] (68.897 ms) : 0, 68897
Remote Config [baseline] (640.661 µs) : 0, 641
Remote Config [candidate] (655.886 µs) : 0, 656
Telemetry [baseline] (8.931 ms) : 0, 8931
Telemetry [candidate] (9.023 ms) : 0, 9023
Flare Poller [baseline] (3.716 ms) : 0, 3716
Flare Poller [candidate] (3.781 ms) : 0, 3781
ProfilingAgent [baseline] (97.001 ms) : 0, 97001
ProfilingAgent [candidate] (97.833 ms) : 0, 97833
Profiling [baseline] (97.591 ms) : 0, 97591
Profiling [candidate] (98.405 ms) : 0, 98405
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 2 performance regressions! Performance is the same for 19 metrics, 15 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~00495f3e1c, baseline=1.57.0-SNAPSHOT~1ba60ee63f
dateFormat X
axisFormat %s
section baseline
no_agent (17.3 ms) : 17129, 17471
. : milestone, 17300,
appsec (18.575 ms) : 18388, 18762
. : milestone, 18575,
code_origins (17.941 ms) : 17761, 18121
. : milestone, 17941,
iast (17.931 ms) : 17755, 18107
. : milestone, 17931,
profiling (20.149 ms) : 19936, 20362
. : milestone, 20149,
tracing (17.895 ms) : 17717, 18074
. : milestone, 17895,
section candidate
no_agent (19.289 ms) : 19094, 19484
. : milestone, 19289,
appsec (18.841 ms) : 18652, 19029
. : milestone, 18841,
code_origins (17.869 ms) : 17693, 18045
. : milestone, 17869,
iast (17.803 ms) : 17626, 17980
. : milestone, 17803,
profiling (19.884 ms) : 19680, 20087
. : milestone, 19884,
tracing (17.819 ms) : 17641, 17998
. : milestone, 17819,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~00495f3e1c, baseline=1.57.0-SNAPSHOT~1ba60ee63f
dateFormat X
axisFormat %s
section baseline
no_agent (1.178 ms) : 1166, 1189
. : milestone, 1178,
iast (3.206 ms) : 3162, 3251
. : milestone, 3206,
iast_FULL (5.844 ms) : 5785, 5902
. : milestone, 5844,
iast_GLOBAL (3.65 ms) : 3598, 3702
. : milestone, 3650,
profiling (2.099 ms) : 2079, 2119
. : milestone, 2099,
tracing (1.808 ms) : 1792, 1823
. : milestone, 1808,
section candidate
no_agent (1.188 ms) : 1177, 1200
. : milestone, 1188,
iast (3.172 ms) : 3132, 3211
. : milestone, 3172,
iast_FULL (5.722 ms) : 5666, 5779
. : milestone, 5722,
iast_GLOBAL (3.593 ms) : 3540, 3646
. : milestone, 3593,
profiling (2.172 ms) : 2152, 2193
. : milestone, 2172,
tracing (1.827 ms) : 1811, 1842
. : milestone, 1827,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~00495f3e1c, baseline=1.57.0-SNAPSHOT~1ba60ee63f
dateFormat X
axisFormat %s
section baseline
no_agent (1.483 ms) : 1472, 1495
. : milestone, 1483,
appsec (3.694 ms) : 3476, 3912
. : milestone, 3694,
iast (2.206 ms) : 2142, 2270
. : milestone, 2206,
iast_GLOBAL (2.248 ms) : 2184, 2312
. : milestone, 2248,
profiling (2.093 ms) : 2039, 2147
. : milestone, 2093,
tracing (2.042 ms) : 1992, 2093
. : milestone, 2042,
section candidate
no_agent (1.478 ms) : 1466, 1489
. : milestone, 1478,
appsec (3.654 ms) : 3440, 3869
. : milestone, 3654,
iast (2.215 ms) : 2151, 2280
. : milestone, 2215,
iast_GLOBAL (2.252 ms) : 2188, 2317
. : milestone, 2252,
profiling (2.053 ms) : 2001, 2105
. : milestone, 2053,
tracing (2.045 ms) : 1994, 2096
. : milestone, 2045,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~00495f3e1c, baseline=1.57.0-SNAPSHOT~1ba60ee63f
dateFormat X
axisFormat %s
section baseline
no_agent (14.953 s) : 14953000, 14953000
. : milestone, 14953000,
appsec (14.992 s) : 14992000, 14992000
. : milestone, 14992000,
iast (17.833 s) : 17833000, 17833000
. : milestone, 17833000,
iast_GLOBAL (17.829 s) : 17829000, 17829000
. : milestone, 17829000,
profiling (15.102 s) : 15102000, 15102000
. : milestone, 15102000,
tracing (14.685 s) : 14685000, 14685000
. : milestone, 14685000,
section candidate
no_agent (14.987 s) : 14987000, 14987000
. : milestone, 14987000,
appsec (14.5 s) : 14500000, 14500000
. : milestone, 14500000,
iast (17.901 s) : 17901000, 17901000
. : milestone, 17901000,
iast_GLOBAL (17.834 s) : 17834000, 17834000
. : milestone, 17834000,
profiling (14.627 s) : 14627000, 14627000
. : milestone, 14627000,
tracing (15.077 s) : 15077000, 15077000
. : milestone, 15077000,
|
| DD_INJECTION_ENABLED: "tracer" | ||
| parallel: | ||
| matrix: *test_matrix_6 | ||
| matrix: *test_matrix_8 |
There was a problem hiding this comment.
question: Does test_ssi_smoke requires eight slot ?
|
About the memory failures on Gradle Daemon disappearing, do you know if we can track in dashboards the type of cgroup we have in the runners. I noticed that Gradle added support for cgroup v1 in 8.x, but cgroup v2 is supposed to land in Gradle 9.3. This could potentially be a cause for the memory exhaustion in the GL runnners maybe because to many worker processes can be started. |
What Does This Do
CI jobs have recently begun failing more frequently with the following error:
DaemonDisappearedException: Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)After inspecting several affected runs, I noticed that the
cgroupmemory peak consistently exceeded the configured memory limit:Additionally, although the
BackingStoreException: Couldn't get file lockissue appears less frequently now, it still occurs occasionally. To address both problems, this PR makes two adjustments to the build environment:DaemonDisappearedException)BackingStoreException)Motivation
Improve CI stability and reduce flaky failures.
Additional Notes
This configuration has been tested on GitLab and passed 6 consecutive CI runs without issues.