Skip to content

Reenable kafka 3.8 by default #8007

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
mcculls merged 2 commits into from
Nov 25, 2024
Merged

Reenable kafka 3.8 by default #8007

mcculls merged 2 commits into from
Nov 25, 2024

Conversation

@nayeem-kamal
Copy link
Contributor

@nayeem-kamal nayeem-kamal commented Nov 25, 2024 •
edited by atlassian bot
Loading

What Does This Do

Re-enables kafka 3.8 because the double instrumentation issue has been resolved in #8006

Motivation

Additional Notes

Contributor Checklist

Jira ticket: AIDM-477

@pr-commenter
Copy link

pr-commenter bot commented Nov 25, 2024 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nayeem-kamal/enable-kafka-3
git_commit_date 1732526948 1732529492
git_commit_sha b36006c 39229df
See matching parameters
Baseline Candidate
ci_job_date 1732530881 1732530881
ci_job_id 718553101 718553101
ci_pipeline_id 49836332 49836332
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce unsure
[-74856.414op/s; -14018.330op/s] or [-4.074%; -0.763%]
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@nayeem-kamal nayeem-kamal marked this pull request as ready for review November 25, 2024 08:10
@nayeem-kamal nayeem-kamal requested review from a team as code owners November 25, 2024 08:10
@nayeem-kamal nayeem-kamal requested a review from dougqh November 25, 2024 08:10
@pr-commenter
Copy link

pr-commenter bot commented Nov 25, 2024 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nayeem-kamal/enable-kafka-3
git_commit_date 1732526948 1732529492
git_commit_sha b36006c 39229df
See matching parameters
Baseline Candidate
ci_job_date 1732530616 1732530616
ci_job_id 718529103 718529103
ci_pipeline_id 49836332 49836332
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

Base automatically changed from mcculls/kafka-client-muzzle-fix to master November 25, 2024 09:29
@mcculls mcculls force-pushed the nayeem-kamal/enable-kafka-3 branch from c21dcfd to 8c09a3d Compare November 25, 2024 09:30
amarziali
amarziali reviewed Nov 25, 2024
View reviewed changes
...n/java/datadog/trace/instrumentation/kafka_clients38/ConsumerCoordinatorInstrumentation.java Outdated
implements Instrumenter.ForSingleType {

public ConsumerCoordinatorInstrumentation() {
super("kafka");
Copy link
Contributor

@amarziali amarziali Nov 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may we also have instrumentation names by version? i.e. ["kafka", "kafka-0.11"] for the old and ["kafka", "kafka-3.8"] for the new? it will give the possibility to selectivelt disable one of them if other issues will arise

Copy link
Contributor

@mcculls mcculls Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

39229df

I didn't add an alias to KafkaDeserializerInstrumentation because it is an IAST instrumentation that works across all known Kafka versions

@pr-commenter
Copy link

pr-commenter bot commented Nov 25, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nayeem-kamal/enable-kafka-3
git_commit_sha b055376 39229df
release_version 1.43.0-SNAPSHOT~b0553764bc 1.43.0-SNAPSHOT~39229df6d9
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1732531837 1732531837
ci_job_id 718529098 718529098
ci_pipeline_id 49836332 49836332
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1732529492 1732529492
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 8 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.09 s) : 0, 1089870
Total [baseline] (10.459 s) : 0, 10458959
Agent [candidate] (1.092 s) : 0, 1092019
Total [candidate] (10.471 s) : 0, 10471268
section appsec
Agent [baseline] (1.223 s) : 0, 1222823
Total [baseline] (10.734 s) : 0, 10733672
Agent [candidate] (1.228 s) : 0, 1228206
Total [candidate] (10.739 s) : 0, 10738829
section iast
Agent [baseline] (1.216 s) : 0, 1216003
Total [baseline] (10.912 s) : 0, 10912136
Agent [candidate] (1.216 s) : 0, 1215910
Total [candidate] (10.911 s) : 0, 10910890
section profiling
Agent [baseline] (1.325 s) : 0, 1324802
Total [baseline] (10.876 s) : 0, 10876431
Agent [candidate] (1.317 s) : 0, 1317066
Total [candidate] (10.849 s) : 0, 10848939
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.09 s -
Agent appsec 1.223 s 132.953 ms (12.2%)
Agent iast 1.216 s 126.134 ms (11.6%)
Agent profiling 1.325 s 234.932 ms (21.6%)
Total tracing 10.459 s -
Total appsec 10.734 s 274.713 ms (2.6%)
Total iast 10.912 s 453.178 ms (4.3%)
Total profiling 10.876 s 417.472 ms (4.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.092 s -
Agent appsec 1.228 s 136.187 ms (12.5%)
Agent iast 1.216 s 123.891 ms (11.3%)
Agent profiling 1.317 s 225.047 ms (20.6%)
Total tracing 10.471 s -
Total appsec 10.739 s 267.561 ms (2.6%)
Total iast 10.911 s 439.622 ms (4.2%)
Total profiling 10.849 s 377.671 ms (3.6%) 
gantt
    title petclinic - break down per module: candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (693.047 ms) : 0, 693047
BytebuddyAgent [candidate] (694.563 ms) : 0, 694563
GlobalTracer [baseline] (317.617 ms) : 0, 317617
GlobalTracer [candidate] (319.107 ms) : 0, 319107
AppSec [baseline] (54.886 ms) : 0, 54886
AppSec [candidate] (54.646 ms) : 0, 54646
Remote Config [baseline] (693.039 µs) : 0, 693
Remote Config [candidate] (684.682 µs) : 0, 685
Telemetry [baseline] (9.931 ms) : 0, 9931
Telemetry [candidate] (9.299 ms) : 0, 9299
section appsec
BytebuddyAgent [baseline] (709.015 ms) : 0, 709015
BytebuddyAgent [candidate] (713.287 ms) : 0, 713287
GlobalTracer [baseline] (314.157 ms) : 0, 314157
GlobalTracer [candidate] (315.745 ms) : 0, 315745
AppSec [baseline] (168.048 ms) : 0, 168048
AppSec [candidate] (167.359 ms) : 0, 167359
Remote Config [baseline] (650.333 µs) : 0, 650
Remote Config [candidate] (653.91 µs) : 0, 654
Telemetry [baseline] (7.779 ms) : 0, 7779
Telemetry [candidate] (7.88 ms) : 0, 7880
IAST [baseline] (19.039 ms) : 0, 19039
IAST [candidate] (19.063 ms) : 0, 19063
section iast
BytebuddyAgent [baseline] (809.797 ms) : 0, 809797
BytebuddyAgent [candidate] (809.131 ms) : 0, 809131
GlobalTracer [baseline] (306.113 ms) : 0, 306113
GlobalTracer [candidate] (305.834 ms) : 0, 305834
AppSec [baseline] (57.799 ms) : 0, 57799
AppSec [candidate] (58.169 ms) : 0, 58169
Remote Config [baseline] (614.771 µs) : 0, 615
Remote Config [candidate] (613.646 µs) : 0, 614
Telemetry [baseline] (7.403 ms) : 0, 7403
Telemetry [candidate] (7.546 ms) : 0, 7546
IAST [baseline] (20.602 ms) : 0, 20602
IAST [candidate] (20.881 ms) : 0, 20881
section profiling
BytebuddyAgent [baseline] (691.462 ms) : 0, 691462
BytebuddyAgent [candidate] (688.515 ms) : 0, 688515
GlobalTracer [baseline] (438.043 ms) : 0, 438043
GlobalTracer [candidate] (433.688 ms) : 0, 433688
AppSec [baseline] (54.026 ms) : 0, 54026
AppSec [candidate] (53.838 ms) : 0, 53838
Remote Config [baseline] (662.132 µs) : 0, 662
Remote Config [candidate] (668.869 µs) : 0, 669
Telemetry [baseline] (7.785 ms) : 0, 7785
Telemetry [candidate] (7.715 ms) : 0, 7715
ProfilingAgent [baseline] (93.578 ms) : 0, 93578
ProfilingAgent [candidate] (93.581 ms) : 0, 93581
Profiling [baseline] (93.602 ms) : 0, 93602
Profiling [candidate] (93.604 ms) : 0, 93604
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.098 s) : 0, 1097969
Total [baseline] (8.682 s) : 0, 8682243
Agent [candidate] (1.092 s) : 0, 1091686
Total [candidate] (8.7 s) : 0, 8699962
section iast
Agent [baseline] (1.218 s) : 0, 1217833
Total [baseline] (9.17 s) : 0, 9170420
Agent [candidate] (1.219 s) : 0, 1219249
Total [candidate] (9.175 s) : 0, 9175256
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.221 s) : 0, 1220879
Total [baseline] (9.204 s) : 0, 9203858
Agent [candidate] (1.223 s) : 0, 1222914
Total [candidate] (9.182 s) : 0, 9181890
section iast_TELEMETRY_OFF
Agent [baseline] (1.22 s) : 0, 1219812
Total [baseline] (9.216 s) : 0, 9215553
Agent [candidate] (1.216 s) : 0, 1215760
Total [candidate] (9.18 s) : 0, 9180221
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.098 s -
Agent iast 1.218 s 119.864 ms (10.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.221 s 122.909 ms (11.2%)
Agent iast_TELEMETRY_OFF 1.22 s 121.842 ms (11.1%)
Total tracing 8.682 s -
Total iast 9.17 s 488.178 ms (5.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.204 s 521.616 ms (6.0%)
Total iast_TELEMETRY_OFF 9.216 s 533.311 ms (6.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.092 s -
Agent iast 1.219 s 127.564 ms (11.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.223 s 131.229 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.216 s 124.074 ms (11.4%)
Total tracing 8.7 s -
Total iast 9.175 s 475.294 ms (5.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.182 s 481.928 ms (5.5%)
Total iast_TELEMETRY_OFF 9.18 s 480.26 ms (5.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (698.519 ms) : 0, 698519
BytebuddyAgent [candidate] (694.579 ms) : 0, 694579
GlobalTracer [baseline] (319.38 ms) : 0, 319380
GlobalTracer [candidate] (319.377 ms) : 0, 319377
AppSec [baseline] (54.852 ms) : 0, 54852
AppSec [candidate] (54.743 ms) : 0, 54743
Remote Config [baseline] (685.244 µs) : 0, 685
Remote Config [candidate] (680.692 µs) : 0, 681
Telemetry [baseline] (10.71 ms) : 0, 10710
Telemetry [candidate] (8.541 ms) : 0, 8541
section iast
BytebuddyAgent [baseline] (811.013 ms) : 0, 811013
BytebuddyAgent [candidate] (811.876 ms) : 0, 811876
GlobalTracer [baseline] (306.241 ms) : 0, 306241
GlobalTracer [candidate] (306.266 ms) : 0, 306266
AppSec [baseline] (57.187 ms) : 0, 57187
AppSec [candidate] (58.172 ms) : 0, 58172
Remote Config [baseline] (650.899 µs) : 0, 651
Remote Config [candidate] (627.512 µs) : 0, 628
Telemetry [baseline] (7.479 ms) : 0, 7479
Telemetry [candidate] (7.563 ms) : 0, 7563
IAST [baseline] (21.568 ms) : 0, 21568
IAST [candidate] (20.911 ms) : 0, 20911
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (812.539 ms) : 0, 812539
BytebuddyAgent [candidate] (815.297 ms) : 0, 815297
GlobalTracer [baseline] (307.408 ms) : 0, 307408
GlobalTracer [candidate] (306.758 ms) : 0, 306758
AppSec [baseline] (56.555 ms) : 0, 56555
AppSec [candidate] (56.466 ms) : 0, 56466
Remote Config [baseline] (632.38 µs) : 0, 632
Remote Config [candidate] (613.393 µs) : 0, 613
Telemetry [baseline] (7.492 ms) : 0, 7492
Telemetry [candidate] (7.466 ms) : 0, 7466
IAST [baseline] (22.542 ms) : 0, 22542
IAST [candidate] (22.489 ms) : 0, 22489
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (812.08 ms) : 0, 812080
BytebuddyAgent [candidate] (808.199 ms) : 0, 808199
GlobalTracer [baseline] (307.197 ms) : 0, 307197
GlobalTracer [candidate] (306.48 ms) : 0, 306480
AppSec [baseline] (57.493 ms) : 0, 57493
AppSec [candidate] (57.938 ms) : 0, 57938
Remote Config [baseline] (645.532 µs) : 0, 646
Remote Config [candidate] (628.511 µs) : 0, 629
Telemetry [baseline] (7.402 ms) : 0, 7402
Telemetry [candidate] (8.253 ms) : 0, 8253
IAST [baseline] (21.225 ms) : 0, 21225
IAST [candidate] (20.554 ms) : 0, 20554
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-11-25T10:20:45 2024-11-25T10:27:45
git_branch master nayeem-kamal/enable-kafka-3
git_commit_sha b055376 39229df
release_version 1.43.0-SNAPSHOT~b0553764bc 1.43.0-SNAPSHOT~39229df6d9
start_time 2024-11-25T10:20:31 2024-11-25T10:27:32
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1732530818 1732530818
ci_job_id 718529099 718529099
ci_pipeline_id 49836332 49836332
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1732529492 1732529492
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 15 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling better
[-90.730µs; -39.410µs] or [-5.833%; -2.534%]		 unstable
[-460.534op/s; +688.454op/s] or [-15.543%; +23.235%]		 1.490ms 3076.923op/s 1.555ms 2962.963op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.353 ms) : 1333, 1374
.   : milestone, 1353,
appsec (1.772 ms) : 1748, 1797
.   : milestone, 1772,
appsec_no_iast (1.777 ms) : 1753, 1801
.   : milestone, 1777,
iast (1.514 ms) : 1491, 1537
.   : milestone, 1514,
profiling (1.555 ms) : 1531, 1579
.   : milestone, 1555,
tracing (1.482 ms) : 1457, 1506
.   : milestone, 1482,
section candidate
no_agent (1.359 ms) : 1339, 1379
.   : milestone, 1359,
appsec (1.756 ms) : 1731, 1781
.   : milestone, 1756,
appsec_no_iast (1.771 ms) : 1746, 1796
.   : milestone, 1771,
iast (1.497 ms) : 1474, 1520
.   : milestone, 1497,
profiling (1.49 ms) : 1467, 1514
.   : milestone, 1490,
tracing (1.481 ms) : 1456, 1507
.   : milestone, 1481,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.353 ms [1.333 ms, 1.374 ms] -
appsec 1.772 ms [1.748 ms, 1.797 ms] 418.946 µs (31.0%)
appsec_no_iast 1.777 ms [1.753 ms, 1.801 ms] 423.632 µs (31.3%)
iast 1.514 ms [1.491 ms, 1.537 ms] 160.648 µs (11.9%)
profiling 1.555 ms [1.531 ms, 1.579 ms] 202.073 µs (14.9%)
tracing 1.482 ms [1.457 ms, 1.506 ms] 128.31 µs (9.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.359 ms [1.339 ms, 1.379 ms] -
appsec 1.756 ms [1.731 ms, 1.781 ms] 397.305 µs (29.2%)
appsec_no_iast 1.771 ms [1.746 ms, 1.796 ms] 411.727 µs (30.3%)
iast 1.497 ms [1.474 ms, 1.52 ms] 137.947 µs (10.2%)
profiling 1.49 ms [1.467 ms, 1.514 ms] 131.485 µs (9.7%)
tracing 1.481 ms [1.456 ms, 1.507 ms] 122.591 µs (9.0%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc
    dateFormat X
    axisFormat %s
section baseline
no_agent (378.019 µs) : 358, 398
.   : milestone, 378,
iast (491.513 µs) : 470, 513
.   : milestone, 492,
iast_FULL (655.907 µs) : 634, 678
.   : milestone, 656,
iast_GLOBAL (525.269 µs) : 504, 547
.   : milestone, 525,
iast_HARDCODED_SECRET_DISABLED (493.166 µs) : 472, 514
.   : milestone, 493,
iast_INACTIVE (450.204 µs) : 429, 471
.   : milestone, 450,
iast_TELEMETRY_OFF (486.252 µs) : 464, 508
.   : milestone, 486,
tracing (448.597 µs) : 428, 469
.   : milestone, 449,
section candidate
no_agent (379.839 µs) : 359, 400
.   : milestone, 380,
iast (496.872 µs) : 476, 518
.   : milestone, 497,
iast_FULL (659.88 µs) : 638, 681
.   : milestone, 660,
iast_GLOBAL (530.225 µs) : 508, 552
.   : milestone, 530,
iast_HARDCODED_SECRET_DISABLED (490.802 µs) : 469, 513
.   : milestone, 491,
iast_INACTIVE (448.027 µs) : 427, 469
.   : milestone, 448,
iast_TELEMETRY_OFF (478.56 µs) : 457, 500
.   : milestone, 479,
tracing (454.741 µs) : 434, 476
.   : milestone, 455,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 378.019 µs [358.072 µs, 397.965 µs] -
iast 491.513 µs [470.017 µs, 513.009 µs] 113.494 µs (30.0%)
iast_FULL 655.907 µs [633.888 µs, 677.925 µs] 277.888 µs (73.5%)
iast_GLOBAL 525.269 µs [503.939 µs, 546.598 µs] 147.25 µs (39.0%)
iast_HARDCODED_SECRET_DISABLED 493.166 µs [471.947 µs, 514.386 µs] 115.148 µs (30.5%)
iast_INACTIVE 450.204 µs [429.49 µs, 470.917 µs] 72.185 µs (19.1%)
iast_TELEMETRY_OFF 486.252 µs [464.19 µs, 508.314 µs] 108.233 µs (28.6%)
tracing 448.597 µs [428.04 µs, 469.154 µs] 70.578 µs (18.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 379.839 µs [359.485 µs, 400.193 µs] -
iast 496.872 µs [475.577 µs, 518.167 µs] 117.033 µs (30.8%)
iast_FULL 659.88 µs [638.392 µs, 681.368 µs] 280.041 µs (73.7%)
iast_GLOBAL 530.225 µs [508.053 µs, 552.397 µs] 150.386 µs (39.6%)
iast_HARDCODED_SECRET_DISABLED 490.802 µs [468.824 µs, 512.78 µs] 110.964 µs (29.2%)
iast_INACTIVE 448.027 µs [427.417 µs, 468.637 µs] 68.189 µs (18.0%)
iast_TELEMETRY_OFF 478.56 µs [457.455 µs, 499.665 µs] 98.721 µs (26.0%)
tracing 454.741 µs [433.844 µs, 475.638 µs] 74.902 µs (19.7%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nayeem-kamal/enable-kafka-3
git_commit_sha b055376 39229df
release_version 1.43.0-SNAPSHOT~b0553764bc 1.43.0-SNAPSHOT~39229df6d9
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1732531476 1732531476
ci_job_id 718529100 718529100
ci_pipeline_id 49836332 49836332
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1732529492 1732529492
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.621 s) : 15621000, 15621000
.   : milestone, 15621000,
appsec (15.024 s) : 15024000, 15024000
.   : milestone, 15024000,
iast (19.658 s) : 19658000, 19658000
.   : milestone, 19658000,
iast_GLOBAL (17.798 s) : 17798000, 17798000
.   : milestone, 17798000,
profiling (15.103 s) : 15103000, 15103000
.   : milestone, 15103000,
tracing (15.131 s) : 15131000, 15131000
.   : milestone, 15131000,
section candidate
no_agent (15.269 s) : 15269000, 15269000
.   : milestone, 15269000,
appsec (15.018 s) : 15018000, 15018000
.   : milestone, 15018000,
iast (18.55 s) : 18550000, 18550000
.   : milestone, 18550000,
iast_GLOBAL (17.917 s) : 17917000, 17917000
.   : milestone, 17917000,
profiling (14.953 s) : 14953000, 14953000
.   : milestone, 14953000,
tracing (14.938 s) : 14938000, 14938000
.   : milestone, 14938000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.621 s [15.621 s, 15.621 s] -
appsec 15.024 s [15.024 s, 15.024 s] -597.0 ms (-3.8%)
iast 19.658 s [19.658 s, 19.658 s] 4.037 s (25.8%)
iast_GLOBAL 17.798 s [17.798 s, 17.798 s] 2.177 s (13.9%)
profiling 15.103 s [15.103 s, 15.103 s] -518.0 ms (-3.3%)
tracing 15.131 s [15.131 s, 15.131 s] -490.0 ms (-3.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.269 s [15.269 s, 15.269 s] -
appsec 15.018 s [15.018 s, 15.018 s] -251.0 ms (-1.6%)
iast 18.55 s [18.55 s, 18.55 s] 3.281 s (21.5%)
iast_GLOBAL 17.917 s [17.917 s, 17.917 s] 2.648 s (17.3%)
profiling 14.953 s [14.953 s, 14.953 s] -316.0 ms (-2.1%)
tracing 14.938 s [14.938 s, 14.938 s] -331.0 ms (-2.2%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~39229df6d9, baseline=1.43.0-SNAPSHOT~b0553764bc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.463 ms) : 1452, 1475
.   : milestone, 1463,
appsec (2.311 ms) : 2270, 2352
.   : milestone, 2311,
iast (2.078 ms) : 2025, 2130
.   : milestone, 2078,
iast_GLOBAL (2.101 ms) : 2048, 2153
.   : milestone, 2101,
profiling (1.959 ms) : 1916, 2002
.   : milestone, 1959,
tracing (1.921 ms) : 1881, 1962
.   : milestone, 1921,
section candidate
no_agent (1.461 ms) : 1449, 1472
.   : milestone, 1461,
appsec (2.318 ms) : 2277, 2359
.   : milestone, 2318,
iast (2.068 ms) : 2015, 2121
.   : milestone, 2068,
iast_GLOBAL (2.118 ms) : 2065, 2171
.   : milestone, 2118,
profiling (1.947 ms) : 1904, 1990
.   : milestone, 1947,
tracing (1.914 ms) : 1874, 1953
.   : milestone, 1914,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.463 ms [1.452 ms, 1.475 ms] -
appsec 2.311 ms [2.27 ms, 2.352 ms] 847.493 µs (57.9%)
iast 2.078 ms [2.025 ms, 2.13 ms] 614.064 µs (42.0%)
iast_GLOBAL 2.101 ms [2.048 ms, 2.153 ms] 637.171 µs (43.5%)
profiling 1.959 ms [1.916 ms, 2.002 ms] 495.649 µs (33.9%)
tracing 1.921 ms [1.881 ms, 1.962 ms] 457.651 µs (31.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.461 ms [1.449 ms, 1.472 ms] -
appsec 2.318 ms [2.277 ms, 2.359 ms] 857.298 µs (58.7%)
iast 2.068 ms [2.015 ms, 2.121 ms] 607.306 µs (41.6%)
iast_GLOBAL 2.118 ms [2.065 ms, 2.171 ms] 657.669 µs (45.0%)
profiling 1.947 ms [1.904 ms, 1.99 ms] 486.374 µs (33.3%)
tracing 1.914 ms [1.874 ms, 1.953 ms] 452.764 µs (31.0%)

@mcculls mcculls merged commit f2d21ae into master Nov 25, 2024
6 checks passed
@mcculls mcculls deleted the nayeem-kamal/enable-kafka-3 branch November 25, 2024 11:04
@github-actions github-actions bot added this to the 1.43.0 milestone Nov 25, 2024
@mcculls mcculls added type: bug Bug report and fix inst: kafka Kafka instrumentation labels Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcculls mcculls mcculls left review comments

@amarziali amarziali amarziali approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

inst: kafka Kafka instrumentation type: bug Bug report and fix

Projects

None yet

Milestone

1.43.0

Development

Successfully merging this pull request may close these issues.

4 participants

@nayeem-kamal @mcculls @amarziali