Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Decouple gradle JDK from compile/test JDK #7506

Draft
wants to merge 21 commits into
base: master
Choose a base branch
from
Draft

Decouple gradle JDK from compile/test JDK #7506

wants to merge 21 commits into from

Conversation

smola
Copy link
Member

@smola smola commented Aug 26, 2024 •
edited by jira bot
Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: APMJAVA-1342

@smola smola added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Aug 26, 2024
@smola smola force-pushed the smola/gradle-java11 branch from dc1fa6e to 702851a Compare August 30, 2024 20:49
@pr-commenter
Copy link

pr-commenter bot commented Aug 30, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1737654035 1737707604
git_commit_sha 56d4919 8573d24
release_version 1.46.0-SNAPSHOT~56d4919491 1.46.0-SNAPSHOT~8573d24524
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737710154 1737710154
ci_job_id 779035797 779035797
ci_pipeline_id 53890093 53890093
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-nvtssqy9-project-304-concurrent-0-358t5yec 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-nvtssqy9-project-304-concurrent-0-358t5yec 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 5 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~8573d24524, baseline=1.46.0-SNAPSHOT~56d4919491

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.045 s) : 0, 1045044
Total [baseline] (10.463 s) : 0, 10463116
Agent [candidate] (1.06 s) : 0, 1059520
Total [candidate] (10.51 s) : 0, 10510104
section appsec
Agent [baseline] (1.181 s) : 0, 1180601
Total [baseline] (10.82 s) : 0, 10819778
Agent [candidate] (1.181 s) : 0, 1181001
Total [candidate] (10.738 s) : 0, 10737761
section iast
Agent [baseline] (1.164 s) : 0, 1163968
Total [baseline] (10.95 s) : 0, 10949667
Agent [candidate] (1.166 s) : 0, 1166043
Total [candidate] (10.981 s) : 0, 10980824
section profiling
Agent [baseline] (1.254 s) : 0, 1254468
Total [baseline] (10.766 s) : 0, 10765976
Agent [candidate] (1.255 s) : 0, 1254737
Total [candidate] (10.84 s) : 0, 10839798
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.045 s -
Agent appsec 1.181 s 135.557 ms (13.0%)
Agent iast 1.164 s 118.924 ms (11.4%)
Agent profiling 1.254 s 209.424 ms (20.0%)
Total tracing 10.463 s -
Total appsec 10.82 s 356.662 ms (3.4%)
Total iast 10.95 s 486.551 ms (4.7%)
Total profiling 10.766 s 302.86 ms (2.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.06 s -
Agent appsec 1.181 s 121.481 ms (11.5%)
Agent iast 1.166 s 106.523 ms (10.1%)
Agent profiling 1.255 s 195.218 ms (18.4%)
Total tracing 10.51 s -
Total appsec 10.738 s 227.657 ms (2.2%)
Total iast 10.981 s 470.719 ms (4.5%)
Total profiling 10.84 s 329.694 ms (3.1%) 
gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~8573d24524, baseline=1.46.0-SNAPSHOT~56d4919491

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.219 ms) : 0, 720219
BytebuddyAgent [candidate] (729.91 ms) : 0, 729910
GlobalTracer [baseline] (240.529 ms) : 0, 240529
GlobalTracer [candidate] (244.154 ms) : 0, 244154
AppSec [baseline] (55.523 ms) : 0, 55523
AppSec [candidate] (56.208 ms) : 0, 56208
Remote Config [baseline] (711.508 µs) : 0, 712
Remote Config [candidate] (727.984 µs) : 0, 728
Telemetry [baseline] (12.836 ms) : 0, 12836
Telemetry [candidate] (13.069 ms) : 0, 13069
section appsec
BytebuddyAgent [baseline] (733.019 ms) : 0, 733019
BytebuddyAgent [candidate] (733.613 ms) : 0, 733613
GlobalTracer [baseline] (236.937 ms) : 0, 236937
GlobalTracer [candidate] (236.781 ms) : 0, 236781
AppSec [baseline] (176.278 ms) : 0, 176278
AppSec [candidate] (175.884 ms) : 0, 175884
Remote Config [baseline] (661.585 µs) : 0, 662
Remote Config [candidate] (666.735 µs) : 0, 667
Telemetry [baseline] (8.565 ms) : 0, 8565
Telemetry [candidate] (9.293 ms) : 0, 9293
IAST [baseline] (20.239 ms) : 0, 20239
IAST [candidate] (19.364 ms) : 0, 19364
section iast
BytebuddyAgent [baseline] (831.489 ms) : 0, 831489
BytebuddyAgent [candidate] (832.811 ms) : 0, 832811
GlobalTracer [baseline] (229.136 ms) : 0, 229136
GlobalTracer [candidate] (229.655 ms) : 0, 229655
AppSec [baseline] (57.708 ms) : 0, 57708
AppSec [candidate] (57.922 ms) : 0, 57922
Remote Config [baseline] (652.561 µs) : 0, 653
Remote Config [candidate] (650.005 µs) : 0, 650
Telemetry [baseline] (8.606 ms) : 0, 8606
Telemetry [candidate] (8.648 ms) : 0, 8648
IAST [baseline] (21.34 ms) : 0, 21340
IAST [candidate] (21.306 ms) : 0, 21306
section profiling
ProfilingAgent [baseline] (95.251 ms) : 0, 95251
ProfilingAgent [candidate] (95.372 ms) : 0, 95372
BytebuddyAgent [baseline] (704.331 ms) : 0, 704331
BytebuddyAgent [candidate] (704.343 ms) : 0, 704343
GlobalTracer [baseline] (347.823 ms) : 0, 347823
GlobalTracer [candidate] (349.149 ms) : 0, 349149
AppSec [baseline] (55.449 ms) : 0, 55449
AppSec [candidate] (54.205 ms) : 0, 54205
Remote Config [baseline] (706.904 µs) : 0, 707
Remote Config [candidate] (714.235 µs) : 0, 714
Telemetry [baseline] (8.839 ms) : 0, 8839
Telemetry [candidate] (8.814 ms) : 0, 8814
Profiling [baseline] (95.275 ms) : 0, 95275
Profiling [candidate] (95.396 ms) : 0, 95396
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~8573d24524, baseline=1.46.0-SNAPSHOT~56d4919491

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1043180
Total [baseline] (8.655 s) : 0, 8655283
Agent [candidate] (1.043 s) : 0, 1043482
Total [candidate] (8.632 s) : 0, 8631606
section iast
Agent [baseline] (1.165 s) : 0, 1164621
Total [baseline] (9.197 s) : 0, 9197078
Agent [candidate] (1.167 s) : 0, 1166592
Total [candidate] (9.188 s) : 0, 9188250
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.175 s) : 0, 1174863
Total [baseline] (9.228 s) : 0, 9227640
Agent [candidate] (1.168 s) : 0, 1167648
Total [candidate] (9.171 s) : 0, 9171223
section iast_TELEMETRY_OFF
Agent [baseline] (1.169 s) : 0, 1168520
Total [baseline] (9.226 s) : 0, 9225756
Agent [candidate] (1.163 s) : 0, 1163096
Total [candidate] (9.21 s) : 0, 9210277
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.043 s -
Agent iast 1.165 s 121.441 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.175 s 131.683 ms (12.6%)
Agent iast_TELEMETRY_OFF 1.169 s 125.34 ms (12.0%)
Total tracing 8.655 s -
Total iast 9.197 s 541.795 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.228 s 572.357 ms (6.6%)
Total iast_TELEMETRY_OFF 9.226 s 570.473 ms (6.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.043 s -
Agent iast 1.167 s 123.11 ms (11.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.168 s 124.166 ms (11.9%)
Agent iast_TELEMETRY_OFF 1.163 s 119.614 ms (11.5%)
Total tracing 8.632 s -
Total iast 9.188 s 556.644 ms (6.4%)
Total iast_HARDCODED_SECRET_DISABLED 9.171 s 539.617 ms (6.3%)
Total iast_TELEMETRY_OFF 9.21 s 578.671 ms (6.7%) 
gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~8573d24524, baseline=1.46.0-SNAPSHOT~56d4919491

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (718.821 ms) : 0, 718821
BytebuddyAgent [candidate] (719.597 ms) : 0, 719597
GlobalTracer [baseline] (240.666 ms) : 0, 240666
GlobalTracer [candidate] (239.444 ms) : 0, 239444
AppSec [baseline] (55.732 ms) : 0, 55732
AppSec [candidate] (55.171 ms) : 0, 55171
Remote Config [baseline] (714.081 µs) : 0, 714
Remote Config [candidate] (695.986 µs) : 0, 696
Telemetry [baseline] (12.209 ms) : 0, 12209
Telemetry [candidate] (13.389 ms) : 0, 13389
section iast
BytebuddyAgent [baseline] (832.474 ms) : 0, 832474
BytebuddyAgent [candidate] (832.978 ms) : 0, 832978
GlobalTracer [baseline] (229.136 ms) : 0, 229136
GlobalTracer [candidate] (229.653 ms) : 0, 229653
AppSec [baseline] (57.515 ms) : 0, 57515
AppSec [candidate] (58.164 ms) : 0, 58164
IAST [baseline] (21.271 ms) : 0, 21271
IAST [candidate] (21.264 ms) : 0, 21264
Remote Config [baseline] (647.598 µs) : 0, 648
Remote Config [candidate] (652.512 µs) : 0, 653
Telemetry [baseline] (8.6 ms) : 0, 8600
Telemetry [candidate] (8.792 ms) : 0, 8792
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (838.865 ms) : 0, 838865
BytebuddyAgent [candidate] (834.017 ms) : 0, 834017
GlobalTracer [baseline] (230.928 ms) : 0, 230928
GlobalTracer [candidate] (230.115 ms) : 0, 230115
AppSec [baseline] (58.939 ms) : 0, 58939
AppSec [candidate] (57.851 ms) : 0, 57851
IAST [baseline] (21.507 ms) : 0, 21507
IAST [candidate] (21.279 ms) : 0, 21279
Remote Config [baseline] (647.47 µs) : 0, 647
Remote Config [candidate] (644.286 µs) : 0, 644
Telemetry [baseline] (8.89 ms) : 0, 8890
Telemetry [candidate] (8.694 ms) : 0, 8694
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (835.876 ms) : 0, 835876
BytebuddyAgent [candidate] (830.668 ms) : 0, 830668
GlobalTracer [baseline] (229.557 ms) : 0, 229557
GlobalTracer [candidate] (229.366 ms) : 0, 229366
AppSec [baseline] (57.871 ms) : 0, 57871
AppSec [candidate] (58.085 ms) : 0, 58085
IAST [baseline] (20.87 ms) : 0, 20870
IAST [candidate] (20.712 ms) : 0, 20712
Remote Config [baseline] (643.845 µs) : 0, 644
Remote Config [candidate] (639.762 µs) : 0, 640
Telemetry [baseline] (8.596 ms) : 0, 8596
Telemetry [candidate] (8.532 ms) : 0, 8532
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-01-24T08:45:52 2025-01-24T08:52:55
git_branch master smola/gradle-java11
git_commit_date 1737654035 1737707604
git_commit_sha 56d4919 8573d24
release_version 1.46.0-SNAPSHOT~56d4919491 1.46.0-SNAPSHOT~8573d24524
start_time 2025-01-24T08:45:38 2025-01-24T08:52:41
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737709133 1737709133
ci_job_id 779035798 779035798
ci_pipeline_id 53890093 53890093
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-nvtssqy9-project-304-concurrent-1-xykk6umx 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-nvtssqy9-project-304-concurrent-1-xykk6umx 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~8573d24524, baseline=1.46.0-SNAPSHOT~56d4919491
    dateFormat X
    axisFormat %s
section baseline
no_agent (381.161 µs) : 362, 401
.   : milestone, 381,
iast (506.51 µs) : 483, 530
.   : milestone, 507,
iast_FULL (739.625 µs) : 717, 762
.   : milestone, 740,
iast_GLOBAL (559.228 µs) : 536, 583
.   : milestone, 559,
iast_HARDCODED_SECRET_DISABLED (510.766 µs) : 487, 535
.   : milestone, 511,
iast_INACTIVE (460.152 µs) : 439, 481
.   : milestone, 460,
iast_TELEMETRY_OFF (491.785 µs) : 469, 515
.   : milestone, 492,
tracing (459.17 µs) : 438, 480
.   : milestone, 459,
section candidate
no_agent (381.538 µs) : 361, 402
.   : milestone, 382,
iast (515.759 µs) : 492, 539
.   : milestone, 516,
iast_FULL (746.418 µs) : 724, 769
.   : milestone, 746,
iast_GLOBAL (551.396 µs) : 530, 573
.   : milestone, 551,
iast_HARDCODED_SECRET_DISABLED (509.131 µs) : 486, 532
.   : milestone, 509,
iast_INACTIVE (460.076 µs) : 439, 481
.   : milestone, 460,
iast_TELEMETRY_OFF (501.699 µs) : 478, 525
.   : milestone, 502,
tracing (453.571 µs) : 433, 474
.   : milestone, 454,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 381.161 µs [361.641 µs, 400.681 µs] -
iast 506.51 µs [483.129 µs, 529.89 µs] 125.349 µs (32.9%)
iast_FULL 739.625 µs [716.934 µs, 762.316 µs] 358.464 µs (94.0%)
iast_GLOBAL 559.228 µs [535.856 µs, 582.6 µs] 178.067 µs (46.7%)
iast_HARDCODED_SECRET_DISABLED 510.766 µs [486.896 µs, 534.637 µs] 129.605 µs (34.0%)
iast_INACTIVE 460.152 µs [438.856 µs, 481.448 µs] 78.991 µs (20.7%)
iast_TELEMETRY_OFF 491.785 µs [468.889 µs, 514.681 µs] 110.624 µs (29.0%)
tracing 459.17 µs [438.284 µs, 480.055 µs] 78.009 µs (20.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 381.538 µs [360.835 µs, 402.241 µs] -
iast 515.759 µs [492.242 µs, 539.276 µs] 134.221 µs (35.2%)
iast_FULL 746.418 µs [724.26 µs, 768.575 µs] 364.879 µs (95.6%)
iast_GLOBAL 551.396 µs [530.033 µs, 572.76 µs] 169.858 µs (44.5%)
iast_HARDCODED_SECRET_DISABLED 509.131 µs [486.043 µs, 532.219 µs] 127.593 µs (33.4%)
iast_INACTIVE 460.076 µs [438.96 µs, 481.191 µs] 78.537 µs (20.6%)
iast_TELEMETRY_OFF 501.699 µs [478.189 µs, 525.209 µs] 120.161 µs (31.5%)
tracing 453.571 µs [432.704 µs, 474.439 µs] 72.033 µs (18.9%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~8573d24524, baseline=1.46.0-SNAPSHOT~56d4919491
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.353 ms) : 1333, 1373
.   : milestone, 1353,
appsec (1.737 ms) : 1713, 1761
.   : milestone, 1737,
appsec_no_iast (1.746 ms) : 1722, 1771
.   : milestone, 1746,
iast (1.509 ms) : 1484, 1533
.   : milestone, 1509,
profiling (1.533 ms) : 1507, 1558
.   : milestone, 1533,
tracing (1.49 ms) : 1465, 1515
.   : milestone, 1490,
section candidate
no_agent (1.345 ms) : 1325, 1365
.   : milestone, 1345,
appsec (1.77 ms) : 1746, 1794
.   : milestone, 1770,
appsec_no_iast (1.775 ms) : 1752, 1799
.   : milestone, 1775,
iast (1.506 ms) : 1482, 1530
.   : milestone, 1506,
profiling (1.499 ms) : 1474, 1523
.   : milestone, 1499,
tracing (1.499 ms) : 1473, 1524
.   : milestone, 1499,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.353 ms [1.333 ms, 1.373 ms] -
appsec 1.737 ms [1.713 ms, 1.761 ms] 383.959 µs (28.4%)
appsec_no_iast 1.746 ms [1.722 ms, 1.771 ms] 393.239 µs (29.1%)
iast 1.509 ms [1.484 ms, 1.533 ms] 155.512 µs (11.5%)
profiling 1.533 ms [1.507 ms, 1.558 ms] 179.511 µs (13.3%)
tracing 1.49 ms [1.465 ms, 1.515 ms] 137.118 µs (10.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.345 ms [1.325 ms, 1.365 ms] -
appsec 1.77 ms [1.746 ms, 1.794 ms] 424.305 µs (31.5%)
appsec_no_iast 1.775 ms [1.752 ms, 1.799 ms] 429.964 µs (32.0%)
iast 1.506 ms [1.482 ms, 1.53 ms] 160.355 µs (11.9%)
profiling 1.499 ms [1.474 ms, 1.523 ms] 153.141 µs (11.4%)
tracing 1.499 ms [1.473 ms, 1.524 ms] 153.232 µs (11.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1737654035 1737707604
git_commit_sha 56d4919 8573d24
release_version 1.46.0-SNAPSHOT~56d4919491 1.46.0-SNAPSHOT~8573d24524
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1737709705 1737709705
ci_job_id 779035799 779035799
ci_pipeline_id 53890093 53890093
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-nvtssqy9-project-304-concurrent-2-23veozpx 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-nvtssqy9-project-304-concurrent-2-23veozpx 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~8573d24524, baseline=1.46.0-SNAPSHOT~56d4919491
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.426 s) : 15426000, 15426000
.   : milestone, 15426000,
appsec (14.89 s) : 14890000, 14890000
.   : milestone, 14890000,
iast (19.02 s) : 19020000, 19020000
.   : milestone, 19020000,
iast_GLOBAL (17.902 s) : 17902000, 17902000
.   : milestone, 17902000,
profiling (15.664 s) : 15664000, 15664000
.   : milestone, 15664000,
tracing (14.83 s) : 14830000, 14830000
.   : milestone, 14830000,
section candidate
no_agent (14.889 s) : 14889000, 14889000
.   : milestone, 14889000,
appsec (15.134 s) : 15134000, 15134000
.   : milestone, 15134000,
iast (18.911 s) : 18911000, 18911000
.   : milestone, 18911000,
iast_GLOBAL (18.083 s) : 18083000, 18083000
.   : milestone, 18083000,
profiling (14.927 s) : 14927000, 14927000
.   : milestone, 14927000,
tracing (14.957 s) : 14957000, 14957000
.   : milestone, 14957000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.426 s [15.426 s, 15.426 s] -
appsec 14.89 s [14.89 s, 14.89 s] -536.0 ms (-3.5%)
iast 19.02 s [19.02 s, 19.02 s] 3.594 s (23.3%)
iast_GLOBAL 17.902 s [17.902 s, 17.902 s] 2.476 s (16.1%)
profiling 15.664 s [15.664 s, 15.664 s] 238.0 ms (1.5%)
tracing 14.83 s [14.83 s, 14.83 s] -596.0 ms (-3.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.889 s [14.889 s, 14.889 s] -
appsec 15.134 s [15.134 s, 15.134 s] 245.0 ms (1.6%)
iast 18.911 s [18.911 s, 18.911 s] 4.022 s (27.0%)
iast_GLOBAL 18.083 s [18.083 s, 18.083 s] 3.194 s (21.5%)
profiling 14.927 s [14.927 s, 14.927 s] 38.0 ms (0.3%)
tracing 14.957 s [14.957 s, 14.957 s] 68.0 ms (0.5%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~8573d24524, baseline=1.46.0-SNAPSHOT~56d4919491
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.463 ms) : 1452, 1474
.   : milestone, 1463,
appsec (2.347 ms) : 2304, 2390
.   : milestone, 2347,
iast (2.096 ms) : 2042, 2151
.   : milestone, 2096,
iast_GLOBAL (2.145 ms) : 2090, 2200
.   : milestone, 2145,
profiling (1.956 ms) : 1913, 2000
.   : milestone, 1956,
tracing (1.933 ms) : 1892, 1975
.   : milestone, 1933,
section candidate
no_agent (1.461 ms) : 1449, 1472
.   : milestone, 1461,
appsec (2.346 ms) : 2303, 2390
.   : milestone, 2346,
iast (2.088 ms) : 2034, 2142
.   : milestone, 2088,
iast_GLOBAL (2.131 ms) : 2076, 2186
.   : milestone, 2131,
profiling (1.96 ms) : 1916, 2005
.   : milestone, 1960,
tracing (1.931 ms) : 1890, 1973
.   : milestone, 1931,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.463 ms [1.452 ms, 1.474 ms] -
appsec 2.347 ms [2.304 ms, 2.39 ms] 883.904 µs (60.4%)
iast 2.096 ms [2.042 ms, 2.151 ms] 633.269 µs (43.3%)
iast_GLOBAL 2.145 ms [2.09 ms, 2.2 ms] 681.88 µs (46.6%)
profiling 1.956 ms [1.913 ms, 2.0 ms] 493.417 µs (33.7%)
tracing 1.933 ms [1.892 ms, 1.975 ms] 470.379 µs (32.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.461 ms [1.449 ms, 1.472 ms] -
appsec 2.346 ms [2.303 ms, 2.39 ms] 885.608 µs (60.6%)
iast 2.088 ms [2.034 ms, 2.142 ms] 627.216 µs (42.9%)
iast_GLOBAL 2.131 ms [2.076 ms, 2.186 ms] 670.094 µs (45.9%)
profiling 1.96 ms [1.916 ms, 2.005 ms] 499.692 µs (34.2%)
tracing 1.931 ms [1.89 ms, 1.973 ms] 470.759 µs (32.2%)

@smola smola force-pushed the smola/gradle-java11 branch from 702851a to d1abf3b Compare September 23, 2024 11:18
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 1, 2025

This pull request has been marked as stale because it has not had activity over the past quarter. It will be closed in 7 days if no further activity occurs. Feel free to reopen the PR if you are still working on it.

@github-actions github-actions bot added the tag: stale Stale pull requests label Jan 1, 2025
@smola smola force-pushed the smola/gradle-java11 branch 2 times, most recently from 0e79c09 to 0a43968 Compare January 15, 2025 15:19
@pr-commenter
Copy link

pr-commenter bot commented Jan 15, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1737357243 1737357619
end_time 2025-01-20T07:15:19 2025-01-20T07:21:36
git_branch master smola/gradle-java11
git_commit_sha 0b239ae 352736e
start_time 2025-01-20T07:14:04 2025-01-20T07:20:20
See matching parameters
Baseline Candidate
ci_job_id 772223798 772223798
ci_pipeline_id 53445931 53445931
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1737222884 1737222884

Summary

Found 0 performance improvements and 6 performance regressions! Performance is the same for 3 metrics, 6 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:basic worse
[+3.576µs; +11.815µs] or [+1.324%; +4.374%]		 same same unstable
[-23.734µs; +39.109µs] or [-4.673%; +7.700%]		 unstable
[-239.733op/s; +111.528op/s] or [-9.350%; +4.350%]
scenario:loop worse
[+442.270µs; +522.596µs] or [+4.356%; +5.147%]		 worse
[+449.109µs; +550.782µs] or [+4.353%; +5.338%]		 worse
[+410.327µs; +544.125µs] or [+3.935%; +5.218%]		 worse
[+195.678µs; +703.388µs] or [+1.812%; +6.512%]		 worse
[-5.491op/s; -3.248op/s] or [-5.738%; -3.394%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-26.397µs; +21.584µs] or [-9.744%; +7.968%]		 unstable
[-39.342µs; +33.189µs] or [-12.663%; +10.682%]		 unstable
[-52.143µs; +42.935µs] or [-16.038%; +13.206%]		 unstable
[-94.545µs; +97.240µs] or [-16.186%; +16.647%]		 same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (310.695 µs) : 270, 352
.   : milestone, 311,
basic (301.977 µs) : 293, 310
.   : milestone, 302,
loop (10.317 ms) : 10257, 10378
.   : milestone, 10317,
section candidate
noprobe (307.618 µs) : 281, 334
.   : milestone, 308,
basic (309.887 µs) : 301, 319
.   : milestone, 310,
loop (10.817 ms) : 10788, 10847
.   : milestone, 10817,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 310.695 µs [269.778 µs, 351.611 µs]
basic 301.977 µs [293.457 µs, 310.498 µs]
loop 10.317 ms [10.257 ms, 10.378 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 307.618 µs [280.843 µs, 334.393 µs]
basic 309.887 µs [301.101 µs, 318.674 µs]
loop 10.817 ms [10.788 ms, 10.847 ms]

@smola smola force-pushed the smola/gradle-java11 branch 3 times, most recently from 22e3221 to 352736e Compare January 20, 2025 06:59
@pr-commenter
Copy link

pr-commenter bot commented Jan 20, 2025 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1737654035 1737707604
git_commit_sha 56d4919 8573d24
See matching parameters
Baseline Candidate
ci_job_date 1737708769 1737708769
ci_job_id 779035801 779035801
ci_pipeline_id 53890093 53890093
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter
Copy link

pr-commenter bot commented Jan 20, 2025 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1737654035 1737707604
git_commit_sha 56d4919 8573d24
See matching parameters
Baseline Candidate
ci_job_date 1737708805 1737708805
ci_job_id 779035802 779035802
ci_pipeline_id 53890093 53890093
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@smola smola removed the tag: stale Stale pull requests label Jan 20, 2025
@smola smola force-pushed the smola/gradle-java11 branch from 352736e to 920f3ba Compare January 20, 2025 11:50
@smola smola force-pushed the smola/gradle-java11 branch from 920f3ba to 8573d24 Compare January 24, 2025 08:33
@smola smola mentioned this pull request Jan 24, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@smola