Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove restriction to not run vertx4 latest tests on java 17 #8133

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Remove restriction to not run vertx4 latest tests on java 17 #8133

wants to merge 3 commits into from

Conversation

vandonr
Copy link
Contributor

@vandonr vandonr commented Dec 30, 2024 •
edited
Loading

What Does This Do

edit gradle file to remove exclusion for java 17 for latest dep tests

Motivation

this restriction was probably copy-pasted from the previous version's gradle file:

dd-trace-java/dd-java-agent/instrumentation/vertx-web-3.9/build.gradle

Line 3 in ec18d0e

// TODO Java 17: This version of vertx-web doesn't support Java 17

But vertx supports java 17 now, so we need to test the latest version with it.

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@vandonr vandonr marked this pull request as ready for review December 30, 2024 14:56
@vandonr vandonr requested a review from a team as a code owner December 30, 2024 14:56
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 30, 2024 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@vandonr vandonr added comp: testing Testing inst: vertx Eclipse Vert.x instrumentation run-tests: ubuntu17 Run tests using Ubuntu 17 JDK type: bug labels Dec 30, 2024
@pr-commenter
Copy link

pr-commenter bot commented Dec 30, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master vandonr/fix2
git_commit_date 1735568221 1735639012
git_commit_sha 0dc9c08 d3eef23
release_version 1.45.0-SNAPSHOT~0dc9c087f0 1.45.0-SNAPSHOT~d3eef233a2
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1735641376 1735641376
ci_job_id 751418430 751418430
ci_pipeline_id 51935313 51935313
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 3 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.45.0-SNAPSHOT~d3eef233a2, baseline=1.45.0-SNAPSHOT~0dc9c087f0

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.049 s) : 0, 1049210
Total [baseline] (10.403 s) : 0, 10403450
Agent [candidate] (1.058 s) : 0, 1058488
Total [candidate] (10.487 s) : 0, 10486904
section appsec
Agent [baseline] (1.189 s) : 0, 1188815
Total [baseline] (10.705 s) : 0, 10704570
Agent [candidate] (1.189 s) : 0, 1189419
Total [candidate] (10.755 s) : 0, 10755091
section iast
Agent [baseline] (1.181 s) : 0, 1180744
Total [baseline] (11.014 s) : 0, 11014354
Agent [candidate] (1.186 s) : 0, 1185685
Total [candidate] (10.975 s) : 0, 10974750
section profiling
Agent [baseline] (1.271 s) : 0, 1271080
Total [baseline] (10.821 s) : 0, 10820650
Agent [candidate] (1.272 s) : 0, 1272388
Total [candidate] (10.821 s) : 0, 10820549
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.049 s -
Agent appsec 1.189 s 139.605 ms (13.3%)
Agent iast 1.181 s 131.533 ms (12.5%)
Agent profiling 1.271 s 221.87 ms (21.1%)
Total tracing 10.403 s -
Total appsec 10.705 s 301.12 ms (2.9%)
Total iast 11.014 s 610.904 ms (5.9%)
Total profiling 10.821 s 417.2 ms (4.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.058 s -
Agent appsec 1.189 s 130.932 ms (12.4%)
Agent iast 1.186 s 127.198 ms (12.0%)
Agent profiling 1.272 s 213.901 ms (20.2%)
Total tracing 10.487 s -
Total appsec 10.755 s 268.187 ms (2.6%)
Total iast 10.975 s 487.846 ms (4.7%)
Total profiling 10.821 s 333.645 ms (3.2%) 
gantt
    title petclinic - break down per module: candidate=1.45.0-SNAPSHOT~d3eef233a2, baseline=1.45.0-SNAPSHOT~0dc9c087f0

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (710.219 ms) : 0, 710219
BytebuddyAgent [candidate] (715.866 ms) : 0, 715866
GlobalTracer [baseline] (256.021 ms) : 0, 256021
GlobalTracer [candidate] (257.499 ms) : 0, 257499
AppSec [baseline] (55.25 ms) : 0, 55250
AppSec [candidate] (55.124 ms) : 0, 55124
Remote Config [baseline] (709.525 µs) : 0, 710
Remote Config [candidate] (709.12 µs) : 0, 709
Telemetry [baseline] (12.073 ms) : 0, 12073
Telemetry [candidate] (14.206 ms) : 0, 14206
section appsec
BytebuddyAgent [baseline] (730.599 ms) : 0, 730599
BytebuddyAgent [candidate] (730.941 ms) : 0, 730941
GlobalTracer [baseline] (253.898 ms) : 0, 253898
GlobalTracer [candidate] (254.085 ms) : 0, 254085
AppSec [baseline] (170.773 ms) : 0, 170773
AppSec [candidate] (170.995 ms) : 0, 170995
Remote Config [baseline] (658.046 µs) : 0, 658
Remote Config [candidate] (657.306 µs) : 0, 657
Telemetry [baseline] (8.046 ms) : 0, 8046
Telemetry [candidate] (8.009 ms) : 0, 8009
IAST [baseline] (19.589 ms) : 0, 19589
IAST [candidate] (19.37 ms) : 0, 19370
section iast
BytebuddyAgent [baseline] (829.965 ms) : 0, 829965
BytebuddyAgent [candidate] (833.418 ms) : 0, 833418
GlobalTracer [baseline] (247.094 ms) : 0, 247094
GlobalTracer [candidate] (248.106 ms) : 0, 248106
AppSec [baseline] (58.079 ms) : 0, 58079
AppSec [candidate] (57.332 ms) : 0, 57332
Remote Config [baseline] (669.076 µs) : 0, 669
Remote Config [candidate] (662.145 µs) : 0, 662
Telemetry [baseline] (8.532 ms) : 0, 8532
Telemetry [candidate] (8.563 ms) : 0, 8563
IAST [baseline] (21.442 ms) : 0, 21442
IAST [candidate] (22.457 ms) : 0, 22457
section profiling
BytebuddyAgent [baseline] (700.385 ms) : 0, 700385
BytebuddyAgent [candidate] (701.316 ms) : 0, 701316
GlobalTracer [baseline] (371.059 ms) : 0, 371059
GlobalTracer [candidate] (370.541 ms) : 0, 370541
AppSec [baseline] (53.39 ms) : 0, 53390
AppSec [candidate] (54.709 ms) : 0, 54709
Remote Config [baseline] (675.01 µs) : 0, 675
Remote Config [candidate] (675.867 µs) : 0, 676
Telemetry [baseline] (7.831 ms) : 0, 7831
Telemetry [candidate] (7.843 ms) : 0, 7843
ProfilingAgent [baseline] (95.972 ms) : 0, 95972
ProfilingAgent [candidate] (95.623 ms) : 0, 95623
Profiling [baseline] (95.998 ms) : 0, 95998
Profiling [candidate] (95.652 ms) : 0, 95652
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.45.0-SNAPSHOT~d3eef233a2, baseline=1.45.0-SNAPSHOT~0dc9c087f0

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.053 s) : 0, 1053050
Total [baseline] (8.625 s) : 0, 8624541
Agent [candidate] (1.052 s) : 0, 1052208
Total [candidate] (8.623 s) : 0, 8623310
section iast
Agent [baseline] (1.194 s) : 0, 1193526
Total [baseline] (9.211 s) : 0, 9211295
Agent [candidate] (1.179 s) : 0, 1178958
Total [candidate] (9.183 s) : 0, 9182899
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.18 s) : 0, 1179907
Total [baseline] (9.176 s) : 0, 9176207
Agent [candidate] (1.178 s) : 0, 1178087
Total [candidate] (9.214 s) : 0, 9213732
section iast_TELEMETRY_OFF
Agent [baseline] (1.177 s) : 0, 1177028
Total [baseline] (9.255 s) : 0, 9255160
Agent [candidate] (1.183 s) : 0, 1183121
Total [candidate] (9.256 s) : 0, 9256131
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.053 s -
Agent iast 1.194 s 140.476 ms (13.3%)
Agent iast_HARDCODED_SECRET_DISABLED 1.18 s 126.857 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.177 s 123.978 ms (11.8%)
Total tracing 8.625 s -
Total iast 9.211 s 586.754 ms (6.8%)
Total iast_HARDCODED_SECRET_DISABLED 9.176 s 551.666 ms (6.4%)
Total iast_TELEMETRY_OFF 9.255 s 630.618 ms (7.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.052 s -
Agent iast 1.179 s 126.75 ms (12.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.178 s 125.879 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.183 s 130.913 ms (12.4%)
Total tracing 8.623 s -
Total iast 9.183 s 559.589 ms (6.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.214 s 590.422 ms (6.8%)
Total iast_TELEMETRY_OFF 9.256 s 632.822 ms (7.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.45.0-SNAPSHOT~d3eef233a2, baseline=1.45.0-SNAPSHOT~0dc9c087f0

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (710.172 ms) : 0, 710172
BytebuddyAgent [candidate] (711.697 ms) : 0, 711697
GlobalTracer [baseline] (256.199 ms) : 0, 256199
GlobalTracer [candidate] (257.556 ms) : 0, 257556
AppSec [baseline] (55.081 ms) : 0, 55081
AppSec [candidate] (55.229 ms) : 0, 55229
Remote Config [baseline] (715.24 µs) : 0, 715
Remote Config [candidate] (710.751 µs) : 0, 711
Telemetry [baseline] (15.805 ms) : 0, 15805
Telemetry [candidate] (11.94 ms) : 0, 11940
section iast
BytebuddyAgent [baseline] (840.212 ms) : 0, 840212
BytebuddyAgent [candidate] (828.403 ms) : 0, 828403
GlobalTracer [baseline] (249.24 ms) : 0, 249240
GlobalTracer [candidate] (246.836 ms) : 0, 246836
AppSec [baseline] (58.037 ms) : 0, 58037
AppSec [candidate] (58.11 ms) : 0, 58110
IAST [baseline] (21.592 ms) : 0, 21592
IAST [candidate] (21.405 ms) : 0, 21405
Remote Config [baseline] (658.073 µs) : 0, 658
Remote Config [candidate] (651.301 µs) : 0, 651
Telemetry [baseline] (8.606 ms) : 0, 8606
Telemetry [candidate] (8.505 ms) : 0, 8505
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (829.933 ms) : 0, 829933
BytebuddyAgent [candidate] (827.611 ms) : 0, 827611
GlobalTracer [baseline] (246.757 ms) : 0, 246757
GlobalTracer [candidate] (247.067 ms) : 0, 247067
AppSec [baseline] (57.873 ms) : 0, 57873
AppSec [candidate] (58.067 ms) : 0, 58067
IAST [baseline] (21.162 ms) : 0, 21162
IAST [candidate] (21.149 ms) : 0, 21149
Remote Config [baseline] (652.207 µs) : 0, 652
Remote Config [candidate] (644.164 µs) : 0, 644
Telemetry [baseline] (8.513 ms) : 0, 8513
Telemetry [candidate] (8.498 ms) : 0, 8498
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (827.154 ms) : 0, 827154
BytebuddyAgent [candidate] (829.705 ms) : 0, 829705
GlobalTracer [baseline] (247.393 ms) : 0, 247393
GlobalTracer [candidate] (249.619 ms) : 0, 249619
AppSec [baseline] (57.723 ms) : 0, 57723
AppSec [candidate] (58.196 ms) : 0, 58196
IAST [baseline] (20.763 ms) : 0, 20763
IAST [candidate] (21.25 ms) : 0, 21250
Remote Config [baseline] (649.627 µs) : 0, 650
Remote Config [candidate] (669.116 µs) : 0, 669
Telemetry [baseline] (8.347 ms) : 0, 8347
Telemetry [candidate] (8.591 ms) : 0, 8591
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-12-31T10:06:34 2024-12-31T10:13:35
git_branch master vandonr/fix2
git_commit_date 1735568221 1735639012
git_commit_sha 0dc9c08 d3eef23
release_version 1.45.0-SNAPSHOT~0dc9c087f0 1.45.0-SNAPSHOT~d3eef233a2
start_time 2024-12-31T10:06:21 2024-12-31T10:13:21
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1735640368 1735640368
ci_job_id 751418431 751418431
ci_pipeline_id 51935313 51935313
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 14 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~d3eef233a2, baseline=1.45.0-SNAPSHOT~0dc9c087f0
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.368 ms) : 1349, 1388
.   : milestone, 1368,
appsec (1.765 ms) : 1742, 1788
.   : milestone, 1765,
appsec_no_iast (1.756 ms) : 1732, 1781
.   : milestone, 1756,
iast (1.5 ms) : 1477, 1524
.   : milestone, 1500,
profiling (1.512 ms) : 1488, 1536
.   : milestone, 1512,
tracing (1.475 ms) : 1449, 1500
.   : milestone, 1475,
section candidate
no_agent (1.365 ms) : 1345, 1386
.   : milestone, 1365,
appsec (1.759 ms) : 1736, 1782
.   : milestone, 1759,
appsec_no_iast (1.752 ms) : 1727, 1776
.   : milestone, 1752,
iast (1.498 ms) : 1475, 1522
.   : milestone, 1498,
profiling (1.514 ms) : 1490, 1537
.   : milestone, 1514,
tracing (1.491 ms) : 1465, 1516
.   : milestone, 1491,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.368 ms [1.349 ms, 1.388 ms] -
appsec 1.765 ms [1.742 ms, 1.788 ms] 396.896 µs (29.0%)
appsec_no_iast 1.756 ms [1.732 ms, 1.781 ms] 388.207 µs (28.4%)
iast 1.5 ms [1.477 ms, 1.524 ms] 132.095 µs (9.7%)
profiling 1.512 ms [1.488 ms, 1.536 ms] 143.771 µs (10.5%)
tracing 1.475 ms [1.449 ms, 1.5 ms] 106.289 µs (7.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.365 ms [1.345 ms, 1.386 ms] -
appsec 1.759 ms [1.736 ms, 1.782 ms] 393.742 µs (28.8%)
appsec_no_iast 1.752 ms [1.727 ms, 1.776 ms] 386.08 µs (28.3%)
iast 1.498 ms [1.475 ms, 1.522 ms] 132.846 µs (9.7%)
profiling 1.514 ms [1.49 ms, 1.537 ms] 148.105 µs (10.8%)
tracing 1.491 ms [1.465 ms, 1.516 ms] 125.101 µs (9.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master vandonr/fix2
git_commit_date 1735568221 1735639012
git_commit_sha 0dc9c08 d3eef23
release_version 1.45.0-SNAPSHOT~0dc9c087f0 1.45.0-SNAPSHOT~d3eef233a2
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1735641082 1735641082
ci_job_id 751418432 751418432
ci_pipeline_id 51935313 51935313
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~d3eef233a2, baseline=1.45.0-SNAPSHOT~0dc9c087f0
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.474 ms) : 1463, 1486
.   : milestone, 1474,
appsec (2.359 ms) : 2317, 2400
.   : milestone, 2359,
iast (2.089 ms) : 2037, 2142
.   : milestone, 2089,
iast_GLOBAL (2.131 ms) : 2078, 2184
.   : milestone, 2131,
profiling (1.949 ms) : 1908, 1991
.   : milestone, 1949,
tracing (1.931 ms) : 1891, 1971
.   : milestone, 1931,
section candidate
no_agent (1.473 ms) : 1462, 1485
.   : milestone, 1473,
appsec (2.354 ms) : 2312, 2396
.   : milestone, 2354,
iast (2.096 ms) : 2043, 2148
.   : milestone, 2096,
iast_GLOBAL (2.135 ms) : 2082, 2188
.   : milestone, 2135,
profiling (1.982 ms) : 1939, 2025
.   : milestone, 1982,
tracing (1.947 ms) : 1906, 1988
.   : milestone, 1947,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.474 ms [1.463 ms, 1.486 ms] -
appsec 2.359 ms [2.317 ms, 2.4 ms] 884.336 µs (60.0%)
iast 2.089 ms [2.037 ms, 2.142 ms] 614.927 µs (41.7%)
iast_GLOBAL 2.131 ms [2.078 ms, 2.184 ms] 656.772 µs (44.5%)
profiling 1.949 ms [1.908 ms, 1.991 ms] 475.16 µs (32.2%)
tracing 1.931 ms [1.891 ms, 1.971 ms] 456.754 µs (31.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.462 ms, 1.485 ms] -
appsec 2.354 ms [2.312 ms, 2.396 ms] 880.871 µs (59.8%)
iast 2.096 ms [2.043 ms, 2.148 ms] 622.357 µs (42.2%)
iast_GLOBAL 2.135 ms [2.082 ms, 2.188 ms] 661.615 µs (44.9%)
profiling 1.982 ms [1.939 ms, 2.025 ms] 508.603 µs (34.5%)
tracing 1.947 ms [1.906 ms, 1.988 ms] 473.852 µs (32.2%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~d3eef233a2, baseline=1.45.0-SNAPSHOT~0dc9c087f0
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.72 s) : 14720000, 14720000
.   : milestone, 14720000,
appsec (14.971 s) : 14971000, 14971000
.   : milestone, 14971000,
iast (18.499 s) : 18499000, 18499000
.   : milestone, 18499000,
iast_GLOBAL (18.228 s) : 18228000, 18228000
.   : milestone, 18228000,
profiling (15.137 s) : 15137000, 15137000
.   : milestone, 15137000,
tracing (15.064 s) : 15064000, 15064000
.   : milestone, 15064000,
section candidate
no_agent (15.685 s) : 15685000, 15685000
.   : milestone, 15685000,
appsec (14.946 s) : 14946000, 14946000
.   : milestone, 14946000,
iast (18.398 s) : 18398000, 18398000
.   : milestone, 18398000,
iast_GLOBAL (18.048 s) : 18048000, 18048000
.   : milestone, 18048000,
profiling (15.143 s) : 15143000, 15143000
.   : milestone, 15143000,
tracing (15.162 s) : 15162000, 15162000
.   : milestone, 15162000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.72 s [14.72 s, 14.72 s] -
appsec 14.971 s [14.971 s, 14.971 s] 251.0 ms (1.7%)
iast 18.499 s [18.499 s, 18.499 s] 3.779 s (25.7%)
iast_GLOBAL 18.228 s [18.228 s, 18.228 s] 3.508 s (23.8%)
profiling 15.137 s [15.137 s, 15.137 s] 417.0 ms (2.8%)
tracing 15.064 s [15.064 s, 15.064 s] 344.0 ms (2.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.685 s [15.685 s, 15.685 s] -
appsec 14.946 s [14.946 s, 14.946 s] -739.0 ms (-4.7%)
iast 18.398 s [18.398 s, 18.398 s] 2.713 s (17.3%)
iast_GLOBAL 18.048 s [18.048 s, 18.048 s] 2.363 s (15.1%)
profiling 15.143 s [15.143 s, 15.143 s] -542.0 ms (-3.5%)
tracing 15.162 s [15.162 s, 15.162 s] -523.0 ms (-3.3%)

@vandonr vandonr changed the title Remove restriction to not run vertx4 tests on java 17 Remove restriction to not run vertx4 latest tests on java 17 Dec 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@amarziali amarziali amarziali left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
comp: testing Testing inst: vertx Eclipse Vert.x instrumentation run-tests: ubuntu17 Run tests using Ubuntu 17 JDK type: bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vandonr @amarziali