Skip to content

[TEST] Testing changes for #7464 #7465

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 8 commits into from
Closed

[TEST] Testing changes for #7464 #7465

wants to merge 8 commits into from

Conversation

@smola
Copy link
Member

@smola smola commented Aug 18, 2024

What Does This Do

Test #7464

@smola smola added the tag: do not merge Do not merge changes label Aug 18, 2024
@smola smola force-pushed the smola/git-changes-test-verification branch from 78424c9 to 4af5abd Compare August 18, 2024 18:13
@pr-commenter
Copy link

pr-commenter bot commented Aug 18, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/git-changes-test-verification
git_commit_date 1724232069 1724232595
git_commit_sha 99cfdba e420db6
release_version 1.39.0-SNAPSHOT~99cfdba356 1.39.0-SNAPSHOT~e420db6f76
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1724234830 1724234830
ci_job_id 611816333 611816333
ci_pipeline_id 42395068 42395068
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 18 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1062769
Total [baseline] (10.391 s) : 0, 10390994
Agent [candidate] (1.049 s) : 0, 1049151
Total [candidate] (10.326 s) : 0, 10326048
section appsec
Agent [baseline] (1.175 s) : 0, 1174543
Total [baseline] (10.513 s) : 0, 10513112
Agent [candidate] (1.169 s) : 0, 1169292
Total [candidate] (10.454 s) : 0, 10454407
section iast
Agent [baseline] (1.184 s) : 0, 1184177
Total [baseline] (10.821 s) : 0, 10820818
Agent [candidate] (1.179 s) : 0, 1178681
Total [candidate] (10.85 s) : 0, 10849534
section profiling
Agent [baseline] (1.245 s) : 0, 1245154
Total [baseline] (10.56 s) : 0, 10559996
Agent [candidate] (1.263 s) : 0, 1262595
Total [candidate] (10.696 s) : 0, 10695611
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent appsec 1.175 s 111.773 ms (10.5%)
Agent iast 1.184 s 121.408 ms (11.4%)
Agent profiling 1.245 s 182.385 ms (17.2%)
Total tracing 10.391 s -
Total appsec 10.513 s 122.118 ms (1.2%)
Total iast 10.821 s 429.824 ms (4.1%)
Total profiling 10.56 s 169.003 ms (1.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.049 s -
Agent appsec 1.169 s 120.141 ms (11.5%)
Agent iast 1.179 s 129.53 ms (12.3%)
Agent profiling 1.263 s 213.444 ms (20.3%)
Total tracing 10.326 s -
Total appsec 10.454 s 128.359 ms (1.2%)
Total iast 10.85 s 523.486 ms (5.1%)
Total profiling 10.696 s 369.563 ms (3.6%) 
gantt
    title petclinic - break down per module: candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (678.581 ms) : 0, 678581
BytebuddyAgent [candidate] (669.037 ms) : 0, 669037
GlobalTracer [baseline] (310.454 ms) : 0, 310454
GlobalTracer [candidate] (307.132 ms) : 0, 307132
AppSec [baseline] (51.807 ms) : 0, 51807
AppSec [candidate] (51.283 ms) : 0, 51283
Remote Config [baseline] (699.502 µs) : 0, 700
Remote Config [candidate] (701.782 µs) : 0, 702
Telemetry [baseline] (7.624 ms) : 0, 7624
Telemetry [candidate] (7.517 ms) : 0, 7517
section appsec
BytebuddyAgent [baseline] (682.074 ms) : 0, 682074
BytebuddyAgent [candidate] (678.356 ms) : 0, 678356
GlobalTracer [baseline] (301.305 ms) : 0, 301305
GlobalTracer [candidate] (300.098 ms) : 0, 300098
AppSec [baseline] (157.458 ms) : 0, 157458
AppSec [candidate] (157.904 ms) : 0, 157904
Remote Config [baseline] (612.285 µs) : 0, 612
Remote Config [candidate] (603.19 µs) : 0, 603
Telemetry [baseline] (9.076 ms) : 0, 9076
Telemetry [candidate] (8.422 ms) : 0, 8422
IAST [baseline] (20.943 ms) : 0, 20943
IAST [candidate] (20.811 ms) : 0, 20811
section iast
BytebuddyAgent [baseline] (788.851 ms) : 0, 788851
BytebuddyAgent [candidate] (783.757 ms) : 0, 783757
GlobalTracer [baseline] (297.223 ms) : 0, 297223
GlobalTracer [candidate] (297.091 ms) : 0, 297091
AppSec [baseline] (52.71 ms) : 0, 52710
AppSec [candidate] (51.756 ms) : 0, 51756
Remote Config [baseline] (576.89 µs) : 0, 577
Remote Config [candidate] (584.753 µs) : 0, 585
Telemetry [baseline] (7.951 ms) : 0, 7951
Telemetry [candidate] (7.054 ms) : 0, 7054
IAST [baseline] (23.275 ms) : 0, 23275
IAST [candidate] (24.891 ms) : 0, 24891
section profiling
BytebuddyAgent [baseline] (663.646 ms) : 0, 663646
BytebuddyAgent [candidate] (673.935 ms) : 0, 673935
GlobalTracer [baseline] (388.961 ms) : 0, 388961
GlobalTracer [candidate] (393.93 ms) : 0, 393930
AppSec [baseline] (52.213 ms) : 0, 52213
AppSec [candidate] (52.362 ms) : 0, 52362
Remote Config [baseline] (705.453 µs) : 0, 705
Remote Config [candidate] (706.969 µs) : 0, 707
Telemetry [baseline] (7.474 ms) : 0, 7474
Telemetry [candidate] (7.517 ms) : 0, 7517
ProfilingAgent [baseline] (94.96 ms) : 0, 94960
ProfilingAgent [candidate] (96.309 ms) : 0, 96309
Profiling [baseline] (94.984 ms) : 0, 94984
Profiling [candidate] (96.334 ms) : 0, 96334
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1048357
Total [baseline] (8.492 s) : 0, 8491624
Agent [candidate] (1.047 s) : 0, 1047208
Total [candidate] (8.488 s) : 0, 8487706
section iast
Agent [baseline] (1.18 s) : 0, 1180398
Total [baseline] (8.973 s) : 0, 8973123
Agent [candidate] (1.179 s) : 0, 1178825
Total [candidate] (9.023 s) : 0, 9022958
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.18 s) : 0, 1179623
Total [baseline] (8.978 s) : 0, 8977807
Agent [candidate] (1.188 s) : 0, 1188073
Total [candidate] (8.999 s) : 0, 8999448
section iast_TELEMETRY_OFF
Agent [baseline] (1.172 s) : 0, 1171825
Total [baseline] (8.955 s) : 0, 8954933
Agent [candidate] (1.174 s) : 0, 1173695
Total [candidate] (9.003 s) : 0, 9002694
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.048 s -
Agent iast 1.18 s 132.041 ms (12.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.18 s 131.266 ms (12.5%)
Agent iast_TELEMETRY_OFF 1.172 s 123.468 ms (11.8%)
Total tracing 8.492 s -
Total iast 8.973 s 481.5 ms (5.7%)
Total iast_HARDCODED_SECRET_DISABLED 8.978 s 486.183 ms (5.7%)
Total iast_TELEMETRY_OFF 8.955 s 463.31 ms (5.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.047 s -
Agent iast 1.179 s 131.617 ms (12.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.188 s 140.865 ms (13.5%)
Agent iast_TELEMETRY_OFF 1.174 s 126.487 ms (12.1%)
Total tracing 8.488 s -
Total iast 9.023 s 535.252 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 8.999 s 511.743 ms (6.0%)
Total iast_TELEMETRY_OFF 9.003 s 514.989 ms (6.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (668.588 ms) : 0, 668588
BytebuddyAgent [candidate] (668.198 ms) : 0, 668198
GlobalTracer [baseline] (307.105 ms) : 0, 307105
GlobalTracer [candidate] (306.252 ms) : 0, 306252
AppSec [baseline] (50.98 ms) : 0, 50980
AppSec [candidate] (51.047 ms) : 0, 51047
Remote Config [baseline] (688.539 µs) : 0, 689
Remote Config [candidate] (698.977 µs) : 0, 699
Telemetry [baseline] (7.535 ms) : 0, 7535
Telemetry [candidate] (7.495 ms) : 0, 7495
section iast
BytebuddyAgent [baseline] (785.299 ms) : 0, 785299
BytebuddyAgent [candidate] (784.045 ms) : 0, 784045
GlobalTracer [baseline] (297.623 ms) : 0, 297623
GlobalTracer [candidate] (297.012 ms) : 0, 297012
AppSec [baseline] (50.824 ms) : 0, 50824
AppSec [candidate] (51.667 ms) : 0, 51667
IAST [baseline] (25.48 ms) : 0, 25480
IAST [candidate] (24.115 ms) : 0, 24115
Remote Config [baseline] (589.602 µs) : 0, 590
Remote Config [candidate] (568.219 µs) : 0, 568
Telemetry [baseline] (7.078 ms) : 0, 7078
Telemetry [candidate] (7.88 ms) : 0, 7880
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (784.736 ms) : 0, 784736
BytebuddyAgent [candidate] (789.657 ms) : 0, 789657
GlobalTracer [baseline] (298.018 ms) : 0, 298018
GlobalTracer [candidate] (299.389 ms) : 0, 299389
AppSec [baseline] (50.083 ms) : 0, 50083
AppSec [candidate] (50.674 ms) : 0, 50674
IAST [baseline] (23.289 ms) : 0, 23289
IAST [candidate] (22.846 ms) : 0, 22846
Remote Config [baseline] (583.943 µs) : 0, 584
Remote Config [candidate] (603.555 µs) : 0, 604
Telemetry [baseline] (9.387 ms) : 0, 9387
Telemetry [candidate] (11.197 ms) : 0, 11197
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (779.472 ms) : 0, 779472
BytebuddyAgent [candidate] (781.044 ms) : 0, 781044
GlobalTracer [baseline] (296.801 ms) : 0, 296801
GlobalTracer [candidate] (297.273 ms) : 0, 297273
AppSec [baseline] (52.406 ms) : 0, 52406
AppSec [candidate] (52.818 ms) : 0, 52818
IAST [baseline] (22.017 ms) : 0, 22017
IAST [candidate] (20.624 ms) : 0, 20624
Remote Config [baseline] (613.328 µs) : 0, 613
Remote Config [candidate] (611.792 µs) : 0, 612
Telemetry [baseline] (7.019 ms) : 0, 7019
Telemetry [candidate] (7.763 ms) : 0, 7763
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-08-21T09:39:52 2024-08-21T09:46:39
git_branch master smola/git-changes-test-verification
git_commit_date 1724232069 1724232595
git_commit_sha 99cfdba e420db6
release_version 1.39.0-SNAPSHOT~99cfdba356 1.39.0-SNAPSHOT~e420db6f76
start_time 2024-08-21T09:39:39 2024-08-21T09:46:26
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1724233942 1724233942
ci_job_id 611816335 611816335
ci_pipeline_id 42395068 42395068
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.355 ms) : 1336, 1375
.   : milestone, 1355,
appsec (1.727 ms) : 1702, 1752
.   : milestone, 1727,
appsec_no_iast (1.713 ms) : 1689, 1737
.   : milestone, 1713,
iast (1.461 ms) : 1439, 1483
.   : milestone, 1461,
profiling (1.497 ms) : 1473, 1522
.   : milestone, 1497,
tracing (1.459 ms) : 1434, 1484
.   : milestone, 1459,
section candidate
no_agent (1.349 ms) : 1329, 1368
.   : milestone, 1349,
appsec (1.727 ms) : 1704, 1750
.   : milestone, 1727,
appsec_no_iast (1.721 ms) : 1697, 1744
.   : milestone, 1721,
iast (1.465 ms) : 1442, 1488
.   : milestone, 1465,
profiling (1.492 ms) : 1468, 1516
.   : milestone, 1492,
tracing (1.461 ms) : 1437, 1486
.   : milestone, 1461,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.336 ms, 1.375 ms] -
appsec 1.727 ms [1.702 ms, 1.752 ms] 371.633 µs (27.4%)
appsec_no_iast 1.713 ms [1.689 ms, 1.737 ms] 357.728 µs (26.4%)
iast 1.461 ms [1.439 ms, 1.483 ms] 105.824 µs (7.8%)
profiling 1.497 ms [1.473 ms, 1.522 ms] 141.967 µs (10.5%)
tracing 1.459 ms [1.434 ms, 1.484 ms] 103.644 µs (7.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.349 ms [1.329 ms, 1.368 ms] -
appsec 1.727 ms [1.704 ms, 1.75 ms] 378.548 µs (28.1%)
appsec_no_iast 1.721 ms [1.697 ms, 1.744 ms] 371.963 µs (27.6%)
iast 1.465 ms [1.442 ms, 1.488 ms] 116.052 µs (8.6%)
profiling 1.492 ms [1.468 ms, 1.516 ms] 143.444 µs (10.6%)
tracing 1.461 ms [1.437 ms, 1.486 ms] 112.656 µs (8.4%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356
    dateFormat X
    axisFormat %s
section baseline
no_agent (364.429 µs) : 345, 384
.   : milestone, 364,
iast (471.273 µs) : 450, 492
.   : milestone, 471,
iast_FULL (546.244 µs) : 524, 568
.   : milestone, 546,
iast_GLOBAL (496.504 µs) : 475, 518
.   : milestone, 497,
iast_HARDCODED_SECRET_DISABLED (476.719 µs) : 456, 498
.   : milestone, 477,
iast_INACTIVE (447.687 µs) : 425, 470
.   : milestone, 448,
iast_TELEMETRY_OFF (458.909 µs) : 438, 480
.   : milestone, 459,
tracing (443.566 µs) : 423, 464
.   : milestone, 444,
section candidate
no_agent (362.951 µs) : 344, 382
.   : milestone, 363,
iast (477.626 µs) : 456, 499
.   : milestone, 478,
iast_FULL (544.967 µs) : 523, 567
.   : milestone, 545,
iast_GLOBAL (493.727 µs) : 472, 515
.   : milestone, 494,
iast_HARDCODED_SECRET_DISABLED (475.381 µs) : 454, 497
.   : milestone, 475,
iast_INACTIVE (448.362 µs) : 426, 470
.   : milestone, 448,
iast_TELEMETRY_OFF (459.272 µs) : 438, 480
.   : milestone, 459,
tracing (440.115 µs) : 419, 461
.   : milestone, 440,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.429 µs [345.178 µs, 383.68 µs] -
iast 471.273 µs [450.377 µs, 492.168 µs] 106.844 µs (29.3%)
iast_FULL 546.244 µs [524.004 µs, 568.485 µs] 181.816 µs (49.9%)
iast_GLOBAL 496.504 µs [474.648 µs, 518.36 µs] 132.075 µs (36.2%)
iast_HARDCODED_SECRET_DISABLED 476.719 µs [455.683 µs, 497.755 µs] 112.29 µs (30.8%)
iast_INACTIVE 447.687 µs [425.052 µs, 470.321 µs] 83.258 µs (22.8%)
iast_TELEMETRY_OFF 458.909 µs [438.077 µs, 479.741 µs] 94.48 µs (25.9%)
tracing 443.566 µs [422.847 µs, 464.285 µs] 79.137 µs (21.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 362.951 µs [343.696 µs, 382.207 µs] -
iast 477.626 µs [456.398 µs, 498.854 µs] 114.675 µs (31.6%)
iast_FULL 544.967 µs [522.642 µs, 567.293 µs] 182.016 µs (50.1%)
iast_GLOBAL 493.727 µs [472.031 µs, 515.424 µs] 130.776 µs (36.0%)
iast_HARDCODED_SECRET_DISABLED 475.381 µs [454.098 µs, 496.663 µs] 112.429 µs (31.0%)
iast_INACTIVE 448.362 µs [426.226 µs, 470.499 µs] 85.411 µs (23.5%)
iast_TELEMETRY_OFF 459.272 µs [438.433 µs, 480.111 µs] 96.321 µs (26.5%)
tracing 440.115 µs [419.403 µs, 460.827 µs] 77.163 µs (21.3%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/git-changes-test-verification
git_commit_date 1724232069 1724232595
git_commit_sha 99cfdba e420db6
release_version 1.39.0-SNAPSHOT~99cfdba356 1.39.0-SNAPSHOT~e420db6f76
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1724234464 1724234464
ci_job_id 611816336 611816336
ci_pipeline_id 42395068 42395068
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.455 ms) : 1444, 1467
.   : milestone, 1455,
appsec (2.218 ms) : 2182, 2254
.   : milestone, 2218,
iast (1.976 ms) : 1934, 2018
.   : milestone, 1976,
iast_GLOBAL (2.015 ms) : 1972, 2058
.   : milestone, 2015,
profiling (1.86 ms) : 1826, 1895
.   : milestone, 1860,
tracing (1.823 ms) : 1790, 1855
.   : milestone, 1823,
section candidate
no_agent (1.454 ms) : 1443, 1465
.   : milestone, 1454,
appsec (2.224 ms) : 2189, 2259
.   : milestone, 2224,
iast (1.955 ms) : 1913, 1997
.   : milestone, 1955,
iast_GLOBAL (2.002 ms) : 1959, 2044
.   : milestone, 2002,
profiling (1.862 ms) : 1828, 1896
.   : milestone, 1862,
tracing (1.838 ms) : 1805, 1870
.   : milestone, 1838,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.455 ms [1.444 ms, 1.467 ms] -
appsec 2.218 ms [2.182 ms, 2.254 ms] 762.679 µs (52.4%)
iast 1.976 ms [1.934 ms, 2.018 ms] 520.854 µs (35.8%)
iast_GLOBAL 2.015 ms [1.972 ms, 2.058 ms] 559.728 µs (38.5%)
profiling 1.86 ms [1.826 ms, 1.895 ms] 404.85 µs (27.8%)
tracing 1.823 ms [1.79 ms, 1.855 ms] 367.174 µs (25.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.454 ms [1.443 ms, 1.465 ms] -
appsec 2.224 ms [2.189 ms, 2.259 ms] 770.251 µs (53.0%)
iast 1.955 ms [1.913 ms, 1.997 ms] 501.346 µs (34.5%)
iast_GLOBAL 2.002 ms [1.959 ms, 2.044 ms] 547.744 µs (37.7%)
profiling 1.862 ms [1.828 ms, 1.896 ms] 408.211 µs (28.1%)
tracing 1.838 ms [1.805 ms, 1.87 ms] 383.728 µs (26.4%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~e420db6f76, baseline=1.39.0-SNAPSHOT~99cfdba356
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.492 s) : 15492000, 15492000
.   : milestone, 15492000,
appsec (15.256 s) : 15256000, 15256000
.   : milestone, 15256000,
iast (18.785 s) : 18785000, 18785000
.   : milestone, 18785000,
iast_GLOBAL (17.926 s) : 17926000, 17926000
.   : milestone, 17926000,
profiling (15.169 s) : 15169000, 15169000
.   : milestone, 15169000,
tracing (14.965 s) : 14965000, 14965000
.   : milestone, 14965000,
section candidate
no_agent (15.41 s) : 15410000, 15410000
.   : milestone, 15410000,
appsec (15.182 s) : 15182000, 15182000
.   : milestone, 15182000,
iast (18.602 s) : 18602000, 18602000
.   : milestone, 18602000,
iast_GLOBAL (17.847 s) : 17847000, 17847000
.   : milestone, 17847000,
profiling (15.363 s) : 15363000, 15363000
.   : milestone, 15363000,
tracing (15.063 s) : 15063000, 15063000
.   : milestone, 15063000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.492 s [15.492 s, 15.492 s] -
appsec 15.256 s [15.256 s, 15.256 s] -236.0 ms (-1.5%)
iast 18.785 s [18.785 s, 18.785 s] 3.293 s (21.3%)
iast_GLOBAL 17.926 s [17.926 s, 17.926 s] 2.434 s (15.7%)
profiling 15.169 s [15.169 s, 15.169 s] -323.0 ms (-2.1%)
tracing 14.965 s [14.965 s, 14.965 s] -527.0 ms (-3.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.41 s [15.41 s, 15.41 s] -
appsec 15.182 s [15.182 s, 15.182 s] -228.0 ms (-1.5%)
iast 18.602 s [18.602 s, 18.602 s] 3.192 s (20.7%)
iast_GLOBAL 17.847 s [17.847 s, 17.847 s] 2.437 s (15.8%)
profiling 15.363 s [15.363 s, 15.363 s] -47.0 ms (-0.3%)
tracing 15.063 s [15.063 s, 15.063 s] -347.0 ms (-2.3%)

@smola smola force-pushed the smola/git-changes-test branch from eebe257 to 096fc38 Compare August 18, 2024 19:45
@smola smola force-pushed the smola/git-changes-test-verification branch from 4af5abd to ce30672 Compare August 18, 2024 19:46
@smola smola force-pushed the smola/git-changes-test branch from 096fc38 to cc47be1 Compare August 19, 2024 07:18
@smola smola force-pushed the smola/git-changes-test-verification branch from ce30672 to 0dde356 Compare August 19, 2024 07:19
@pr-commenter
Copy link

pr-commenter bot commented Aug 19, 2024 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1724070506 1724070885
end_time 2024-08-19T12:29:40 2024-08-19T12:35:58
git_branch master smola/git-changes-test-verification
git_commit_sha 0c5f7c2 de8ff6a
start_time 2024-08-19T12:28:27 2024-08-19T12:34:46
See matching parameters
Baseline Candidate
ci_job_id 608692090 608692090
ci_pipeline_id 42109453 42109453
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1724069861 1724069861

Summary

Found 3 performance improvements and 0 performance regressions! Performance is the same for 5 metrics, 7 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:basic better
[-15.615µs; -4.689µs] or [-5.722%; -1.718%]		 better
[-23.254µs; -6.166µs] or [-7.757%; -2.057%]		 better
[-24.812µs; -4.496µs] or [-8.046%; -1.458%]		 unstable
[-47.518µs; +24.800µs] or [-8.126%; +4.241%]		 unstable
[-21.535op/s; +298.735op/s] or [-0.840%; +11.651%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-19.627µs; +21.967µs] or [-7.698%; +8.616%]		 unstable
[-32017.584ns; +32458.783ns] or [-11.078%; +11.231%]		 unstable
[-43402.685ns; +44547.579ns] or [-14.380%; +14.759%]		 unstable
[-112.147µs; +55.555µs] or [-17.615%; +8.726%]		 unstable
[-102.863op/s; +176.910op/s] or [-3.806%; +6.546%]
scenario:loop unsure
[-39.932µs; -33.645µs] or [-0.386%; -0.325%]		 same same same same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (289.008 µs) : 264, 314
.   : milestone, 289,
basic (299.79 µs) : 293, 307
.   : milestone, 300,
loop (10.531 ms) : 10494, 10568
.   : milestone, 10531,
section candidate
noprobe (289.228 µs) : 254, 325
.   : milestone, 289,
basic (285.08 µs) : 276, 295
.   : milestone, 285,
loop (10.497 ms) : 10456, 10537
.   : milestone, 10497,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 289.008 µs [264.122 µs, 313.893 µs]
basic 299.79 µs [292.826 µs, 306.753 µs]
loop 10.531 ms [10.494 ms, 10.568 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 289.228 µs [253.532 µs, 324.925 µs]
basic 285.08 µs [275.5 µs, 294.659 µs]
loop 10.497 ms [10.456 ms, 10.537 ms]

@smola smola force-pushed the smola/git-changes-test branch 2 times, most recently from 87b138c to d664538 Compare August 19, 2024 11:07
@smola smola force-pushed the smola/git-changes-test-verification branch 2 times, most recently from a182355 to de8ff6a Compare August 19, 2024 12:17
@smola smola force-pushed the smola/git-changes-test branch from f8991a1 to c7f43cb Compare August 19, 2024 13:34
@smola smola force-pushed the smola/git-changes-test-verification branch from de8ff6a to 7ea5ef8 Compare August 19, 2024 13:35
@smola smola force-pushed the smola/git-changes-test branch from a7047ea to 31efcdf Compare August 20, 2024 14:32
@smola smola force-pushed the smola/git-changes-test-verification branch from 7ea5ef8 to d70548b Compare August 20, 2024 14:32
@smola smola force-pushed the smola/git-changes-test branch from 31efcdf to a89f547 Compare August 20, 2024 16:18
@smola smola force-pushed the smola/git-changes-test-verification branch from d70548b to a37d9c4 Compare August 20, 2024 16:18
@smola smola force-pushed the smola/git-changes-test-verification branch from a37d9c4 to e420db6 Compare August 21, 2024 09:30
Base automatically changed from smola/git-changes-test to master August 21, 2024 13:34
@smola smola closed this Aug 21, 2024
@smola smola deleted the smola/git-changes-test-verification branch August 21, 2024 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

tag: do not merge Do not merge changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@smola