Skip to content

🍒 9125 - Use token to access Akka secure repository #9239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

🍒 9125 - Use token to access Akka secure repository #9239

wants to merge 1 commit into from

Conversation

@sarahchen6
Copy link
Contributor

Backport #9125 to release/v1.50.x

@sarahchen6 sarahchen6 added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Jul 24, 2025
@sarahchen6 sarahchen6 requested review from a team as code owners July 24, 2025 13:58
@sarahchen6 sarahchen6 requested review from TonyCTHsu and removed request for a team July 24, 2025 13:58
@sarahchen6 sarahchen6 added this to the 1.50.2 milestone Jul 24, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jul 24, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarah.chen/backport-pr-9125
git_commit_date 1753362658 1753367867
git_commit_sha 4fde84f 47d6160b5d
release_version 1.51.1-SNAPSHOT~4fde84fc67 1.51.0-SNAPSHOT~747d6160b5d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1753369122 1753369122
ci_job_id 1045743734 1045743734
ci_pipeline_id 71585664 71585664
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-ro974uoz 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-ro974uoz 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 10 performance improvements and 5 performance regressions! Performance is the same for 24 metrics, 14 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:BytebuddyAgent better
[-49.607ms; -24.549ms] or [-5.851%; -2.895%]		 810.821ms 847.900ms
scenario:startup:insecure-bank:tracing:BytebuddyAgent better
[-54.442ms; -40.014ms] or [-7.426%; -5.458%]		 685.930ms 733.158ms
scenario:startup:petclinic:appsec:Agent.start better
[-46.581ms; -40.164ms] or [-3.821%; -3.295%]		 1.176s 1.219s
scenario:startup:petclinic:appsec:BytebuddyAgent better
[-46.599ms; -41.505ms] or [-6.195%; -5.518%]		 708.181ms 752.233ms
scenario:startup:petclinic:appsec:IAST better
[-1.969ms; -1.457ms] or [-8.341%; -6.171%]		 21.896ms 23.609ms
scenario:startup:petclinic:appsec:AppSec worse
[+5.377ms; +9.807ms] or [+3.195%; +5.828%]		 175.874ms 168.282ms
scenario:startup:petclinic:appsec:Telemetry better
[-1071.574µs; -814.709µs] or [-13.039%; -9.913%]		 7.275ms 8.218ms
scenario:startup:petclinic:iast:BytebuddyAgent better
[-63.500ms; -21.017ms] or [-7.379%; -2.442%]		 818.344ms 860.602ms
scenario:startup:petclinic:profiling:Agent.start worse
[+71.581ms; +105.550ms] or [+6.015%; +8.869%]		 1.279s 1.190s
scenario:startup:petclinic:profiling:BytebuddyAgent better
[-86.431ms; -66.529ms] or [-11.403%; -8.778%]		 681.461ms 757.942ms
scenario:startup:petclinic:profiling:GlobalTracer worse
[+138.765ms; +146.608ms] or [+62.999%; +66.560%]		 362.952ms 220.265ms
scenario:startup:petclinic:profiling:AppSec worse
[+31.087ms; +32.921ms] or [+102.963%; +109.037%]		 62.196ms 30.192ms
scenario:startup:petclinic:profiling:Telemetry better
[-8.172ms; -7.297ms] or [-50.895%; -45.440%]		 8.323ms 16.058ms
scenario:startup:petclinic:tracing:BytebuddyAgent better
[-45.418ms; -41.267ms] or [-6.227%; -5.658%]		 686.011ms 729.354ms
scenario:startup:petclinic:tracing:Remote Config worse
[+16.156µs; +41.072µs] or [+2.466%; +6.268%]		 683.878µs 655.263µs
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~747d6160b5d, baseline=1.51.1-SNAPSHOT~4fde84fc67

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.049 s) : 0, 1048677
Total [baseline] (8.592 s) : 0, 8592317
Agent [candidate] (1.025 s) : 0, 1025336
Total [candidate] (8.567 s) : 0, 8567437
section iast
Agent [baseline] (1.175 s) : 0, 1174528
Total [baseline] (9.302 s) : 0, 9302477
Agent [candidate] (1.158 s) : 0, 1158034
Total [candidate] (9.216 s) : 0, 9215792
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.049 s -
Agent iast 1.175 s 125.851 ms (12.0%)
Total tracing 8.592 s -
Total iast 9.302 s 710.16 ms (8.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.025 s -
Agent iast 1.158 s 132.698 ms (12.9%)
Total tracing 8.567 s -
Total iast 9.216 s 648.354 ms (7.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~747d6160b5d, baseline=1.51.1-SNAPSHOT~4fde84fc67

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (733.158 ms) : 0, 733158
BytebuddyAgent [candidate] (685.93 ms) : 0, 685930
GlobalTracer [baseline] (241.258 ms) : 0, 241258
GlobalTracer [candidate] (241.593 ms) : 0, 241593
AppSec [baseline] (30.396 ms) : 0, 30396
AppSec [candidate] (59.639 ms) : 0, 59639
Debugger [baseline] (5.976 ms) : 0, 5976
Debugger [candidate] (7.006 ms) : 0, 7006
Remote Config [baseline] (642.547 µs) : 0, 643
Remote Config [candidate] (656.388 µs) : 0, 656
Telemetry [baseline] (14.776 ms) : 0, 14776
Telemetry [candidate] (9.665 ms) : 0, 9665
crashtracking [baseline] (1.445 ms) : 0, 1445
section iast
BytebuddyAgent [baseline] (847.9 ms) : 0, 847900
BytebuddyAgent [candidate] (810.821 ms) : 0, 810821
GlobalTracer [baseline] (231.39 ms) : 0, 231390
GlobalTracer [candidate] (232.484 ms) : 0, 232484
AppSec [baseline] (26.546 ms) : 0, 26546
AppSec [candidate] (51.395 ms) : 0, 51395
Debugger [baseline] (7.591 ms) : 0, 7591
Debugger [candidate] (5.999 ms) : 0, 5999
Remote Config [baseline] (583.556 µs) : 0, 584
Remote Config [candidate] (603.085 µs) : 0, 603
Telemetry [baseline] (8.077 ms) : 0, 8077
Telemetry [candidate] (8.04 ms) : 0, 8040
crashtracking [baseline] (1.427 ms) : 0, 1427
IAST [baseline] (30.058 ms) : 0, 30058
IAST [candidate] (27.767 ms) : 0, 27767
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~747d6160b5d, baseline=1.51.1-SNAPSHOT~4fde84fc67

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.041 s) : 0, 1041174
Total [baseline] (10.696 s) : 0, 10695874
Agent [candidate] (1.025 s) : 0, 1025467
Total [candidate] (10.672 s) : 0, 10671886
section appsec
Agent [baseline] (1.219 s) : 0, 1218973
Total [baseline] (10.8 s) : 0, 10800124
Agent [candidate] (1.176 s) : 0, 1175600
Total [candidate] (10.713 s) : 0, 10712853
section iast
Agent [baseline] (1.19 s) : 0, 1190097
Total [baseline] (10.917 s) : 0, 10917207
Agent [candidate] (1.165 s) : 0, 1165105
Total [candidate] (10.908 s) : 0, 10907653
section profiling
Agent [baseline] (1.19 s) : 0, 1190075
Total [baseline] (10.955 s) : 0, 10955114
Agent [candidate] (1.279 s) : 0, 1278640
Total [candidate] (11.02 s) : 0, 11020372
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.041 s -
Agent appsec 1.219 s 177.8 ms (17.1%)
Agent iast 1.19 s 148.923 ms (14.3%)
Agent profiling 1.19 s 148.901 ms (14.3%)
Total tracing 10.696 s -
Total appsec 10.8 s 104.25 ms (1.0%)
Total iast 10.917 s 221.333 ms (2.1%)
Total profiling 10.955 s 259.239 ms (2.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.025 s -
Agent appsec 1.176 s 150.133 ms (14.6%)
Agent iast 1.165 s 139.638 ms (13.6%)
Agent profiling 1.279 s 253.173 ms (24.7%)
Total tracing 10.672 s -
Total appsec 10.713 s 40.967 ms (0.4%)
Total iast 10.908 s 235.768 ms (2.2%)
Total profiling 11.02 s 348.486 ms (3.3%) 
gantt
    title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~747d6160b5d, baseline=1.51.1-SNAPSHOT~4fde84fc67

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (729.354 ms) : 0, 729354
BytebuddyAgent [candidate] (686.011 ms) : 0, 686011
GlobalTracer [baseline] (241.389 ms) : 0, 241389
GlobalTracer [candidate] (241.935 ms) : 0, 241935
AppSec [baseline] (30.283 ms) : 0, 30283
AppSec [candidate] (56.169 ms) : 0, 56169
Debugger [baseline] (5.977 ms) : 0, 5977
Debugger [candidate] (7.032 ms) : 0, 7032
Remote Config [baseline] (655.263 µs) : 0, 655
Remote Config [candidate] (683.878 µs) : 0, 684
Telemetry [baseline] (11.102 ms) : 0, 11102
Telemetry [candidate] (12.726 ms) : 0, 12726
crashtracking [baseline] (1.429 ms) : 0, 1429
section appsec
BytebuddyAgent [baseline] (752.233 ms) : 0, 752233
BytebuddyAgent [candidate] (708.181 ms) : 0, 708181
GlobalTracer [baseline] (234.704 ms) : 0, 234704
GlobalTracer [candidate] (235.072 ms) : 0, 235072
AppSec [baseline] (168.282 ms) : 0, 168282
AppSec [candidate] (175.874 ms) : 0, 175874
Debugger [baseline] (8.588 ms) : 0, 8588
Debugger [candidate] (5.928 ms) : 0, 5928
Remote Config [baseline] (613.454 µs) : 0, 613
Remote Config [candidate] (612.762 µs) : 0, 613
Telemetry [baseline] (8.218 ms) : 0, 8218
Telemetry [candidate] (7.275 ms) : 0, 7275
crashtracking [baseline] (1.426 ms) : 0, 1426
IAST [baseline] (23.609 ms) : 0, 23609
IAST [candidate] (21.896 ms) : 0, 21896
section iast
BytebuddyAgent [baseline] (860.602 ms) : 0, 860602
BytebuddyAgent [candidate] (818.344 ms) : 0, 818344
GlobalTracer [baseline] (234.667 ms) : 0, 234667
GlobalTracer [candidate] (232.71 ms) : 0, 232710
AppSec [baseline] (27.563 ms) : 0, 27563
AppSec [candidate] (49.415 ms) : 0, 49415
Debugger [baseline] (7.411 ms) : 0, 7411
Debugger [candidate] (5.941 ms) : 0, 5941
Remote Config [baseline] (581.086 µs) : 0, 581
Remote Config [candidate] (616.308 µs) : 0, 616
Telemetry [baseline] (8.102 ms) : 0, 8102
Telemetry [candidate] (7.988 ms) : 0, 7988
crashtracking [baseline] (1.44 ms) : 0, 1440
IAST [baseline] (28.665 ms) : 0, 28665
IAST [candidate] (29.088 ms) : 0, 29088
section profiling
BytebuddyAgent [baseline] (757.942 ms) : 0, 757942
BytebuddyAgent [candidate] (681.461 ms) : 0, 681461
GlobalTracer [baseline] (220.265 ms) : 0, 220265
GlobalTracer [candidate] (362.952 ms) : 0, 362952
AppSec [baseline] (30.192 ms) : 0, 30192
AppSec [candidate] (62.196 ms) : 0, 62196
Debugger [baseline] (6.182 ms) : 0, 6182
Debugger [candidate] (6.143 ms) : 0, 6143
Remote Config [baseline] (680.244 µs) : 0, 680
Remote Config [candidate] (707.995 µs) : 0, 708
Telemetry [baseline] (16.058 ms) : 0, 16058
Telemetry [candidate] (8.323 ms) : 0, 8323
crashtracking [baseline] (1.408 ms) : 0, 1408
ProfilingAgent [baseline] (108.836 ms) : 0, 108836
ProfilingAgent [candidate] (108.05 ms) : 0, 108050
Profiling [baseline] (108.86 ms) : 0, 108860
Profiling [candidate] (108.074 ms) : 0, 108074
Loading

Load

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarah.chen/backport-pr-9125
git_commit_date 1753362658 1753367863
git_commit_sha 4fde84f 47d6160b5d
release_version 1.51.1-SNAPSHOT~4fde84fc67 1.51.0-SNAPSHOT~747d6160b5d
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1753369275 1753369275
ci_job_id 1045743736 1045743736
ci_pipeline_id 71585664 71585664
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-tz8tci9z 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-tz8tci9z 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 1 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:dacapo:tomcat:appsec better
[-1.399ms; -1.061ms] or [-38.564%; -29.245%]		 2.398ms 3.629ms
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~747d6160b5d, baseline=1.51.1-SNAPSHOT~4fde84fc67
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1463, 1487
.   : milestone, 1475,
appsec (3.629 ms) : 3412, 3845
.   : milestone, 3629,
iast (2.186 ms) : 2123, 2249
.   : milestone, 2186,
iast_GLOBAL (2.234 ms) : 2171, 2296
.   : milestone, 2234,
profiling (2.484 ms) : 2316, 2653
.   : milestone, 2484,
tracing (2.012 ms) : 1963, 2060
.   : milestone, 2012,
section candidate
no_agent (1.471 ms) : 1459, 1482
.   : milestone, 1471,
appsec (2.398 ms) : 2349, 2447
.   : milestone, 2398,
iast (2.191 ms) : 2129, 2253
.   : milestone, 2191,
iast_GLOBAL (2.232 ms) : 2170, 2294
.   : milestone, 2232,
profiling (2.052 ms) : 2000, 2103
.   : milestone, 2052,
tracing (2.014 ms) : 1966, 2063
.   : milestone, 2014,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.463 ms, 1.487 ms] -
appsec 3.629 ms [3.412 ms, 3.845 ms] 2.154 ms (146.0%)
iast 2.186 ms [2.123 ms, 2.249 ms] 710.938 µs (48.2%)
iast_GLOBAL 2.234 ms [2.171 ms, 2.296 ms] 758.677 µs (51.4%)
profiling 2.484 ms [2.316 ms, 2.653 ms] 1.009 ms (68.4%)
tracing 2.012 ms [1.963 ms, 2.06 ms] 536.679 µs (36.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.471 ms [1.459 ms, 1.482 ms] -
appsec 2.398 ms [2.349 ms, 2.447 ms] 927.554 µs (63.1%)
iast 2.191 ms [2.129 ms, 2.253 ms] 720.046 µs (49.0%)
iast_GLOBAL 2.232 ms [2.17 ms, 2.294 ms] 761.229 µs (51.8%)
profiling 2.052 ms [2.0 ms, 2.103 ms] 580.995 µs (39.5%)
tracing 2.014 ms [1.966 ms, 2.063 ms] 543.604 µs (37.0%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~747d6160b5d, baseline=1.51.1-SNAPSHOT~4fde84fc67
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.151 s) : 15151000, 15151000
.   : milestone, 15151000,
appsec (14.814 s) : 14814000, 14814000
.   : milestone, 14814000,
iast (18.325 s) : 18325000, 18325000
.   : milestone, 18325000,
iast_GLOBAL (17.928 s) : 17928000, 17928000
.   : milestone, 17928000,
profiling (15.16 s) : 15160000, 15160000
.   : milestone, 15160000,
tracing (14.941 s) : 14941000, 14941000
.   : milestone, 14941000,
section candidate
no_agent (15.238 s) : 15238000, 15238000
.   : milestone, 15238000,
appsec (14.796 s) : 14796000, 14796000
.   : milestone, 14796000,
iast (18.143 s) : 18143000, 18143000
.   : milestone, 18143000,
iast_GLOBAL (17.953 s) : 17953000, 17953000
.   : milestone, 17953000,
profiling (15.189 s) : 15189000, 15189000
.   : milestone, 15189000,
tracing (15.114 s) : 15114000, 15114000
.   : milestone, 15114000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.151 s [15.151 s, 15.151 s] -
appsec 14.814 s [14.814 s, 14.814 s] -337.0 ms (-2.2%)
iast 18.325 s [18.325 s, 18.325 s] 3.174 s (20.9%)
iast_GLOBAL 17.928 s [17.928 s, 17.928 s] 2.777 s (18.3%)
profiling 15.16 s [15.16 s, 15.16 s] 9.0 ms (0.1%)
tracing 14.941 s [14.941 s, 14.941 s] -210.0 ms (-1.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.238 s [15.238 s, 15.238 s] -
appsec 14.796 s [14.796 s, 14.796 s] -442.0 ms (-2.9%)
iast 18.143 s [18.143 s, 18.143 s] 2.905 s (19.1%)
iast_GLOBAL 17.953 s [17.953 s, 17.953 s] 2.715 s (17.8%)
profiling 15.189 s [15.189 s, 15.189 s] -49.0 ms (-0.3%)
tracing 15.114 s [15.114 s, 15.114 s] -124.0 ms (-0.8%)

@sarahchen6
Copy link
Contributor Author

We no longer need to proceed with the 1.50.2 release.

@sarahchen6 sarahchen6 closed this Jul 24, 2025
@sarahchen6 sarahchen6 deleted the sarah.chen/backport-pr-9125 branch October 16, 2025 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcculls mcculls mcculls approved these changes

@TonyCTHsu TonyCTHsu Awaiting requested review from TonyCTHsu TonyCTHsu is a code owner automatically assigned from DataDog/apm-release-platform

Assignees

No one assigned

Labels

comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.50.2

Development

Successfully merging this pull request may close these issues.

4 participants

@sarahchen6 @mcculls @randomanderson