Skip to content

Conversation

@datadog-datadog-prod-us1
Copy link
Contributor

PR by Bits for chat 61249c30-95a0-4cc1-b232-cf5ac97bd137.

You can ask for changes by mentioning @DataDog in a comment.

Feedback (especially what can be better) welcome in #code-gen-feedback!

@pr-commenter
Copy link

pr-commenter bot commented Jul 8, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master dd-I2iP0S3LNcYq
git_commit_date 1751965004 1751967678
git_commit_sha 0f0b07e d8811a7
release_version 1.51.0-SNAPSHOT~0f0b07e5b7 1.51.0-SNAPSHOT~d8811a7374
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1751969421 1751969421
ci_job_id 1017769853 1017769853
ci_pipeline_id 69880499 69880499
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-1zlg6o1c 6.8.0-1030-aws #32~22.04.1-Ubuntu SMP Thu Jun 5 08:38:24 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-1zlg6o1c 6.8.0-1030-aws #32~22.04.1-Ubuntu SMP Thu Jun 5 08:38:24 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 8 unstable metrics.

Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~d8811a7374, baseline=1.51.0-SNAPSHOT~0f0b07e5b7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (996.62 ms) : 0, 996620
Total [baseline] (10.616 s) : 0, 10616031
Agent [candidate] (996.544 ms) : 0, 996544
Total [candidate] (10.601 s) : 0, 10601100
section appsec
Agent [baseline] (1.176 s) : 0, 1176423
Total [baseline] (10.78 s) : 0, 10780322
Agent [candidate] (1.176 s) : 0, 1176161
Total [candidate] (10.753 s) : 0, 10753366
section iast
Agent [baseline] (1.133 s) : 0, 1132525
Total [baseline] (10.866 s) : 0, 10866374
Agent [candidate] (1.138 s) : 0, 1138003
Total [candidate] (10.831 s) : 0, 10830606
section profiling
Agent [baseline] (1.244 s) : 0, 1243593
Total [baseline] (10.951 s) : 0, 10951301
Agent [candidate] (1.248 s) : 0, 1247521
Total [candidate] (11.003 s) : 0, 11003058
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 996.62 ms -
Agent appsec 1.176 s 179.803 ms (18.0%)
Agent iast 1.133 s 135.905 ms (13.6%)
Agent profiling 1.244 s 246.974 ms (24.8%)
Total tracing 10.616 s -
Total appsec 10.78 s 164.291 ms (1.5%)
Total iast 10.866 s 250.342 ms (2.4%)
Total profiling 10.951 s 335.27 ms (3.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 996.544 ms -
Agent appsec 1.176 s 179.617 ms (18.0%)
Agent iast 1.138 s 141.458 ms (14.2%)
Agent profiling 1.248 s 250.977 ms (25.2%)
Total tracing 10.601 s -
Total appsec 10.753 s 152.265 ms (1.4%)
Total iast 10.831 s 229.506 ms (2.2%)
Total profiling 11.003 s 401.958 ms (3.8%)
gantt
    title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~d8811a7374, baseline=1.51.0-SNAPSHOT~0f0b07e5b7

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (687.67 ms) : 0, 687670
BytebuddyAgent [candidate] (687.947 ms) : 0, 687947
GlobalTracer [baseline] (242.789 ms) : 0, 242789
GlobalTracer [candidate] (242.452 ms) : 0, 242452
AppSec [baseline] (30.282 ms) : 0, 30282
AppSec [candidate] (30.316 ms) : 0, 30316
Debugger [baseline] (6.056 ms) : 0, 6056
Debugger [candidate] (6.047 ms) : 0, 6047
Remote Config [baseline] (688.671 µs) : 0, 689
Remote Config [candidate] (679.036 µs) : 0, 679
Telemetry [baseline] (8.229 ms) : 0, 8229
Telemetry [candidate] (8.253 ms) : 0, 8253
section appsec
BytebuddyAgent [baseline] (710.774 ms) : 0, 710774
BytebuddyAgent [candidate] (710.563 ms) : 0, 710563
GlobalTracer [baseline] (235.248 ms) : 0, 235248
GlobalTracer [candidate] (235.681 ms) : 0, 235681
AppSec [baseline] (171.788 ms) : 0, 171788
AppSec [candidate] (171.552 ms) : 0, 171552
Debugger [baseline] (5.739 ms) : 0, 5739
Debugger [candidate] (5.716 ms) : 0, 5716
Remote Config [baseline] (604.063 µs) : 0, 604
Remote Config [candidate] (599.648 µs) : 0, 600
Telemetry [baseline] (8.088 ms) : 0, 8088
Telemetry [candidate] (8.02 ms) : 0, 8020
IAST [baseline] (23.268 ms) : 0, 23268
IAST [candidate] (23.201 ms) : 0, 23201
section iast
BytebuddyAgent [baseline] (807.311 ms) : 0, 807311
BytebuddyAgent [candidate] (811.116 ms) : 0, 811116
GlobalTracer [baseline] (232.631 ms) : 0, 232631
GlobalTracer [candidate] (233.702 ms) : 0, 233702
AppSec [baseline] (29.513 ms) : 0, 29513
AppSec [candidate] (32.389 ms) : 0, 32389
Debugger [baseline] (5.818 ms) : 0, 5818
Debugger [candidate] (5.868 ms) : 0, 5868
Remote Config [baseline] (589.491 µs) : 0, 589
Remote Config [candidate] (583.922 µs) : 0, 584
Telemetry [baseline] (7.964 ms) : 0, 7964
Telemetry [candidate] (7.987 ms) : 0, 7987
IAST [baseline] (27.864 ms) : 0, 27864
IAST [candidate] (25.384 ms) : 0, 25384
section profiling
ProfilingAgent [baseline] (103.853 ms) : 0, 103853
ProfilingAgent [candidate] (103.936 ms) : 0, 103936
BytebuddyAgent [baseline] (677.308 ms) : 0, 677308
BytebuddyAgent [candidate] (679.266 ms) : 0, 679266
GlobalTracer [baseline] (361.061 ms) : 0, 361061
GlobalTracer [candidate] (362.312 ms) : 0, 362312
AppSec [baseline] (32.347 ms) : 0, 32347
AppSec [candidate] (33.235 ms) : 0, 33235
Debugger [baseline] (11.245 ms) : 0, 11245
Debugger [candidate] (10.681 ms) : 0, 10681
Remote Config [baseline] (656.695 µs) : 0, 657
Remote Config [candidate] (661.211 µs) : 0, 661
Telemetry [baseline] (8.701 ms) : 0, 8701
Telemetry [candidate] (8.906 ms) : 0, 8906
Profiling [baseline] (103.877 ms) : 0, 103877
Profiling [candidate] (103.96 ms) : 0, 103960
Loading
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~d8811a7374, baseline=1.51.0-SNAPSHOT~0f0b07e5b7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.004 s) : 0, 1004454
Total [baseline] (8.646 s) : 0, 8646248
Agent [candidate] (996.173 ms) : 0, 996173
Total [candidate] (8.585 s) : 0, 8584610
section iast
Agent [baseline] (1.134 s) : 0, 1133750
Total [baseline] (9.309 s) : 0, 9309007
Agent [candidate] (1.138 s) : 0, 1138306
Total [candidate] (9.309 s) : 0, 9308866
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.004 s -
Agent iast 1.134 s 129.296 ms (12.9%)
Total tracing 8.646 s -
Total iast 9.309 s 662.759 ms (7.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 996.173 ms -
Agent iast 1.138 s 142.134 ms (14.3%)
Total tracing 8.585 s -
Total iast 9.309 s 724.256 ms (8.4%)
gantt
    title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~d8811a7374, baseline=1.51.0-SNAPSHOT~0f0b07e5b7

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (693.243 ms) : 0, 693243
BytebuddyAgent [candidate] (687.913 ms) : 0, 687913
GlobalTracer [baseline] (244.548 ms) : 0, 244548
GlobalTracer [candidate] (242.248 ms) : 0, 242248
AppSec [baseline] (30.557 ms) : 0, 30557
AppSec [candidate] (30.197 ms) : 0, 30197
Debugger [baseline] (6.133 ms) : 0, 6133
Debugger [candidate] (6.02 ms) : 0, 6020
Remote Config [baseline] (694.004 µs) : 0, 694
Remote Config [candidate] (685.311 µs) : 0, 685
Telemetry [baseline] (8.365 ms) : 0, 8365
Telemetry [candidate] (8.271 ms) : 0, 8271
section iast
BytebuddyAgent [baseline] (808.176 ms) : 0, 808176
BytebuddyAgent [candidate] (813.163 ms) : 0, 813163
GlobalTracer [baseline] (232.514 ms) : 0, 232514
GlobalTracer [candidate] (232.421 ms) : 0, 232421
AppSec [baseline] (31.361 ms) : 0, 31361
AppSec [candidate] (29.659 ms) : 0, 29659
Debugger [baseline] (6.688 ms) : 0, 6688
Debugger [candidate] (5.838 ms) : 0, 5838
Remote Config [baseline] (583.071 µs) : 0, 583
Remote Config [candidate] (569.289 µs) : 0, 569
Telemetry [baseline] (7.929 ms) : 0, 7929
Telemetry [candidate] (7.933 ms) : 0, 7933
IAST [baseline] (25.544 ms) : 0, 25544
IAST [candidate] (27.789 ms) : 0, 27789
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master dd-I2iP0S3LNcYq
git_commit_date 1751965004 1751967678
git_commit_sha 0f0b07e d8811a7
release_version 1.51.0-SNAPSHOT~0f0b07e5b7 1.51.0-SNAPSHOT~d8811a7374
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1751969097 1751969097
ci_job_id 1017769856 1017769856
ci_pipeline_id 69880499 69880499
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-ilw05qr2 6.8.0-1030-aws #32~22.04.1-Ubuntu SMP Thu Jun 5 08:38:24 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-ilw05qr2 6.8.0-1030-aws #32~22.04.1-Ubuntu SMP Thu Jun 5 08:38:24 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load better
[-875.513µs; -489.138µs] or [-8.201%; -4.582%]
unstable
[-20.817op/s; +79.692op/s] or [-4.780%; +18.300%]
9.993ms 464.906op/s 10.675ms 435.469op/s
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~d8811a7374, baseline=1.51.0-SNAPSHOT~0f0b07e5b7
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.297 ms) : 4242, 4353
.   : milestone, 4297,
iast (9.137 ms) : 8982, 9292
.   : milestone, 9137,
iast_FULL (14.081 ms) : 13803, 14358
.   : milestone, 14081,
iast_GLOBAL (10.675 ms) : 10488, 10863
.   : milestone, 10675,
profiling (8.87 ms) : 8728, 9012
.   : milestone, 8870,
tracing (7.554 ms) : 7447, 7660
.   : milestone, 7554,
section candidate
no_agent (4.421 ms) : 4372, 4470
.   : milestone, 4421,
iast (9.274 ms) : 9121, 9427
.   : milestone, 9274,
iast_FULL (13.819 ms) : 13543, 14096
.   : milestone, 13819,
iast_GLOBAL (9.993 ms) : 9822, 10164
.   : milestone, 9993,
profiling (8.709 ms) : 8558, 8860
.   : milestone, 8709,
tracing (7.612 ms) : 7498, 7726
.   : milestone, 7612,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.297 ms [4.242 ms, 4.353 ms] -
iast 9.137 ms [8.982 ms, 9.292 ms] 4.84 ms (112.6%)
iast_FULL 14.081 ms [13.803 ms, 14.358 ms] 9.783 ms (227.7%)
iast_GLOBAL 10.675 ms [10.488 ms, 10.863 ms] 6.378 ms (148.4%)
profiling 8.87 ms [8.728 ms, 9.012 ms] 4.573 ms (106.4%)
tracing 7.554 ms [7.447 ms, 7.66 ms] 3.256 ms (75.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.421 ms [4.372 ms, 4.47 ms] -
iast 9.274 ms [9.121 ms, 9.427 ms] 4.853 ms (109.8%)
iast_FULL 13.819 ms [13.543 ms, 14.096 ms] 9.398 ms (212.6%)
iast_GLOBAL 9.993 ms [9.822 ms, 10.164 ms] 5.572 ms (126.0%)
profiling 8.709 ms [8.558 ms, 8.86 ms] 4.288 ms (97.0%)
tracing 7.612 ms [7.498 ms, 7.726 ms] 3.191 ms (72.2%)
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~d8811a7374, baseline=1.51.0-SNAPSHOT~0f0b07e5b7
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.32 ms) : 36023, 36617
.   : milestone, 36320,
appsec (47.812 ms) : 47372, 48252
.   : milestone, 47812,
code_origins (45.131 ms) : 44756, 45505
.   : milestone, 45131,
iast (44.237 ms) : 43852, 44621
.   : milestone, 44237,
profiling (48.508 ms) : 48015, 49000
.   : milestone, 48508,
tracing (44.24 ms) : 43873, 44607
.   : milestone, 44240,
section candidate
no_agent (37.204 ms) : 36906, 37501
.   : milestone, 37204,
appsec (46.556 ms) : 46133, 46979
.   : milestone, 46556,
code_origins (45.972 ms) : 45561, 46384
.   : milestone, 45972,
iast (43.743 ms) : 43371, 44115
.   : milestone, 43743,
profiling (49.175 ms) : 48688, 49661
.   : milestone, 49175,
tracing (44.353 ms) : 43963, 44742
.   : milestone, 44353,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.32 ms [36.023 ms, 36.617 ms] -
appsec 47.812 ms [47.372 ms, 48.252 ms] 11.493 ms (31.6%)
code_origins 45.131 ms [44.756 ms, 45.505 ms] 8.811 ms (24.3%)
iast 44.237 ms [43.852 ms, 44.621 ms] 7.917 ms (21.8%)
profiling 48.508 ms [48.015 ms, 49.0 ms] 12.188 ms (33.6%)
tracing 44.24 ms [43.873 ms, 44.607 ms] 7.92 ms (21.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 37.204 ms [36.906 ms, 37.501 ms] -
appsec 46.556 ms [46.133 ms, 46.979 ms] 9.352 ms (25.1%)
code_origins 45.972 ms [45.561 ms, 46.384 ms] 8.769 ms (23.6%)
iast 43.743 ms [43.371 ms, 44.115 ms] 6.539 ms (17.6%)
profiling 49.175 ms [48.688 ms, 49.661 ms] 11.971 ms (32.2%)
tracing 44.353 ms [43.963 ms, 44.742 ms] 7.149 ms (19.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master dd-I2iP0S3LNcYq
git_commit_date 1751965004 1751967678
git_commit_sha 0f0b07e d8811a7
release_version 1.51.0-SNAPSHOT~0f0b07e5b7 1.51.0-SNAPSHOT~d8811a7374
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1751969566 1751969566
ci_job_id 1017769859 1017769859
ci_pipeline_id 69880499 69880499
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-k9bzqvpn 6.8.0-1030-aws #32~22.04.1-Ubuntu SMP Thu Jun 5 08:38:24 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-k9bzqvpn 6.8.0-1030-aws #32~22.04.1-Ubuntu SMP Thu Jun 5 08:38:24 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~d8811a7374, baseline=1.51.0-SNAPSHOT~0f0b07e5b7
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.632 s) : 15632000, 15632000
.   : milestone, 15632000,
appsec (14.808 s) : 14808000, 14808000
.   : milestone, 14808000,
iast (18.863 s) : 18863000, 18863000
.   : milestone, 18863000,
iast_GLOBAL (18.202 s) : 18202000, 18202000
.   : milestone, 18202000,
profiling (15.306 s) : 15306000, 15306000
.   : milestone, 15306000,
tracing (15.029 s) : 15029000, 15029000
.   : milestone, 15029000,
section candidate
no_agent (15.304 s) : 15304000, 15304000
.   : milestone, 15304000,
appsec (14.99 s) : 14990000, 14990000
.   : milestone, 14990000,
iast (17.927 s) : 17927000, 17927000
.   : milestone, 17927000,
iast_GLOBAL (17.903 s) : 17903000, 17903000
.   : milestone, 17903000,
profiling (15.247 s) : 15247000, 15247000
.   : milestone, 15247000,
tracing (15.19 s) : 15190000, 15190000
.   : milestone, 15190000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.632 s [15.632 s, 15.632 s] -
appsec 14.808 s [14.808 s, 14.808 s] -824.0 ms (-5.3%)
iast 18.863 s [18.863 s, 18.863 s] 3.231 s (20.7%)
iast_GLOBAL 18.202 s [18.202 s, 18.202 s] 2.57 s (16.4%)
profiling 15.306 s [15.306 s, 15.306 s] -326.0 ms (-2.1%)
tracing 15.029 s [15.029 s, 15.029 s] -603.0 ms (-3.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.304 s [15.304 s, 15.304 s] -
appsec 14.99 s [14.99 s, 14.99 s] -314.0 ms (-2.1%)
iast 17.927 s [17.927 s, 17.927 s] 2.623 s (17.1%)
iast_GLOBAL 17.903 s [17.903 s, 17.903 s] 2.599 s (17.0%)
profiling 15.247 s [15.247 s, 15.247 s] -57.0 ms (-0.4%)
tracing 15.19 s [15.19 s, 15.19 s] -114.0 ms (-0.7%)
Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~d8811a7374, baseline=1.51.0-SNAPSHOT~0f0b07e5b7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.468 ms) : 1457, 1480
.   : milestone, 1468,
appsec (2.391 ms) : 2342, 2440
.   : milestone, 2391,
iast (2.185 ms) : 2123, 2247
.   : milestone, 2185,
iast_GLOBAL (2.218 ms) : 2157, 2280
.   : milestone, 2218,
profiling (2.021 ms) : 1972, 2071
.   : milestone, 2021,
tracing (1.997 ms) : 1949, 2045
.   : milestone, 1997,
section candidate
no_agent (1.471 ms) : 1459, 1482
.   : milestone, 1471,
appsec (2.395 ms) : 2345, 2445
.   : milestone, 2395,
iast (2.186 ms) : 2124, 2249
.   : milestone, 2186,
iast_GLOBAL (2.232 ms) : 2170, 2295
.   : milestone, 2232,
profiling (2.035 ms) : 1984, 2086
.   : milestone, 2035,
tracing (2.001 ms) : 1953, 2049
.   : milestone, 2001,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.468 ms [1.457 ms, 1.48 ms] -
appsec 2.391 ms [2.342 ms, 2.44 ms] 922.912 µs (62.9%)
iast 2.185 ms [2.123 ms, 2.247 ms] 717.037 µs (48.8%)
iast_GLOBAL 2.218 ms [2.157 ms, 2.28 ms] 750.421 µs (51.1%)
profiling 2.021 ms [1.972 ms, 2.071 ms] 552.986 µs (37.7%)
tracing 1.997 ms [1.949 ms, 2.045 ms] 528.958 µs (36.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.471 ms [1.459 ms, 1.482 ms] -
appsec 2.395 ms [2.345 ms, 2.445 ms] 924.381 µs (62.9%)
iast 2.186 ms [2.124 ms, 2.249 ms] 715.641 µs (48.7%)
iast_GLOBAL 2.232 ms [2.17 ms, 2.295 ms] 761.86 µs (51.8%)
profiling 2.035 ms [1.984 ms, 2.086 ms] 564.819 µs (38.4%)
tracing 2.001 ms [1.953 ms, 2.049 ms] 530.19 µs (36.1%)

@PerfectSlayer PerfectSlayer added tag: ai generated Largely based on code generated by an AI or LLM and removed Bits AI labels Jul 8, 2025
@PerfectSlayer PerfectSlayer deleted the dd-I2iP0S3LNcYq branch July 8, 2025 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tag: ai generated Largely based on code generated by an AI or LLM

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants