Skip to content

Conversation

@nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Feb 12, 2024

What Does This Do

Updates CI Visibility core and instrumentations so that caught exceptions are never swallowed.

Motivation

In future releases of the tracer it will be possible to report exceptions in the tracer code to Datadog using telemetry.
The goal is to see if there are any unexpected exceptions when CI Visibility is used in various envs.

Jira ticket: CIVIS-2427

@nikita-tkachenko-datadog nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Feb 12, 2024
@nikita-tkachenko-datadog nikita-tkachenko-datadog force-pushed the nikita-tkachenko/do-not-swallow-exceptions-in-ci-vis branch from b749a5c to 91afe0b Compare February 12, 2024 15:18
@pr-commenter
Copy link

pr-commenter bot commented Feb 12, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/do-not-swallow-exceptions-in-ci-vis
git_commit_date 1707748828 1707751053
git_commit_sha c42d72c 91afe0b
release_version 1.31.0-SNAPSHOT~c42d72c221 1.31.0-SNAPSHOT~91afe0b3fe
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1707754025 1707754025
ci_job_id 431788308 431788308
ci_pipeline_id 28131494 28131494
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 10 unstable metrics.

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-02-12T15:43:02 2024-02-12T16:02:02
git_branch master nikita-tkachenko/do-not-swallow-exceptions-in-ci-vis
git_commit_date 1707748828 1707751053
git_commit_sha c42d72c 91afe0b
release_version 1.31.0-SNAPSHOT~c42d72c221 1.31.0-SNAPSHOT~91afe0b3fe
start_time 2024-02-12T15:42:49 2024-02-12T16:01:49
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1707754025 1707754025
ci_job_id 431788308 431788308
ci_pipeline_id 28131494 28131494
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 16 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling better
[-120.983µs; -65.326µs] or [-7.507%; -4.054%]
unstable
[-404.541op/s; +632.461op/s] or [-13.653%; +21.346%]
1.518ms 3076.923op/s 1.612ms 2962.963op/s
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~91afe0b3fe, baseline=1.31.0-SNAPSHOT~c42d72c221
    dateFormat X
    axisFormat %s
section baseline
no_agent (372.148 µs) : 350, 394
.   : milestone, 372,
iast (476.091 µs) : 456, 497
.   : milestone, 476,
iast_FULL (545.006 µs) : 525, 565
.   : milestone, 545,
iast_GLOBAL (504.397 µs) : 483, 526
.   : milestone, 504,
iast_HARDCODED_SECRET_DISABLED (475.017 µs) : 455, 495
.   : milestone, 475,
iast_INACTIVE (450.733 µs) : 429, 472
.   : milestone, 451,
iast_TELEMETRY_OFF (479.335 µs) : 458, 501
.   : milestone, 479,
tracing (442.518 µs) : 422, 463
.   : milestone, 443,
section candidate
no_agent (374.987 µs) : 355, 395
.   : milestone, 375,
iast (475.86 µs) : 455, 497
.   : milestone, 476,
iast_FULL (532.512 µs) : 512, 553
.   : milestone, 533,
iast_GLOBAL (514.989 µs) : 493, 537
.   : milestone, 515,
iast_HARDCODED_SECRET_DISABLED (473.01 µs) : 452, 494
.   : milestone, 473,
iast_INACTIVE (454.62 µs) : 434, 476
.   : milestone, 455,
iast_TELEMETRY_OFF (475.683 µs) : 455, 496
.   : milestone, 476,
tracing (450.765 µs) : 430, 472
.   : milestone, 451,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 372.148 µs [350.258 µs, 394.038 µs] -
iast 476.091 µs [455.579 µs, 496.603 µs] 103.943 µs (27.9%)
iast_FULL 545.006 µs [524.694 µs, 565.317 µs] 172.858 µs (46.4%)
iast_GLOBAL 504.397 µs [482.583 µs, 526.211 µs] 132.249 µs (35.5%)
iast_HARDCODED_SECRET_DISABLED 475.017 µs [454.608 µs, 495.425 µs] 102.869 µs (27.6%)
iast_INACTIVE 450.733 µs [429.426 µs, 472.04 µs] 78.585 µs (21.1%)
iast_TELEMETRY_OFF 479.335 µs [457.806 µs, 500.864 µs] 107.187 µs (28.8%)
tracing 442.518 µs [422.124 µs, 462.911 µs] 70.37 µs (18.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 374.987 µs [354.883 µs, 395.09 µs] -
iast 475.86 µs [454.728 µs, 496.991 µs] 100.873 µs (26.9%)
iast_FULL 532.512 µs [511.845 µs, 553.178 µs] 157.525 µs (42.0%)
iast_GLOBAL 514.989 µs [492.682 µs, 537.295 µs] 140.002 µs (37.3%)
iast_HARDCODED_SECRET_DISABLED 473.01 µs [452.229 µs, 493.791 µs] 98.023 µs (26.1%)
iast_INACTIVE 454.62 µs [433.553 µs, 475.686 µs] 79.633 µs (21.2%)
iast_TELEMETRY_OFF 475.683 µs [454.894 µs, 496.471 µs] 100.696 µs (26.9%)
tracing 450.765 µs [429.603 µs, 471.927 µs] 75.778 µs (20.2%)
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~91afe0b3fe, baseline=1.31.0-SNAPSHOT~c42d72c221
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.385 ms) : 1366, 1404
.   : milestone, 1385,
appsec (1.78 ms) : 1754, 1805
.   : milestone, 1780,
iast (1.501 ms) : 1477, 1526
.   : milestone, 1501,
profiling (1.612 ms) : 1585, 1638
.   : milestone, 1612,
tracing (1.501 ms) : 1477, 1526
.   : milestone, 1501,
section candidate
no_agent (1.357 ms) : 1338, 1375
.   : milestone, 1357,
appsec (1.772 ms) : 1746, 1798
.   : milestone, 1772,
iast (1.5 ms) : 1476, 1525
.   : milestone, 1500,
profiling (1.518 ms) : 1493, 1544
.   : milestone, 1518,
tracing (1.516 ms) : 1491, 1541
.   : milestone, 1516,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.385 ms [1.366 ms, 1.404 ms] -
appsec 1.78 ms [1.754 ms, 1.805 ms] 394.286 µs (28.5%)
iast 1.501 ms [1.477 ms, 1.526 ms] 115.992 µs (8.4%)
profiling 1.612 ms [1.585 ms, 1.638 ms] 226.226 µs (16.3%)
tracing 1.501 ms [1.477 ms, 1.526 ms] 116.0 µs (8.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.357 ms [1.338 ms, 1.375 ms] -
appsec 1.772 ms [1.746 ms, 1.798 ms] 415.676 µs (30.6%)
iast 1.5 ms [1.476 ms, 1.525 ms] 143.772 µs (10.6%)
profiling 1.518 ms [1.493 ms, 1.544 ms] 161.89 µs (11.9%)
tracing 1.516 ms [1.491 ms, 1.541 ms] 159.784 µs (11.8%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 8f172ea into master Feb 13, 2024
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/do-not-swallow-exceptions-in-ci-vis branch February 13, 2024 09:24
@github-actions github-actions bot added this to the 1.30.0 milestone Feb 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: ci visibility Continuous Integration Visibility

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants