Redesign of plugin using external coverage model

[uhafner]

Refactoring of the recorder and step using the external Coverage Model.

Since we cannot make the new API (moving of the model classes to an external component, refactoring of the steps parameters, etc.) backward compatible, a new recorder (and step) has been added. The source code has been copied to the new metrics package so that old pipelines will still work out of the box. In order to use the new features, a new recorder needs to be used. The old code will be marked as deprecated and will not be supported anymore. It might get removed in a future version of this plugin.

Coverage of delta changes

The plugin now computes and shows the absolute coverage of the project, the coverage of the modified files and the coverage of the modified lines so you can see how the changes actually affect the code coverage. Additionally, the delta of these coverages with respect to the reference build are computed.

Cyclomatic Complexity and LOC metrics

Several coverage parsers support the measurement of cyclomatic complexity and lines of code. These metrics are now computed and recorded as well:

Support for mutation coverage (PIT)

The new model has been extended so that not only code coverage but also mutation coverage can be evaluated and stored. That means that you can visualize JaCoCo code coverage results along with PIT mutation testing results. So a single code base and pipeline configuration can be used for these different report formats:

Display of all Metrics in tooltips

The coverage column and the build summary will show all available metrics in a tooltip.

Quality Gates

You can now specify an arbitrary number of quality gates that allow to set the build to unstable or failed if the thresholds are not met. For each quality gate the metric (branch coverage, complexity, etc.) and the baseline (whole project, changed files, etc.) can be defined.

Integrate new model into Checks report

The plugin continues to publish checks to supporting SCM platforms. Now the details of the checks report have been updated to be in sync with the details of the Jenkins build results.

Since the new model supports reporting of checks annotations as well, you will now see if lines or branches are missing in a pull request. Additionally, survived mutations are highlighted in a pull request as well.

Native step

The recorder has been extended with a native step that is capable of setting the step status (unstable, failed, ok).

Multiple invocations

You can invoke the new recorder step multiple times. For each call you can specify a different set of features. E.g., your database layer can use different quality gates than your business layer.

Fixes also the following issues:

• #154
• #161
• #191
• #183
• #396
• #424
• #530
• #543

Open tasks

Tasks for first beta:

• Cleanup model API
• Cleanup Element Formatter
• Add option to change the name of the checks
• Add option to suppress checks annotations
• Generate checks annotations per metric
• Optimize XMLStream serialization
• Refactoring of classes: new package and remove obsolete methods
• Convert Checks publisher
• Support multiple actions (with different IDs)
• Support for quality gates
  • Allow different quality gates in a way similar to the warnings plugin
  • Fix UI for quality gates checkbox (depends on JENKINS-70139)
  • Add tooltip that shows the results of the quality gate evaluation
• Use same element selection for coverage column
• #424
• Add a pipeline step
  • Add the step
  • #177
  • Create JobDSL test case
• Adapt UI for PIT mutation test results
  • Summary for mutation coverage (including line coverage)
  • Overview for mutation coverage
  • Tree view for mutation results
  • Source code rendering of mutation results
  • Mutation coverage column support in tables
• #217
• Cleanup integration tests
• Add details table column for complexity

Tasks for 4.0.0:

• Each value needs to provide whether +/- is good or bad (Coverage: good, Complexity: bad)
• Move toggle of modified code into the table
• Restore UI tests
• Add details table column for maximum method complexity

Tasks after 4.0.0:

• New trend charts for the new value types
  • Complexity
  • LOC
  • #208
• #236 Support multiple invocations with same ID
• Support custom icons
• Add column that shows the overall result of a quality gate evaluation (general Jenkins API)

[codecov]

Codecov Report

Merging #512 (dd138a3) into master (ae115e4) will decrease coverage by 0.71%.
The diff coverage is 70.75%.

❗ Current head dd138a3 differs from pull request most recent head 7a5ab20. Consider uploading reports for the commit 7a5ab20 to get more accurate results

@@             Coverage Diff              @@
##             master     #512      +/-   ##
============================================
- Coverage     72.83%   72.13%   -0.71%     
- Complexity     1031     1499     +468     
============================================
  Files            88      129      +41     
  Lines          3825     5810    +1985     
  Branches        440      605     +165     
============================================
+ Hits           2786     4191    +1405     
- Misses          892     1417     +525     
- Partials        147      202      +55     

Impacted Files	Coverage Δ
...kins/plugins/coverage/metrics/SourceViewModel.java	0.00% <0.00%> (ø)
...etrics/visualization/colorization/ColorScheme.java	0.00% <0.00%> (ø)
...s/plugins/coverage/metrics/CoverageTableModel.java	10.40% <10.40%> (ø)
...ins/coverage/metrics/ChangeCoverageTableModel.java	11.53% <11.53%> (ø)
...coverage/metrics/IndirectCoverageChangesTable.java	12.50% <12.50%> (ø)
...ins/plugins/coverage/metrics/ElementFormatter.java	30.12% <30.12%> (ø)
...verage/metrics/visualization/code/PaintedNode.java	38.88% <38.88%> (ø)
...ns/plugins/coverage/metrics/CoverageJobAction.java	46.66% <46.66%> (ø)
.../plugins/coverage/metrics/CoverageBuildAction.java	57.95% <57.95%> (ø)
...ns/plugins/coverage/metrics/CoverageViewModel.java	58.19% <58.19%> (ø)
... and 31 more

... and 3 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more