Add code coverage analysis

[zbeekman]

Use codcov.io to enable tracking and checks for code test coverage. Generate coverage information for as much as we can, but only measure/report on coverage of the core API/ABI files written in C and Fortran, and tested during Travis-CI unit/integration testing.

Browser plugins are available here to overlay coverage info when browsing gitub (Chrome, FireFox, Opera)

Here is a short video on the browser extensions:

[zbeekman]

Got unified Mac and Linux coverage reporting working via #65

See https://codecov.io/github/zbeekman/opencoarrays?view=all&ref=a53ac7e1da99f661fc2f8f8add17f95b793f4cf3

Note the menu item that says "unified". Here you can pull it down to look at the results for just one configuration.

[rouson]

Thanks for this. It's a great idea.

It suddenly strikes me that there are limitations to the code coverage concept (as you've already pointed out). For example, I'm a big fan of the Fortran MERGE intrinsic, which I use to write one-line conditional assignments. Hitting a statement like the following once only tests one branch of the inherent conditional:

function is_positive(x) result(which_am_i)
real, intent(in) :: x
logical :: i_am_positive
i_am_positive = merge(.true.,.false.,x>0)
end subroutine

so even a code coverage that is nominally 100% might not actually test all possible branches in the code. The above assignment could result in either which_am_i having a value of .true. or .false. depending on the value of x.

D

[zbeekman]

Yes, you are right. AND, even if a line of code was executed, there's no guarantee that the test was properly written to expose any defect that might be on that line. However, I think it can be a useful tool assess the state of general test coverage, and help make sure that contributors also write tests for their new code. It does give a macro assessment of the overall state of testing and QA.