Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,8 @@ A working example can be found at [openzeppelin-contracts, here.][35]
| solcoverjs | `--solcoverjs ./../.solcover.js` | Relative path from working directory to config. Useful for monorepo packages that share settings. (Path must be "./" prefixed) |
| network | `--network development` | Use network settings defined in the Truffle or Buidler config |
| temp[<sup>*</sup>][14] | `--temp build` | :warning: **Caution** :warning: Path to a *disposable* folder to store compilation artifacts in. Useful when your test setup scripts include hard-coded paths to a build directory. [More...][14] |
| matrix | `--matrix` | Generate a JSON object that maps which mocha tests hit which lines of code. (Useful
as an input for some fuzzing, mutation testing and fault-localization algorithms.) [More...][38]|

[<sup>*</sup> Advanced use][14]

Expand All @@ -123,6 +125,7 @@ module.exports = {
| measureStatementCoverage | *boolean* | `true` | Computes statement (in addition to line) coverage. [More...][34] |
| measureFunctionCoverage | *boolean* | `true` | Computes function coverage. [More...][34] |
| measureModifierCoverage | *boolean* | `true` | Computes each modifier invocation as a code branch. [More...][34] |
| matrixOutputPath | *String* | `./testMatrix.json` | Relative path to write test matrix JSON object to. [More...][38] |
| istanbulFolder | *String* | `./coverage` | Folder location for Istanbul coverage reports. |
| istanbulReporter | *Array* | `['html', 'lcov', 'text', 'json']` | [Istanbul coverage reporters][2] |
| mocha | *Object* | `{ }` | [Mocha options][3] to merge into existing mocha config. `grep` and `invert` are useful for skipping certain tests under coverage using tags in the test descriptions.|
Expand Down Expand Up @@ -232,4 +235,5 @@ $ yarn
[35]: https://github.com/OpenZeppelin/openzeppelin-contracts/blob/e5fbbda9bac49039847a7ed20c1d966766ecc64a/scripts/coverage.js
[36]: https://hardhat.org/
[37]: https://github.com/sc-forks/solidity-coverage/blob/master/HARDHAT_README.md
[38]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/advanced.md#generating-a-test-matrix

15 changes: 15 additions & 0 deletions docs/advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,3 +97,18 @@ Setting the `measureStatementCoverage` and/or `measureFunctionCoverage` options
improve performance, lower the cost of execution and minimize complications that arise from `solc`'s
limits on how large the compilation payload can be.

## Generating a test matrix

Some advanced testing strategies benefit from knowing which tests in a suite hit a
specific line of code. Examples include:
+ [mutation testing][22], where this data lets you select the correct subset of tests to check
a mutation with.
+ [fault localization techniques][23], where the complete data set is a key input to algorithms that try
to guess where bugs might exist in a given codebase.

Running the coverage command with `--matrix` will write [a JSON test matrix][25] which maps greppable
test names to each line of code to a file named `testMatrix.json` in your project's root.

[22]: https://github.com/JoranHonig/vertigo#vertigo
[23]: http://spideruci.org/papers/jones05.pdf
[25]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/matrix.md
Loading