|
| 1 | +--- |
| 2 | +group: contributor-guide |
| 3 | +title: Pull Request Tests |
| 4 | +--- |
| 5 | + |
| 6 | +## Magento Automated Testing |
| 7 | + |
| 8 | +Magento Automated Testing infrastructure runs all required automated tests against code in pull requests. You can trigger automated tests by adding a comment to a pull request. Tests are represented both as pull request statuses and checks. |
| 9 | + |
| 10 | +The following sections describe the different kinds of tests that are currently available. |
| 11 | + |
| 12 | +### Semantic Version Checker |
| 13 | + |
| 14 | +Semantic Version Checker is a test that compares the changed code against the mainline and detects levels of code changes. It shows a list of changes grouped by the class (level) of changes (MAJOR, MINOR, PATCH) with an explanation of the change (Class was removed, [public] Method implementation changed, [private] Property has been removed etc.) The allowed level is PATCH. |
| 15 | +See [Backward Compatibility Development](https://devdocs.magento.com/guides/v2.3/contributor-guide/backward-compatible-development/index.html) for more information. |
| 16 | + |
| 17 | +### Functional Tests |
| 18 | + |
| 19 | +Functional Tests are PHPUnit-based tests that check Magento functional behavior from the perspective of a user. They click buttons and links in a browser, fill and submit forms, verify what is displayed in the browser window like real users. |
| 20 | + |
| 21 | +### Sample Data Tests |
| 22 | + |
| 23 | +Sample Data Tests are functional tests that run on Magento with Sample Data included. |
| 24 | + |
| 25 | +### Unit Tests |
| 26 | + |
| 27 | +Unit tests are PHPUnit-based (for PHP) and Jasmine-based (for JS) tests that check the behavior of single methods of Magento classes. The purpose is to validate that each unit of Magento performs as designed. |
| 28 | + |
| 29 | +### Integration Tests |
| 30 | + |
| 31 | +Integration Tests are PHPUnit-based tests that check the behavior of a group of Magento components. They include three types of tests: |
| 32 | + |
| 33 | +- Standard Integration tests that check Magento behavior on an installed instance. |
| 34 | +- Setup Integration tests that check Magento behavior during the installation process. |
| 35 | +- Integration Integrity tests that check the consistency of an installed Magento instance (for example, configuration files and database schemes). |
| 36 | + |
| 37 | +### WebAPI Tests |
| 38 | + |
| 39 | +WebAPI Tests are functional tests that check the behavior of Magento Web API endpoints. They send requests to endpoints of an installed Magento instance and then check the response received and the state of the instance. Currently they include three kinds of tests: |
| 40 | + |
| 41 | +- SOAP Web API tests |
| 42 | +- REST Web API tests |
| 43 | +- GraphQL WEB API tests |
| 44 | + |
| 45 | +### Static Tests |
| 46 | + |
| 47 | +Static Tests are a set of different checks that analyze every single file of the Magento code base separately. Most of them analyze the whole Magento code base, whereas some run only against the change set. |
| 48 | + |
| 49 | +### Database Compare |
| 50 | + |
| 51 | +Database Compare is a test that compares the database schema and data between a freshly installed Magento instance (from the pull request code) and another instance that was upgraded to the PR code from the latest minor release version of Magento. |
| 52 | + |
| 53 | +### Magento Component Health Index |
| 54 | + |
| 55 | +Magento Component Health Index is a test that analyses Magento code quality and calculates a weighted health index (100 is great, lower is worse) for each Magento Component (module). The index is calculated based on three groups of indicators: |
| 56 | + |
| 57 | +- [General code quality indicators (GCQI)](http://pdepend.org/documentation/software-metrics/index.html) |
| 58 | +- Magento-specific code quality indicators (MSCQ) |
| 59 | +- Legacy API usage indicators (LAI) |
| 60 | + |
| 61 | +Additional information on Magento Automated Testing can be found on the [Magento 2 project wiki](https://github.com/magento/magento2/wiki/Magento-Automated-Testing). |
| 62 | + |
| 63 | +## Running tests for a pull request |
| 64 | + |
| 65 | +Automated tests can be triggered manually with an appropriate comment: |
| 66 | + |
| 67 | +- `@magento run all tests`—run or re-run all required tests against the pull request changes |
| 68 | +- `@magento run <test-build(s)>`—run or re-run specific test build(s) |
| 69 | + |
| 70 | +For example: `@magento run Unit Tests` or `@magento run Unit Tests,Integration Tests` |
| 71 | + |
| 72 | +`<test-build(s)>` is a comma-separated list of build names. Build names can be copied from the Checks section. |
| 73 | + |
| 74 | +## Test results |
| 75 | + |
| 76 | +The build progress and test results are indicated by check runs under the pull request conversation. |
| 77 | + |
| 78 | +Click on the "Details" link to see the results summary and links to corresponding Allure reports. |
| 79 | + |
| 80 | +## Related pull requests |
| 81 | + |
| 82 | +When a contribution requires changes to more than one repository merged simultaneously, such pull requests should be linked using the "Related Pull Requests" description section. |
| 83 | + |
| 84 | +The related pull requests should be specified in the pull request description in the following format: |
| 85 | + |
| 86 | +```lang-none |
| 87 | +### Related Pull Requests |
| 88 | +https://github.com/<organization>/<repository>/pull/<pull request number> |
| 89 | +<links to other realted PRs> |
| 90 | +<!-- related pull request placeholder --> |
| 91 | +``` |
| 92 | + |
| 93 | +When the tests are launched against a pull request that contains links to related pull request in the description, the related pull requests branches will be used as a code base for the tests instead of mainline. |
| 94 | +Check runs will be updated for each of the related pull requests. |
| 95 | + |
| 96 | +Related pull requests are delivered to Magento repositories simultaneously. |
0 commit comments