Cleanup difficult to maintain Travis setup and addition of enhancements #3567

davidalger · 2016-02-28T00:25:11Z

Cleans up .travis.yml by moving before_install, before_script and script portions to .sh files
All script components are now in a single location (specifically dev/travis)
Static tests and annotation tests are no longer performed twice (excluded from PHP 7.0 as they were for PHP 5.6)
Using builtin travis config to install apt packages such as mysql 5.6 and postfix
Integration test splitting is now 100% maintained by the env vars configured in .travis.yml (INTEGRATION_INDEX and INTEGRATION_SETS)
Added support for using a GITHUB_TOKEN environment variable to speed up builds

This in combination with setting a public_access API token on the repository will speed up the test suite by about 20 minutes by cutting ~85 seconds off each call to composer install since with this token the deps may be installed from the distribution vs using a clone

I've set a public_access GitHub API O-Auth token on the magento/magento2 repository. It will show up in builds, but per the documentation is read-only and limited to public information:

Grants read-only access to public information (includes public user profile info, public repository info, and gists)

UPDATE: Now includes the following as well:

Moved testsuite/Magento/Test/Integrity tests into main integration build
Combined static_phpcs and static_annotation into a single build in the matrix
Integration testsuite preparation is now logging what goes into which build set
All builds are now using the default phpunit configuration file load behavior vs explicit --configuration input

…(done in vm setup already)

- moves everything into one place, the before_script.sh - now removes `update/dev/tests/integration/testsuite` set from all but test set 1 - integration set handling should be flexible by configuration env vars in single location (.travis.xml) - fixed an oops in d2884f0 (accidental change to matrix exclusions)

…ravis-mods

…ion test build

- this is preferable given the test suite preparation being done on the fly for integration testing

…won't run outside of a directory during split testing on travis)

Flyingmana · 2016-02-28T13:16:07Z

cant say much to the actual changes, as mixing them with the refactoring makes it to much work for me.

But a comment to the Github Token.
Even if it is only for read only access, its not a good idea to make it visible. It can be reused by everyone for abusive access, and can so as a side effect kill the rate limit.

davidalger · 2016-02-29T14:44:37Z

@Flyingmana Good point. Unfortunately if the env vars are set securely (i.e. hidden from logs, etc) they are not available to builds running for PRs though.

Does it make a difference in your mind knowing that there is still a rate limit in place when using an API access token? The authenticated rate-limit is set at 5000 requests per hour vs the unauthenticated limit of 60 per hr per IP address.

barryvdh · 2016-02-29T15:16:12Z

There are some open issues about using direct links from Github, so hopefully it soon isn't needed anymore to use the Github tokens: composer/composer#4944 and composer/composer#4884

barryvdh · 2016-02-29T15:27:22Z

Also, just curious, why do you need to install a custom composer version? It that still needed? According to docs, they're updated every 30-60 days: https://docs.travis-ci.com/user/languages/php#Installing-Composer-packages

buskamuza · 2016-03-02T22:10:02Z

dev/travis/script.sh

+        ;;
+    static)
+        cd dev/tests/static
+        phpunit --filter 'Magento\\Test\\Php\\LiveCodeTest::(testCodeStyle|testAnnotationStandard)'


These tests were intentionally broken down into two parts because they took much time. Right now they are fast because, looks like, they just don't test anything. So I'd leave these tests separate, as it may happen that they will become slow again when fixed.

That would make sense. Would it be acceptable to keep as it is here and expect that when the tests are updated to actually run against the changed files, this can be split again if that performance issue is run into (it would be a very minor change to split them at that point)?

Based on my review, these will only run the static tests on files changed in the current PR, but they are not actually doing anything because the changed file lists aren't being populated, and so I suspect that they may not really take longer than the max job run times when setup correctly.

I'm willing to tackle getting those change lists populating (separate PR, after this is merged in) so we can have them running on travis builds, and if any speed issues are encountered can easily split them.

Let me know your thought, and if you'd rather I just leave them split for now, I can do that too.

Actually @mazhalai is already working on fixing static tests. Example can be found here - https://travis-ci.org/mazhalai/magento2/builds/115163946#L746 and https://travis-ci.org/mazhalai/magento2/jobs/115362517#L594 . But thanks for the help :)
It looks pretty fast now, but it may take a lot of time for phpmd in case of changes across whole code base (rare case, but still may happen).
I don't insist, though I don't see a big benefit from merging these tests together.

I went ahead and split them up how they were before in f36c3c7. Looking forward to seeing the result of @mazhalai's work! It'd be great to have them all running here on Travis :)

Interesting, that @mazhalai has also splitted them... :) while we were discussing it here.
Let's leave it as is then and we'll merge all together, when will be merging the PR and internal changed together.
Sorry for this long discussion.

…ravis-mods

hshar7 · 2016-03-08T21:37:18Z

Internal ticket to process this: MAGETWO-50237

hshar7 · 2016-03-08T21:59:02Z

@davidalger it seems like the tests are timing out. Could you please take a look

arkadiych · 2016-04-29T09:08:41Z

@davidalger, please update with the latest code base and resolve conflicts

This reverts commit f36c3c7.

…ravis-mods Conflicts: - .travis.yml (changes made to this file moved into dev/travis/script.sh) - dev/tests/integration/IntegationTestsForTravis.sh (only incoming change was the copyright date change) - dev/tests/integration/phpunit.xml.dist

…commit 342f4e6

…ce reason for introduction is now mute

davidalger · 2016-04-30T16:22:03Z

@arkadiych Done. The updates to the static tests made by @mazhalai have been incorporated, and everything built and passed as green. Let me know if you need anything else on this to get it merged in. I'd love to see it make it into the mainline soon! :)

okorshenko · 2016-05-10T23:04:29Z

@davidalger Thank you for contribution. Your Pull Request merged to develop branch.

[TSG-CSL3] For 2.2 (happy New Year ;)

David Alger added 14 commits February 26, 2016 13:37

Changed travis builds to use addons for getting MySql 5.6 in Trusty

6555bbe

Moved postfix to apt addon configuration and removed explicit update …

6e03869

…(done in vm setup already)

Excluded PHP 7 static tests from matrix

00b7046

Refactored travis scripts out of main travis config file

d2884f0

Added support for GITHUB_TOKEN use in travis build

0c6013c

Merge remote-tracking branch 'upstream/develop' into feature/trusty-t…

eac2553

…ravis-mods

Moves the testsuite/Magento/Test/Integrity tests into main integrat…

1c4fc58

…ion test build

Combines two static tests into a single static test build

4d0825f

Switch phpunit setup to use default configuration file loading mechanism

86e2ac8

- this is preferable given the test suite preparation being done on the fly for integration testing

Trimmed matrix exclusions to reflect current test divisions

f1e19c7

Fixed issue caused by using non POSIX -s flag on the find command… :/

4ca7f7c

Correct test preparation messaging

4bb0cbd

Moved DatabaseTest.php into Test/Integrity where it belongs (also it …

57712e6

…won't run outside of a directory during split testing on travis)

buskamuza reviewed Mar 2, 2016
View reviewed changes

Avoid relative path to composer vendor bin in travis tests

3e8865f

davidalger force-pushed the feature/trusty-travis-mods branch from 8f9903c to 3e8865f Compare March 4, 2016 18:18

Merge remote-tracking branch 'upstream/develop' into feature/trusty-t…

d6951ab

…ravis-mods

okorshenko added the PS label Mar 8, 2016

Merge remote-tracking branch 'upstream/develop' into feature/trusty-t…

03e6ac9

…ravis-mods

hshar7 added Issue: Ready for Work Gate 4. Acknowledged. Issue is added to backlog and ready for development improvement labels Mar 8, 2016

davidalger force-pushed the feature/trusty-travis-mods branch from 734db34 to f36c3c7 Compare March 11, 2016 21:47

David Alger added 4 commits April 29, 2016 20:06

Revert "Split static tests out using env var in travis config"

08d34d4

This reverts commit f36c3c7.

Removed stripping of update injection since injection was removed in …

58a3b48

…commit 342f4e6

Corrected quoting in static test call

963273b

davidalger force-pushed the feature/trusty-travis-mods branch 2 times, most recently from 4d927db to 41c7da3 Compare April 30, 2016 02:31

Moved preparation of static test to before_script.sh and added output

1461e02

davidalger force-pushed the feature/trusty-travis-mods branch 2 times, most recently from 3d5db11 to 1d149b5 Compare April 30, 2016 03:33

Move phpunit call into travis.yml now that it can be a one-liner

6552bbb

davidalger force-pushed the feature/trusty-travis-mods branch 6 times, most recently from eed4b38 to 3bba105 Compare April 30, 2016 15:32

David Alger added 2 commits April 30, 2016 10:35

Reorganize a few things and remove the cache dir for composer bin sin…

0be3a79

…ce reason for introduction is now mute

Fix the matrix exclude list

8b18349

davidalger force-pushed the feature/trusty-travis-mods branch from 3bba105 to 8b18349 Compare April 30, 2016 15:36

okorshenko added Progress: accept and removed Progress: needs update labels May 5, 2016

okorshenko self-assigned this May 9, 2016

okorshenko added the in progress label May 9, 2016

magento-team merged commit 8b18349 into magento:develop May 10, 2016

davidalger deleted the feature/trusty-travis-mods branch February 17, 2017 02:46

magento-engcom-team pushed a commit that referenced this pull request Jan 2, 2019

Merge pull request #3567 from magento-tsg-csl3/2.2-develop-pr16

b073545

[TSG-CSL3] For 2.2 (happy New Year ;)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Cleanup difficult to maintain Travis setup and addition of enhancements #3567

Cleanup difficult to maintain Travis setup and addition of enhancements #3567

Uh oh!

davidalger commented Feb 28, 2016

Uh oh!

Flyingmana commented Feb 28, 2016

Uh oh!

davidalger commented Feb 29, 2016

Uh oh!

barryvdh commented Feb 29, 2016

Uh oh!

barryvdh commented Feb 29, 2016

Uh oh!

buskamuza Mar 2, 2016

Uh oh!

davidalger Mar 11, 2016

Uh oh!

buskamuza Mar 11, 2016

Uh oh!

davidalger Mar 11, 2016

Uh oh!

buskamuza Mar 11, 2016

Uh oh!

hshar7 commented Mar 8, 2016

Uh oh!

hshar7 commented Mar 8, 2016

Uh oh!

arkadiych commented Apr 29, 2016

Uh oh!

davidalger commented Apr 30, 2016

Uh oh!

okorshenko commented May 10, 2016

Uh oh!

Uh oh!

Cleanup difficult to maintain Travis setup and addition of enhancements #3567

Cleanup difficult to maintain Travis setup and addition of enhancements #3567

Uh oh!

Conversation

davidalger commented Feb 28, 2016

Uh oh!

Flyingmana commented Feb 28, 2016

Uh oh!

davidalger commented Feb 29, 2016

Uh oh!

barryvdh commented Feb 29, 2016

Uh oh!

barryvdh commented Feb 29, 2016

Uh oh!

buskamuza Mar 2, 2016

Choose a reason for hiding this comment

Uh oh!

davidalger Mar 11, 2016

Choose a reason for hiding this comment

Uh oh!

buskamuza Mar 11, 2016

Choose a reason for hiding this comment

Uh oh!

davidalger Mar 11, 2016

Choose a reason for hiding this comment

Uh oh!

buskamuza Mar 11, 2016

Choose a reason for hiding this comment

Uh oh!

hshar7 commented Mar 8, 2016

Uh oh!

hshar7 commented Mar 8, 2016

Uh oh!

arkadiych commented Apr 29, 2016

Uh oh!

davidalger commented Apr 30, 2016

Uh oh!

okorshenko commented May 10, 2016

Uh oh!

Uh oh!