fix(ember): Restore local and PR tests #4205
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lobsterkatie
force-pushed
the
kmclb-restore-ember-tests-in-ci
branch
from
659acb1 to
d9da1fb
Compare
lobsterkatie
force-pushed
the
kmclb-restore-ember-tests-in-ci
branch
from
d9da1fb to
0d2517d
Compare
size-limit report
|
AbhiPrasad
approved these changes
Dec 1, 2021
AbhiPrasad
pushed a commit
that referenced
this pull request
Dec 1, 2021
Given that ember tests take longer to run than most other tests (and have been known to fail for reasons having nothing to do with whatever PR we're working on), in #3571 we stopped running them locally and on PRs*, instead only running them when merging to `master` or releasing. *Provided the PR didn't touch any of the `@sentry/ember` code As a result, we didn't catch a currently-failing ember test until later than we would have liked. To prevent that from happening again, this restores the running of ember tests (both locally and on PRs), but restricts which tests run in each circumstance, in order to prevent this change from slowing down the overall repo test suite more than necessary. Specifically: - Locally, we run tests only against the current version of ember. - In the context of a PR, we run tests against the current version, the LTS version, and a version using `embroider` (which is an optional ember compiler many people use). - When merging to `master` or creating a release, for greatest safety, we test against current, LTS, embroider, beta, and "classic" (a legacy ember version). Key changes: - Up until now, the logic controlling which set of ember tests to run has been split between a script (`run_tests.js`, which differentiated local runs from CI runs), and the GHA workflow config (`build.yml`, which differentiated PRs from merges into master and creation of releases). That work is now consolidated into the ember-try config (since ember-try is the runner which needs to know which tests to run in the first place!). - As a result of the above change, the method for forcing tests for all versions of ember to run needed to be modified. That option is now handled by a new script, `run-CI-tests.js`, which simply mimics the the third situation above by setting environment variables, thereby tricking ember-try into thinking it needs to run all tests.
onurtemizkan
pushed a commit
that referenced
this pull request
Dec 19, 2021
Given that ember tests take longer to run than most other tests (and have been known to fail for reasons having nothing to do with whatever PR we're working on), in #3571 we stopped running them locally and on PRs*, instead only running them when merging to `master` or releasing. *Provided the PR didn't touch any of the `@sentry/ember` code As a result, we didn't catch a currently-failing ember test until later than we would have liked. To prevent that from happening again, this restores the running of ember tests (both locally and on PRs), but restricts which tests run in each circumstance, in order to prevent this change from slowing down the overall repo test suite more than necessary. Specifically: - Locally, we run tests only against the current version of ember. - In the context of a PR, we run tests against the current version, the LTS version, and a version using `embroider` (which is an optional ember compiler many people use). - When merging to `master` or creating a release, for greatest safety, we test against current, LTS, embroider, beta, and "classic" (a legacy ember version). Key changes: - Up until now, the logic controlling which set of ember tests to run has been split between a script (`run_tests.js`, which differentiated local runs from CI runs), and the GHA workflow config (`build.yml`, which differentiated PRs from merges into master and creation of releases). That work is now consolidated into the ember-try config (since ember-try is the runner which needs to know which tests to run in the first place!). - As a result of the above change, the method for forcing tests for all versions of ember to run needed to be modified. That option is now handled by a new script, `run-CI-tests.js`, which simply mimics the the third situation above by setting environment variables, thereby tricking ember-try into thinking it needs to run all tests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Given that ember tests take longer to run than most other tests (and have been known to fail for reasons having nothing to do with whatever PR we're working on), in #3571 we stopped running them locally and on PRs*, instead only running them when merging to
masteror releasing.*Provided the PR didn't touch any of the
@sentry/embercodeAs a result, we didn't catch a currently-failing ember test until later than we would have liked. To prevent that from happening again, this PR restores the running of ember tests (both locally and on PRs), but restricts which tests run in each circumstance, in order to prevent this change from slowing down the overall repo test suite more than necessary. Specifically:
embroider(which is an optional ember compiler many people use).masteror creating a release, for greatest safety, we test against current, LTS, embroider, beta, and "classic" (a legacy ember version).Key changes:
run_tests.js, which differentiated local runs from CI runs), and the GHA workflow config (build.yml, which differentiated PRs from merges into master and creation of releases). That work is now consolidated into the ember-try config (since ember-try is the runner which needs to know which tests to run in the first place!).run-CI-tests.js, which simply mimics the the third situation above by setting environment variables, thereby tricking ember-try into thinking it needs to run all tests.Note: This is not meant to address the currently-failing ember tests, just make them more visible. Since they're not required, though, this shouldn't be a blocker.