-
Notifications
You must be signed in to change notification settings - Fork 1.1k
update documentations around test isolation #4797
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 18 commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
06d0eab
CLOUD-295: Add Dashboard CTAs to "Why Cypress -> Features"
debrisapron 6e67656
Address review feedback
debrisapron 5cee97a
Address review feedback
debrisapron aff4a1c
Address review feedback
debrisapron f52dcc9
update experimental warning for session to reflect async/attach chang…
AtofStryker 0d5121c
update documentations around test isolation
emilyrohrbough 22ccb4d
Merge pull request #4284 from cypress-io/content/why-cypress-features…
debrisapron 4b13469
Add note about git metadata issues to AWS CodeBuild page
debrisapron fd6448e
Merge pull request #4804 from cypress-io/content/emphasize-git-metada…
debrisapron 5faab5c
Update content/api/commands/session.md
emilyrohrbough 7942fd9
Update content/api/commands/session.md
emilyrohrbough fb81d64
Update content/guides/core-concepts/writing-and-organizing-tests.md
emilyrohrbough 7c7a654
Apply suggestions from code review
emilyrohrbough 87e71a8
additional doc changes
emilyrohrbough d81aa76
Merge branch 'change-test-isolation' of https://github.com/cypress-io…
emilyrohrbough 7b17ab6
fix markdown
emilyrohrbough 577c2c9
align with recent origin doc changes
emilyrohrbough a661acd
Merge branch 'master' into change-test-isolation
emilyrohrbough 635bed0
remove legacy mode concept & flag test isolation as experimental
emilyrohrbough 2f1a075
Update content/api/commands/session.md
emilyrohrbough dee911b
Update content/api/commands/session.md
emilyrohrbough 0ea202e
Apply suggestions from code review
emilyrohrbough 090f784
Update content/api/commands/session.md
emilyrohrbough 9243bb9
Apply suggestions from code review
emilyrohrbough File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -7,7 +7,13 @@ Cache and restore [cookies](/api/cypress-api/cookies), | |||||
[`localStorage`](https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage), | ||||||
and | ||||||
[`sessionStorage`](https://developer.mozilla.org/en-US/docs/Web/API/Window/sessionStorage) | ||||||
in order to reduce test setup times. | ||||||
(i.e. session data) in order to recreate a consistent browser context between | ||||||
tests. | ||||||
|
||||||
The `cy.session()` command will inherent the | ||||||
[`testIsolation`](/guides/core-concepts/writing-and-organizing-tests#Test-Isolation) | ||||||
mode` value to determine where or not the page is cleared when cacheing and | ||||||
emilyrohrbough marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
restoring the browser context. | ||||||
|
||||||
<Alert type="warning"> | ||||||
|
||||||
|
@@ -24,19 +30,18 @@ Enabling this flag does the following: | |||||
and [`Cypress.session`](/api/cypress-api/session) API. | ||||||
- It adds the following new behaviors (that will be the default in a future | ||||||
major update of Cypress) at the beginning of each test: | ||||||
- The page is cleared (by setting it to `about:blank`). Disable this by | ||||||
setting | ||||||
[`testIsolation=legacy`](/guides/core-concepts/writing-and-organizing-tests#Test-Isolation). | ||||||
- All active session data (cookies, `localStorage` and `sessionStorage`) | ||||||
across all domains are cleared. | ||||||
- It overrides the | ||||||
[`Cypress.Cookies.preserveOnce()`](/api/cypress-api/cookies#Preserve-Once) and | ||||||
- The | ||||||
[`testIsolation`](/guides/core-concepts/writing-and-organizing-tests#Test-Isolation) | ||||||
mode is enhanced from `legacy` mode to `on` mode such that: | ||||||
- The page is cleared (by setting it to `about:blank`). | ||||||
- Cookies, local storage and session storage in all domains are cleared. | ||||||
- It supersedes | ||||||
the [`Cypress.Cookies.preserveOnce()`](/api/cypress-api/cookies#Preserve-Once) and | ||||||
[`Cypress.Cookies.defaults()`](/api/cypress-api/cookies#Defaults) methods. | ||||||
- Cross-origin navigation will no longer fail immediately, but instead, time out | ||||||
based on [`pageLoadTimeout`](/guides/references/configuration#Timeouts). | ||||||
- Tests will no longer wait on page loads before moving on to the next test. | ||||||
- Cross-origin requests will now succeed, however, to interact with a | ||||||
cross-origin page you must use a `cy.origin` block. | ||||||
|
||||||
Because the page is cleared at the beginning of each test, | ||||||
Because the page is cleared at the beginning of each test by default, | ||||||
[`cy.visit()`](/api/commands/visit) must be explicitly called at the beginning | ||||||
of each test. | ||||||
|
||||||
|
@@ -82,7 +87,7 @@ cy.session(name, () => { | |||||
// be done inside the setup function | ||||||
cy.visit('/login') | ||||||
cy.session(name, () => { | ||||||
// need to call cy.visit() here because the page is blank when | ||||||
// need to call cy.visit() here because the page is blank when | ||||||
emilyrohrbough marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
// the setup function runs | ||||||
cy.get('[data-test=name]').type(name) | ||||||
cy.get('[data-test=password]').type('s3cr3t') | ||||||
|
@@ -129,8 +134,11 @@ runs, Cypress will preserve all cookies, `sessionStorage`, and `localStorage`, | |||||
so that subsequent calls to `cy.session()` with the same `id` will bypass | ||||||
`setup` and just restore the cached session data. | ||||||
|
||||||
The page and all active session data (cookies, `localStorage` and | ||||||
`sessionStorage`) across all domains are always cleared before `setup` runs. | ||||||
The page is cleared before `setup` when `testIsolation='on'` and is not cleared | ||||||
when `testIsolation='off'`. | ||||||
|
||||||
Cookies, local storage and session storage in all domains are always cleared | ||||||
before `setup` runs, regardless of the testIsolation configuration. | ||||||
|
||||||
**<Icon name="angle-right"></Icon> options** **_(Object)_** | ||||||
|
||||||
|
@@ -332,10 +340,10 @@ describe('account details', () => { | |||||
|
||||||
### Switching sessions inside tests | ||||||
|
||||||
Because `cy.session()` clears the page and all active session data before | ||||||
running `setup`, you can use it to easily switch between sessions without first | ||||||
needing to log the previous user out. This allows tests to more accurately | ||||||
represent real-world scenarios and helps keep test run times short. | ||||||
Because `cy.session()` clears the page and all session data before running | ||||||
`setup`, you can use it to easily switch between sessions without first needing | ||||||
to log the previous user out. This allows tests to more accurately represent | ||||||
real-world scenarios and helps keep test run times short. | ||||||
|
||||||
```jsx | ||||||
const login = (name) => { | ||||||
|
@@ -670,24 +678,46 @@ it('t3', () => { | |||||
|
||||||
## Notes | ||||||
|
||||||
### When the page and active session data are cleared | ||||||
### When the page and session data are cleared | ||||||
|
||||||
### Test Isolation `on` | ||||||
|
||||||
The page is cleared and cookies, local storage and session storage (session | ||||||
data) in all domains are cleared automatically when `cy.session()` runs and | ||||||
`testIsolation` is `on`. This guarantees consistent behavior whether a session | ||||||
is being created or restored and allows you to switch sessions without first | ||||||
having to explicitly log out. | ||||||
|
||||||
| | Page cleared (test) | Session data cleared | | ||||||
| -------------------------- | :---------------------------------------------: | :---------------------------------------------: | | ||||||
| Before `setup` | <Icon name="check-circle" color="green"></Icon> | <Icon name="check-circle" color="green"></Icon> | | ||||||
| Before `validate` | <Icon name="check-circle" color="green"></Icon> | | | ||||||
| Before `cy.session()` ends | <Icon name="check-circle" color="green"></Icon> | | | ||||||
|
||||||
[`cy.visit()`](/api/commands/visit) must be explicitly called afterwards to | ||||||
ensure the page to test is loaded. | ||||||
|
||||||
### Test Isolation `iff` | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
emilyrohrbough marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
When `testIsolation` is `off`, the page will not clear, however, the session | ||||||
data will clear when `cy.session()` runs. | ||||||
|
||||||
| | Page cleared (test) | Session data cleared | | ||||||
| -------------------------- | :-----------------: | :---------------------------------------------: | | ||||||
| Before `setup` | | <Icon name="check-circle" color="green"></Icon> | | ||||||
| Before `validate` | | | | ||||||
| Before `cy.session()` ends | | | | ||||||
|
||||||
The page is cleared and all active session data (cookies, `localStorage`, and | ||||||
`sessionStorage`) across all domains are cleared automatically when | ||||||
`cy.session()` runs. This guarantees consistent behavior whether a session is | ||||||
being created or restored and allows you to switch sessions without first having | ||||||
to explicitly log out. | ||||||
[`cy.visit()`](/api/commands/visit) does not need to be called afterwards to | ||||||
ensure the page to test is loaded. | ||||||
|
||||||
| | Current page cleared | Active session data cleared | | ||||||
| -------------------- | :---------------------------------------------: | :---------------------------------------------: | | ||||||
| Before `setup` | <Icon name="check-circle" color="green"></Icon> | <Icon name="check-circle" color="green"></Icon> | | ||||||
| Before `validate` | <Icon name="check-circle" color="green"></Icon> | | | ||||||
| After `cy.session()` | <Icon name="check-circle" color="green"></Icon> | | | ||||||
NOTE: Turning test isolation off may improve performance of end-to-end tests, | ||||||
however, previous tests could impact the browser state of the next test and | ||||||
cause inconsistency when using .only(). Be mindful to write isolated tests when | ||||||
test isolation is off. | ||||||
|
||||||
Calling `cy.session()` clears the current page in addition to restoring the | ||||||
cached session data. [`cy.visit()`](/api/commands/visit) must be explicitly | ||||||
called afterwards to ensure a page is visited if you did not provide a | ||||||
`validate` function that called `cy.visit()`. | ||||||
When test isolation is `off`, it is encouraged to setup your session in a before | ||||||
hook or in the first test to ensure a clean setup. | ||||||
|
||||||
### Session caching | ||||||
|
||||||
|
@@ -881,7 +911,7 @@ that were run when creating and/or validating the session. | |||||
In this image, a saved session is restored, but when `/personal` is visited in | ||||||
the `validate` function, the app redirects to `/signin`, which invalidates the | ||||||
session. A new session is created by visiting `/signin` where the user is logged | ||||||
in, after which, validation succeeds, and the session is made active for the | ||||||
in, after which, validation succeeds, and the session is available for the | ||||||
remainder of the test. | ||||||
|
||||||
<DocsImage src="/img/api/session/session-expanded.png" alt="Recreated session (expanded)"></DocsImage> | ||||||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -413,6 +413,21 @@ phases: | |
|
||
## Using the Cypress Dashboard with AWS CodeBuild | ||
|
||
<Alert type="warning"> | ||
|
||
Dashboard analytics are dependent on your CI environment reliably providing | ||
commit SHA data (typically via an environment variable) which is not always | ||
present by default. This is not a problem for most users, but if you are facing | ||
integration issues with your CodeBuild setup, please make sure the git | ||
information is being sent properly by following | ||
[these guidelines](/guides/continuous-integration/introduction#Git-information), | ||
or just see | ||
[the example `codebuild.yml` file at the top of this page](#Basic-Setup). If you | ||
are still facing issues after this, please | ||
[contact us](mailto:[email protected]). | ||
|
||
</Alert> | ||
|
||
In the AWS CodeBuild configuration we have defined in the previous section, we | ||
are leveraging three useful features of the | ||
[Cypress Dashboard](https://on.cypress.io/dashboard): | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.