diff --git a/content/_changelogs/12.0.0.md b/content/_changelogs/12.0.0.md index 8d98cee796..58151442cb 100644 --- a/content/_changelogs/12.0.0.md +++ b/content/_changelogs/12.0.0.md @@ -1,4 +1,4 @@ -## 11.0.0 +## 12.0.0 _Released MM/DD/YYYY_ @@ -6,8 +6,15 @@ _Released MM/DD/YYYY_ - `experimentalSessionAndOrigin` flag has been removed and all functionality is on by default now. -- `testIsolation` defaults to `strict` now. +- The new concept of `testIsolation` has been introduced and it defaults to + `on`. +- Cypress now throws an error if commands are invoked from inside a `.should()` + callback. This previously resulted in unusual and undefined behavior; it is + now explicitly an error. - `Cookies.defaults` and `Cookies.preserveOnce` have been removed. Please update to use [`cy.session()`](/api/commands/session) to preserve session details between tests. Addresses [#21472](https://github.com/cypress-io/cypress/issues/21472). +- The` cy.server()` and` cy.route()` commands and the `Cypress.Server.defaults` + API has been removed. Use [`cy.intercept()`(/api/commands/intercept) instead. + Addressed in [#24411](https://github.com/cypress-io/cypress/pull/24411). diff --git a/content/guides/core-concepts/writing-and-organizing-tests.md b/content/guides/core-concepts/writing-and-organizing-tests.md index 511142e903..587305d97b 100644 --- a/content/guides/core-concepts/writing-and-organizing-tests.md +++ b/content/guides/core-concepts/writing-and-organizing-tests.md @@ -17,9 +17,9 @@ title: Writing and Organizing Tests Best Practices -We recently gave a "Best Practices" conference talk at AssertJS (February 2018). -This video demonstrates how to approach breaking down your application and -organizing your tests. +We gave a "Best Practices" conference talk at AssertJS (February 2018). This +video demonstrates how to approach breaking down your application and organizing +your tests. [https://www.youtube.com/watch?v=5XQOK0v_YRE](https://www.youtube.com/watch?v=5XQOK0v_YRE) @@ -564,17 +564,6 @@ it.skip('returns "fizz" when number is multiple of 3', () => { ### Test Isolation - - - -Experimental - -The concept of test isolation is currently experimental, and can be enabled by -setting the [`experimentalSessionAndOrigin`](/guides/references/experiments) -option to `true` in the Cypress config. - - - **Best Practice:** Tests should @@ -619,19 +608,8 @@ and `off`. ###### On Mode - - - -Experimental - -`on` mode is currently experimental and can be enabled by setting -the [`experimentalSessionAndOrigin`](/guides/references/experiments) flag -to `true` in the Cypress config. This is the default test isolation behavior -when using the `experimentalSessionAndOrigin` experiment. - - - -When in `on` mode, Cypress resets the browser context _before_ each test by: +This is the default test isolation behavior in Cypress. When in `on` mode, +Cypress resets the browser context _before_ each test by: - clearing the dom state by visiting `about:blank` - clearing [cookies](/api/cypress-api/cookies) in all domains @@ -652,28 +630,18 @@ is so tests can reliably pass when run standalone or in a randomized order. ###### Off Mode - - - -Experimental - -`off` mode is currently experimental and can be enabled by setting -the [`experimentalSessionAndOrigin`](/guides/references/experiments) flag -to `true` in the Cypress config. - - - When in `off` mode, Cypress will not alter the browser context before the test starts. The page does not clear between tests and cookies, local storage and session storage will be available across tests in that suite. Additionally, the `cy.session()` command will only clear the current browser context when establishing the browser session - the current page will not clear. -It is important to note that turning test isolation `off` may improve the -overall performance of end-to-end tests, however, previous tests could be impact -the browser state. It is important to be extremely mindful of how test are -written when using this mode and ensure tests continue to run independent from -one other. +It is important to note that while turning test isolation `off` may improve the +overall performance of end-to-end tests, it can however cause state to "leak" +between tests. This can make later tests dependent on the results of earlier +tests, and potentially cause misleading test failures. It is important to be +extremely mindful of how tests are written when using this mode, and ensure that +tests continue to run independently of one another. ###### Mode Comparison @@ -686,13 +654,16 @@ one other. Cypress only support testIsolation `on` in component testing. -When running component tests, the browser context will allow start in a clean -slate because Cypress will +When running component tests, Cypress resets the browser context _before_ each +test by: -- clear the page -- clear [cookies](/api/cypress-api/cookies) -- clear +- clearing the page +- clearing [cookies](/api/cypress-api/cookies) +- clearing [`localStorage`](https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage) +- clearing + [`sessionStorage`](https://developer.mozilla.org/en-US/docs/Web/API/Window/sessionStorage) + in all domains ### Test Configuration diff --git a/content/guides/references/migration-guide.md b/content/guides/references/migration-guide.md index 2ef6fcb79c..101f5b590f 100644 --- a/content/guides/references/migration-guide.md +++ b/content/guides/references/migration-guide.md @@ -2,7 +2,362 @@ title: Migration Guide --- -## Migrating to Cypress version 11.0 +## Migrating to Cypress 12.0 + +This guide details the changes and how to change your code to migrate to Cypress +version 12.0. +[See the full changelog for version 12.0](/guides/references/changelog#12-0-0). + +The Session and Origin experiment has been released as General Availability +(GA), meaning that we have deemed this experiment to be feature complete and +free of issues in the majority of use cases. With releasing this as GA, the +`experimentalSessionAndOrigin` flag has been removed, the +[`cy.origin()`](<(/api/commands/origin)>) and +[`cy.session()`](/api/commands/session) commands are generally available and the +concept of +[Test Isolation](/guides/core-concepts/writing-and-organizing-tests#Test-Isolation) +has been introduced. + +### Node.js 14+ support + +Cypress comes bundled with its own +[Node.js version](https://github.com/cypress-io/cypress/blob/develop/.node-version). +However, installing the `cypress` npm package uses the Node.js version installed +on your system. + +Node.js 12 reached its end of life on April 30, 2022. +[See Node's release schedule](https://github.com/nodejs/Release). This Node.js +version will no longer be supported when installing Cypress. The minimum Node.js +version supported to install Cypress is Node.js 14+. + +### Test Isolation + +The +[`testIsolation`](/guides/core-concepts/writing-and-organizing-tests#Test-Isolation) +config option defaults to `on`. This means Cypress resets the browser context +_before_ each test by: + +- clearing the dom state by visiting `about:blank` +- clearing [cookies](/api/cypress-api/cookies) in all domains +- clearing + [`localStorage`](https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage) + in all domains +- clearing + [`sessionStorage`](https://developer.mozilla.org/en-US/docs/Web/API/Window/sessionStorage) + in all domains + +Test suites that relied on the application to persist between tests may have to +be updated to revisit their application and rebuild the browser state for each +test that needs it. + +Before this change, it was possible to write tests such that you could rely on +the application (i.e. DOM state) to persist between tests. For example you could +log in to a CMS in the first test, change some content in the second test, +verify the new version is displayed on a different URL in the third, and log out +in the fourth. + +Here's a simplified example of such a test strategy. + +Before Multiple small tests against different +origins + +```js +it('logs in', () => { + cy.visit('https://supersecurelogons.com') + cy.get('input#password').type('Password123!') + cy.get('button#submit').click() +}) + +it('updates the content', () => { + // already on page redirect from clicking button#submit + cy.get('#current-user').contains('logged in') + cy.get('button#edit-1').click() + cy.get('input#title').type('Updated title') + cy.get('button#submit').click() + cy.get('.toast').type('Changes saved!') +}) + +it('validates the change', () => { + cy.visit('/items/1') + cy.get('h1').contains('Updated title') +}) +``` + +After migrating, when `testIsolation='on'`, this flow would need to be contained +within a single test. While the above practice has always been +[discouraged](/guides/references/best-practices#Having-tests-rely-on-the-state-of-previous-tests) +we know some users have historically written tests this way, often to get around +the `same-origin` restrictions. But with [`cy.origin()`](/api/commands/origin) +you no longer need these kind of brittle hacks, as your multi-origin logic can +all reside in a single test, like the following. + +After One big test using `cy.origin()` + +```js +it('securely edits content', () => { + cy.origin('supersecurelogons.com', () => { + cy.visit('https://supersecurelogons.com') + cy.get('input#password').type('Password123!') + cy.get('button#submit').click() + }) + + cy.origin('mycms.com', () => { + cy.url().should('contain', 'cms') + cy.get('#current-user').contains('logged in') + cy.get('button#edit-1').click() + cy.get('input#title').type('Updated title') + cy.get('button#submit').click() + cy.get('.toast').type('Changes saved!') + }) + + cy.visit('/items/1') + cy.get('h1').contains('Updated title') +}) +``` + +The just-released `cy.session()` command can be used to setup and cache cookies, +local storage and session storage between tests to easily re-establish the +previous (or common) browser contexts needed in a suite. This command will run +setup on its initial execution and will restore the saved browser state on each +sequential command execution. This command reduces the need for repeated +application logins, while users also benefit from the test isolation guardrails +to write independent, reliable and deterministic tests from the start. + +If for whatever reason you still need to persist the dom and browser context +between tests, you can set `testIsolation='off` on the root configuration or at +the suite-level. For example: + +```js +describe('workflow', { testIsolation: 'off' }, () => { + ... +}) +``` + +It is important to note that while turning test isolation `off` may improve the +overall performance of end-to-end tests, it can however cause state to "leak" +between tests. This can make later tests dependent on the results of earlier +tests, and potentially cause misleading test failures. It is important to be +extremely mindful of how tests are written when using this mode, and ensure that +tests continue to run independently of one another. + +For examplethe following tests are not independent +nor deterministic: + +```js +describe('workflow', { testIsolation: 'off' }, () => { + it('logs in', () => { + cy.visit('my-app.com/log-in) + cy.get('username').type('User1') + cy.get('password').type(Cypress.env('User1_password')) + cy.get('button#login').click() + cy.contains('User1') + }) + + it('clicks user profile', () => { + cy.get('User1').find('#profile_avatar).click() + cy.contains('Email Preferences') + }) + + it('updates profile', () => { + cy.get('button#edit') + cy.get('email').type('user1@email.com') + cy.get('button#save').click() + }) +}) +``` + +In the above example, each test is relying on the previous test to be +_successful_ to correctly execute. If at any point, the first or second test +fails, the sequential test(s) will automatically fail and provide unreliable +debugging errors since the errors are representative of the previous test. + +The best way to ensure your tests are independent is to add a `.only()` to your +test and verify it can run successfully without the test before it. + +### Behavior Changes in Alias Resolution + +Cypress always re-queries aliases when they are referenced. This can result in +certain tests that used to pass could start to fail. For example, + +```js +cy.findByTestId('popover') + .findByRole('button', { expanded: true }) + .as('button') + .click() + +cy.get('@button').should('have.attr', 'aria-expanded', 'false') +``` + +previously passed, because the initial button was collapsed when first queried, +and then later expanded. However, in Cypress 12, this test fails because the +alias is always re-queried from the DOM, effectively resulting in the following +execution: + +```js +cy.findByTestId('popover').findByRole('button', { expanded: true }).click() + +cy.findByTestId('popover') + .findByRole('button', { expanded: true }) // A button which matches here (is expanded)... + .should('have.attr', 'aria-expanded', 'false') // ...will never pass this assertion. +``` + +You can rewrite tests like this to be more specific; in our case, we changed the +alias to be the first button rather than the unexpanded button. + +```js +cy.findByTestId('popover').findAllByRole('button').first().as('button') +``` + +### Command / Cypress API Changes + +#### `Cypress.Cookies.defaults` and `Cypress.Cookies.preserveOnce` + +The `Cypress.Cookies.defaults` and `CypressCookies.preserveOnce` APIs been +removed. Use the [`cy.session()`](/api/commands/session) command to preserve +cookies (and local and session storage) between tests. + +```diff +describe('Dashboard', () => { + beforeEach(() => { +- cy.login() +- Cypress.Cookies.preserveOnce('session_id', 'remember_token') ++ cy.session('unique_identifier', cy.login, { ++ validate () { ++ cy.getCookies().should('have.length', 2) ++ }, ++ cacheAcrossSpecs: true ++ }) + }) +``` + +#### `cy.server()`, `cy.route()` and `Cypress.Server.defaults` + +The` cy.server()` and` cy.route()` commands and the `Cypress.server.defaults` +API has been removed. Use the [`cy.intercept()`](/api/commands/intercept) +command instead. + +```diff + it('can encode + decode headers', () => { +- Cypress.Server.defaults({ +- delay: 500, +- method: 'GET', +- }) +- cy.server() +- cy.route(/api/, () => { +- return { +- 'test': 'We’ll', +- } +- }).as('getApi') ++ cy.intercept('GET', /api/, (req) => { ++ req.on('response', (res) => { ++ res.setDelay(500) ++ }) ++ req.body.'test': 'We’ll' ++ }).as('getApi') + cy.visit('/index.html') + cy.window().then((win) => { + const xhr = new win.XMLHttpRequest + xhr.open('GET', '/api/v1/foo/bar?a=42') + xhr.send() + }) + + cy.wait('@getApi') +- .its('url').should('include', 'api/v1') ++ .its('request.url').should('include', 'api/v1') + }) +``` + +#### `.invoke()` + +[`.invoke()`](/api/commands/invoke) now throws an error if the function returns +a promise. If you wish to call a method that returns a promise and wait for it +to resolve, use [`.then()`](/api/commands/then) instead of `.invoke()`. + +```diff +cy.wrap(myAPI) +- .invoke('makeARequest', 'http://example.com') ++ .then(api => api.makeARequest('http://example.com')) + .then(res => { ...handle response... }) +``` + +If `.invoke()` is followed by additional commands or assertions, it will call +the named function multiple times. This has the benefit that the chained +assertions can more reliably use the function's return value. + +If this behavior is undesirable because you expect the function to be invoked +only once, break the command chain and move the chained commands and/or +assertions to their own chain. For example, rewrite + +```diff +- cy.get('input').invoke('val', 'text').type('newText') ++ cy.get('input').invoke('val', 'text') ++ cy.get('input').type('newText') +``` + +#### `.should()` + +[`.should()`](/api/commands/should) now throws an error if Cypress commands are +invoked from inside a `.should()` callback. This previously resulted in unusual +and undefined behavior. If you wish to execute a series of commands on the +yielded value, use`.then()` instead. + +```diff +cy.get('button') +- .should(($button) => { + + }) ++ .then(api => api.makeARequest('http://example.com')) + .then(res => { ...handle response... }) +``` + +#### `Cypress.Commands.overwrite()` + +In Cypress 12.0.0, we introduced a new command type, called queries. A query is +a small and fast command for getting data from the window or DOM. This +distinction is important because Cypress can retry chains of queries, keeping +the yielded subject up-to-date as a page rerenders. + +With the introduction of query commands, the following commands have been +re-categorized and can no longer be overwritten with +[`Cypress.Commands.overwrite()`](api/cypress-api/custom-commands#Overwrite-Existing-Commands): + +- [`.as()`](/api/commands/as) +- [`.children()`](/api/commands/children) +- [`.closest()`](/api/commands/closest) +- [`.contains()`](/api/commands/contains) +- [`cy.debug()`](/api/commands/debug) +- [`cy.document()`](/api/commands/document) +- [`.eq()`](/api/commands/eq) +- [`.filter()`](/api/commands/filter) +- [`.find()`](/api/commands/find) +- [`.first()`](/api/commands/first) +- [`.focused()`](/api/commands/focused) +- [`.get()`](/api/commands/get) +- [`.hash()`](/api/commands/hash) +- [`.its()`](/api/commands/its) +- [`.last()`](/api/commands/last) +- [`cy.location()`](/api/commands/location) +- [`.next()`](/api/commands/next) +- [`.nextAll()`](/api/commands/nextall) +- [`.not()`](/api/commands/not) +- [`.parent()`](/api/commands/parent) +- [`.parents()`](/api/commands/parents) +- [`.parentsUntil()`](/api/commands/parentsuntil) +- [`.prev()`](/api/commands/prev) +- [`.prevUntil()`](/api/commands/prevuntil) +- [`cy.root()`](/api/commands/root) +- [`.shadow()`](/api/commands/shadow) +- [`.siblings()`](/api/commands/siblings) +- [`cy.title()`](/api/commands/title) +- [`cy.url()`](/api/commands/url) +- [`cy.window()`](/api/commands/window) + +If you were previously overwriting one of the above commands, try adding your +version as a new command using +[`Cypress.Commands.add()`](api/cypress-api/custom-commands) under a different +name. + +## Migrating to Cypress 11.0 This guide details the changes and how to change your code to migrate to Cypress version 11.0. @@ -268,81 +623,7 @@ export default defineConfig({ Vite 3+ users could make use of the [`mergeConfig`](https://vitejs.dev/guide/api-javascript.html#mergeconfig) API. -## Migrating to Cypress version 11.0 - -This guide details the changes and how to change your code to migrate to Cypress -version 11.0. -[See the full changelog for version 11.0](/guides/references/changelog#11-0-0). - -### Test Isolation - -The `testIsolation` config option defaults to `strict`. This means that after -every test, the current page is reset to `about:blank` and all active session -data (cookies, `localStorage` and `sessionStorage`) across all domains are -cleared. Some test suites that rely on the previous behavior may have to be -updated. - -Before this change, it was possible to write tests such that you could, for -example, log in to a CMS in the first test, change some content in the second -test, verify the new version is displayed on a different URL in the third, and -log out in the fourth. Here's a simplified example of such a test strategy. - -Before Multiple small tests against different -origins - -```js -it('logs in', () => { - cy.visit('https://supersecurelogons.com') - cy.get('input#password').type('Password123!') - cy.get('button#submit').click() -}) -it('updates the content', () => { - cy.get('#current-user').contains('logged in') - cy.get('button#edit-1').click() - cy.get('input#title').type('Updated title') - cy.get('button#submit').click() - cy.get('.toast').type('Changes saved!') -}) -it('validates the change', () => { - cy.visit('/items/1') - cy.get('h1').contains('Updated title') -}) -``` - -After migrating, this flow would need to be contained within a single test. -While the above practice has always been -[discouraged](/guides/references/best-practices#Having-tests-rely-on-the-state-of-previous-tests) -we know some users have historically written tests this way, often to get around -the same-origin restrictions. But with `cy.origin()` you no longer need these -kind of brittle hacks, as your multi-origin logic can all reside in a single -test, like the following. - -After One big test using `cy.origin()` - -```js -it('securely edits content', () => { - cy.origin('supersecurelogons.com', () => { - cy.visit('https://supersecurelogons.com') - cy.get('input#password').type('Password123!') - cy.get('button#submit').click() - }) - cy.origin('mycms.com', () => { - cy.url().should('contain', 'cms') - cy.get('#current-user').contains('logged in') - cy.get('button#edit-1').click() - cy.get('input#title').type('Updated title') - cy.get('button#submit').click() - cy.get('.toast').type('Changes saved!') - }) - cy.visit('/items/1') - cy.get('h1').contains('Updated title') -}) -``` - -Always remember, -[Cypress tests are not unit tests](https://docs.cypress.io/guides/references/best-practices#Creating-tiny-tests-with-a-single-assertion). - -## Migrating to Cypress version 10.0 +## Migrating to Cypress 10.0 This guide details the changes and how to change your code to migrate to Cypress version 10.0.