Browser + Device combinations that don't make sense

[nrhoffmann]

This is somewhere between a bug and a feature.

When you put both devices and browsers in your configuration, one test is run for each permutation. But sometimes a permutation doesn't make sense.

Can we do something about this?

[mxschmitt]

We recently have added a defaultBrowserType property in the device list upstream. Will get released with the v1.4 in a few days.
@mmarkelov what do you think?

[mmarkelov]

@mxschmitt is there something that we should change in jest-playwright?

[mxschmitt]

@mmarkelov yeah, see this PR on Playwright: microsoft/playwright#3731

Based on that we can use the defaultBrowserType to decide which browser should be used.

[mmarkelov]

@mxschmitt there is a couple questions. As I can understand there is no need to run tests for browser, device combination? I suppose that there is much work to do if it is right. Also what should we do if we are using for example playwright-chromium package but we got webkit as defaultBrowserType for the device. As workaround I think that we can keep the current logic, but give the user information that he tried to use wrong browser, also we can give the ability to skip this kind of tests and run them only if browser === defaultBrowserType

[mxschmitt]

Regarding the case when the device descriptor says webkit but the user only has installed chromium, then I would raise an error.
Hmm, I would probably handle the devices and browser logic differently. If the user specifies browsers, then loop over them and run the tests on them. But if the users specifies devices, then run them on the devices with the browser which is attached to the device descriptor. If the user specifies both, then thats the question how we should handle that. I would probably run the tests on the browsers without the device options and then on their devices with the attached browser type.

[mmarkelov]

@mxschmitt make sense. But what should we do for the previous playwright versions. defaultBrowserType won't be able to use. Should we ask user to update playwright or should we try to run tests in some other way?

[mxschmitt]

@mxschmitt make sense. But what should we do for the previous playwright versions. defaultBrowserType won't be able to use. Should we ask user to update playwright or should we try to run tests in some other way?

I personally would use chromium by default if the defaultBrowserType field is not persistent and print a warning that 1.4+ is required.

[mmarkelov]

Finally pushed #351 to master and published new rc version. So I added an option useDefaultBrowserType to jest-playwright.config.

// jest-playwright.config.js
module.exports = {
        //...,
	devices: ['iPhone 6', 'Pixel 2', 'iPhone 8'],
	useDefaultBrowserType: true,
}

It will run tests with defaultBrowserType of device

[ValeryVS]

@mmarkelov
Seems there is a bug, when custom devices are added to configuration.
#464