Setting max-sessions for Grid node doesn't work correctly

[JohnA2]

🐛 Bug Report

I want to start a Grid node with a single type of browser and a fixed number of maximum sessions/instances, for example 7. I can do it fine with Selenium 3 like this:

java -Dwebdriver.gecko.driver=drivers\geckodriver.exe -jar selenium-server-standalone-3.14.0.jar -role node -maxSession 7 -browser browserName="firefox",maxInstances=7

But I couldn't achieve it with Selenium 4 using a similar (per my understanding of the documentation) command:

java -Dwebdriver.gecko.driver=drivers\geckodriver.exe -jar selenium-server-4.0.0-beta-2.jar node --detect-drivers false --max-sessions 7 --driver-configuration name="Firefox" max-sessions=7 stereotype={\"browserName\":\"firefox\"}

To Reproduce

1. Start the hub using java -jar selenium-server-4.0.0-beta-2.jar hub
2. Run the above command to start the node

Expected behavior

A Firefox node that can run up to 7 Selenium tests in parallel.

Actual behavior

I see that Firefox was added 2x5 times instead of 7 in the log:

09:11:07.748 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
09:11:07.764 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
09:11:08.273 INFO [UnboundZmqEventBus.<init>] - Connecting to tcp://*:4442 and tcp://*:4443
09:11:08.422 INFO [UnboundZmqEventBus.<init>] - Sockets created
09:11:08.538 INFO [UnboundZmqEventBus.<init>] - Event bus ready
09:11:08.943 INFO [NodeServer.createHandlers] - Reporting self as: http://x.x.x.x:5555
09:11:09.347 INFO [NodeOptions.report] - Adding Firefox for {"browserName": "firefox","platformName": "XP"} 5 times
09:11:09.350 INFO [NodeOptions.report] - Adding Firefox for {"browserName": "firefox","platformName": "XP"} 5 times
09:11:09.420 INFO [Node.<init>] - Binding additional locator mechanisms: name, id
09:11:11.615 INFO [NodeServer$1.start] - Starting registration process for node id 36a5fd26-da4e-4753-9135-e1f689c6f5e4
09:11:11.620 INFO [NodeServer.execute] - Started Selenium node 4.0.0-beta-2 (revision Unknown): http://x.x.x.x:5555
09:11:11.673 INFO [NodeServer$1.lambda$start$1] - Sending registration event...
09:11:12.362 INFO [NodeServer.lambda$createHandlers$1] - Node has been added

I also see "Stereotypes: 10" and "Max. Concurrency: 4" in the Grid web console.

Environment

OS: Windows (any version)
Browser: any
Browser version: any
Browser Driver version: latest
Language Bindings version: 4.0.0-beta-2
Selenium Grid version (if applicable): 4.0.0-beta-2

[diemol]

A change was made to allow this use case from now on.

The initial concept was to allow a max amount of concurrent sessions to be the # of available processors. The idea behind this is to guarantee a more stable Grid by letting each browser session use one processor. This rule of thumb brings more stable Grids and therefore more stable tests.

However, we have gotten the feedback through different issues where users, in some cases, want to have a higher value for max concurrent sessions. This change allows that, it is possible to set a value in --max-sessions higher than the avialble processors, but to make this more explicit, the --override-max-sessions needs to be set to true (which declares the explicit intent of the user to override this value, and it is assumed the user is aware of potential issues by running a high number of sessions in a reduced amount of processors).

This change will be available in beta-3. We also do docker-selenium prereleases, so a jar with this change will be available here in the upcoming days.

[JohnA2]

Thanks! I've tested it by recompiling beta-2 with the changes. Now it works correctly if I start the node like this
java -Dwebdriver.gecko.driver=drivers\geckodriver.exe -jar selenium-server-4.0.0-beta-2.jar node --max-sessions 7 --override-max-sessions true
However there are 2 issues.

1. It still doesn't work with IE. I always get only one session with it. I know that IE should be limited to only 1 simultaneous instance in some cases, but not all, see http://jimevansmusic.blogspot.com/2013/01/revisiting-native-events-in-ie-driver.html We used multiple instances of IE for years with Selenium Grid 2 and 3 without issues, so it's definitely possible.

2. If I use a command like
   java -Dwebdriver.gecko.driver=drivers\geckodriver.exe -jar selenium-server-4.0.0-beta-2.jar node --detect-drivers false --max-sessions 7 --override-max-sessions true --driver-configuration name="Firefox" max-sessions=7 stereotype={\"browserName\":\"firefox\"}
   or
   java -Dwebdriver.gecko.driver=drivers\geckodriver.exe -jar selenium-server-4.0.0-beta-2.jar node --detect-drivers false --override-max-sessions true --driver-configuration name="Firefox" max-sessions=7 stereotype={\"browserName\":\"firefox\"}
   (I'm not sure if one of them is incorrect) it adds the specified number of sessions twice, so I get 14 instead of 7.

[diemol]

@JohnA2, thanks for the feedback! There was indeed a small bug when using the cli options that was duplicating the entries for the --driver-configuration option. However, it was not related to this change. I have a fix that I am currently testing and I will push soon.

About point number 1, I know some users have been successful with concurrent sessions in IE, but limiting the number of sessions to one will bring an overall stability when using IE. If you go through the issues, there are 2-3 open ones where users are bumping into IE issues when running sessions in parallel, and troubleshooting those issues is almost impossible since they depend a lot how each user is doing it.

[JohnA2]

@diemol I agree that concurrent sessions in IE can cause issues in some cases. But I think it would be wrong to limit all users only because of some cases. As I mention, for us it's been working fine for years. Also Selenium 2 and 3 allow to specify multiple IE sessions, so completely removing that ability in Selenium 4 would be a regression from the user perspective.

Maybe you can use the same --override-max-sessions option to remove the limit for IE? If the user specifies the option, it means that they do it at their own risk and can't complain if something breaks. I think that adding a warning to not create an issue here when using that option with IE should resolve the problem that you mentioned.