DRIVERS-3282: Introduce optin setting to await for MinPoolSize population #1834

sanych-sun · 2025-09-05T23:38:18Z

Some CSOT tests are require pre-heat connection pool to have a predictable test outcome. The suggestion is to use minPoolSize together with new setting awaitMinPoolSize that makes unified test runner to await until pool will be populated before run the actual test case.

Please complete the following before merging:

Update changelog.
Test changes in at least one language driver.
Test these changes against all server versions and topologies (including standalone, replica set, and sharded
clusters).

sanych-sun · 2025-09-05T23:57:27Z

source/client-side-operations-timeout/tests/global-timeoutMS.yml

@@ -2,7 +2,7 @@

 description: "timeoutMS can be configured on a MongoClient"

-schemaVersion: "1.9"
+schemaVersion: "1.26"


@prestonvasquez I've updated this file only with the new approach to pre-heat the pool. Should we apply similar settings to more tests?

Suggest updating the runCursorCommand "Non-tailable cursor lifetime remaining timeoutMS applied to getMore if timeoutMode is unset" test, which would close DRIVERS-3134 (PR https://github.com/mongodb/specifications/pull/1769/files).

Otherwise, I will collect a list of candidates during Go Driver's second implementation.

I'll update the test and list of candidates would be much appreciated.

I think we should add this functionality to any CSOT test that (1) doesn't require threads,(2) relies on a failpoint, and (3) uses timeoutMS either as an operation argument or on the client. The following seem relevant:

retryability-timeoutMS.yml

command-execution.jyml

non-tailable-cursors.yml

error-transformations.yml

convenient-transactions.yml

sessions-inherit-timeoutMS.yml

prestonvasquez · 2025-09-10T19:36:03Z

source/client-side-operations-timeout/tests/global-timeoutMS.yml

@@ -2,7 +2,7 @@

 description: "timeoutMS can be configured on a MongoClient"

-schemaVersion: "1.9"
+schemaVersion: "1.26"


Suggest updating the runCursorCommand "Non-tailable cursor lifetime remaining timeoutMS applied to getMore if timeoutMode is unset" test, which would close DRIVERS-3134 (PR https://github.com/mongodb/specifications/pull/1769/files).

Otherwise, I will collect a list of candidates during Go Driver's second implementation.

prestonvasquez · 2025-09-10T22:02:03Z

source/unified-test-format/unified-test-format.md

+    - `awaitMinPoolSize`: Optional boolean. If `true`, the unified spec runner must wait for the connection pool to be
+        populated for all servers according to the `minPoolSize` option. If `false`, not specified, or if minPoolSize
+        equals 0, there is no need to wait for any specific pool state.


Drivers that asynchronously construct the pool size with minPoolSize after client construction could make this check block indefinitely. We should prescribe an appropriate amount of time to await warming the connection, perhaps 500ms? From experimentation, actual connection time seems to be around 50-200ms on local networks.

Let's put limit to 1 second, we can tune it in future if needed. In case if this is not enough, should we throw and report test as failed?

Sounds good to me, thank you

Is there a downside to setting a higher limit? Worst case, the tests takes a bit longer to fail. Best case, we avoid some flakiness and modifications of these tests if 1s proves to be too short.

We could give a recommended timeoutMS based on the 20-500ms observation for 8.1 and let drivers update as needed. Does that seem reasonable? The important distinction is that we don't block indefinitely listening to the event monitor.

I don't feel strongly but I do think that if we're debating "is the timeout high enough" and there aren't downsides of setting it higher, we should just set it higher now to avoid test updates and test syncs in the future if the timeout value doesn't end up being high enough.

To make it resilient we might want to use formula like: waitLimit = minPoolSize * 10s. This way we will automatically extent the limit if there should be more connection to establish.
@prestonvasquez @baileympearson WDYT? Should I add this to spec?

@baileympearson I've updated text to wait for minPoolSize * 10s. I'm positive if connection could not be established during 10seconds - then we really have a problem, and there is no sense to wait any more. Does it sounds as reliable solution?

Works for me. 10s seems plenty.

We could specify the value in the spec test. For example, instead of an awaitMinPoolSize boolean, we could add an awaitMinPoolSizeMS integer.

E.g. to wait for 10 seconds:

awaitMinPoolSizeMS: 10000

prestonvasquez · 2025-09-10T23:00:47Z

source/client-side-operations-timeout/tests/global-timeoutMS.yml

@@ -2,7 +2,7 @@

 description: "timeoutMS can be configured on a MongoClient"

-schemaVersion: "1.9"
+schemaVersion: "1.26"


I think we should add this functionality to any CSOT test that (1) doesn't require threads,(2) relies on a failpoint, and (3) uses timeoutMS either as an operation argument or on the client. The following seem relevant:

retryability-timeoutMS.yml

command-execution.jyml

non-tailable-cursors.yml

error-transformations.yml

convenient-transactions.yml

sessions-inherit-timeoutMS.yml

matthewdale

Looks good! 👍

Parental leave

ShaneHarvey · 2025-09-15T23:21:40Z

source/unified-test-format/unified-test-format.md

+        connections. Only connection pools for data-bearing nodes should be considered. If any connection pool is not
+        populated within the specified timeframe, the Unified Spec Runner must raise an error and mark the test as failed.
+        If the parameter is omitted or if `minPoolSize` is set to 0, no waiting is required for a specific pool state
+        prior to test execution.


Do we need to mention the behavior of CMAP and SDAM events/logs that occur before the pool is populated? Is the intention that those events will show up or will they be ignored?

Ignoring of any events mentioned in the changes.
Thanks

jmikola

Two small suggestions on the spec file. I've not reviewed the test files as I'd be unable to POC those anyway.

jmikola · 2025-09-16T20:12:58Z

source/unified-test-format/unified-test-format.md

+    <span id="entity_client_awaitMinPoolSizeMS"></span>
+
+    - `awaitMinPoolSizeMS`: Optional, integer. When specified, this parameter defines the maximum duration (in
+        milliseconds) that the Unified Spec Runner must wait for each connection pool to be populated with `minPoolSize`


I don't recall ever using this term as a proper noun. I think you can just write "test runner" for consistency with other text.

Likewise on the line below.

jmikola · 2025-09-16T20:17:05Z

source/unified-test-format/unified-test-format.md

+        connections. Only connection pools for data-bearing nodes should be considered. If any connection pool is not
+        populated within the specified timeframe, the Unified Spec Runner must raise an error and mark the test as failed.
+        Any CMAP and SDAM events/logs listeners configured on the client should ignore any events that occur before the
+        pool is being populated. If the parameter is omitted or if `minPoolSize` is set to 0, no waiting is required for a


This can clarify that you're referring to the minPoolSize URI option; lest readers think it's just another client entity option.

…tion

sanych-sun requested review from a team as code owners September 5, 2025 23:38

sanych-sun requested review from jmikola and removed request for a team September 5, 2025 23:38

sanych-sun commented Sep 5, 2025

View reviewed changes

sanych-sun requested a review from prestonvasquez September 8, 2025 22:33

prestonvasquez requested changes Sep 10, 2025

View reviewed changes

prestonvasquez previously requested changes Sep 10, 2025

View reviewed changes

prestonvasquez mentioned this pull request Sep 11, 2025

GODRIVER-3659 Add awaitMinPoolSize to clientEntity constructor mongodb/mongo-go-driver#2196

Merged

sanych-sun requested review from prestonvasquez and matthewdale and removed request for prestonvasquez September 12, 2025 18:08

matthewdale approved these changes Sep 12, 2025

View reviewed changes

sanych-sun removed the request for review from prestonvasquez September 15, 2025 17:10

ShaneHarvey reviewed Sep 15, 2025

View reviewed changes

sanych-sun force-pushed the drivers3282 branch from 981fbec to f337745 Compare September 16, 2025 18:42

jmikola approved these changes Sep 16, 2025

View reviewed changes

DRIVERS-3282: Introduce optin setting to await for MinPoolSize popula…

b6a1b62

…tion

sanych-sun force-pushed the drivers3282 branch from 4c87976 to b6a1b62 Compare September 16, 2025 20:51

sanych-sun merged commit 4244306 into mongodb:master Sep 16, 2025
6 checks passed

sanych-sun deleted the drivers3282 branch September 16, 2025 20:56

DRIVERS-3282: Introduce optin setting to await for MinPoolSize population #1834

DRIVERS-3282: Introduce optin setting to await for MinPoolSize population #1834

Conversation

sanych-sun commented Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prestonvasquez Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prestonvasquez Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

baileympearson Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanych-sun Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prestonvasquez Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

matthewdale left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmikola left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sanych-sun commented Sep 5, 2025 •

edited

Loading

prestonvasquez Sep 10, 2025 •

edited

Loading

prestonvasquez Sep 10, 2025 •

edited

Loading

baileympearson Sep 11, 2025 •

edited

Loading

sanych-sun Sep 11, 2025 •

edited

Loading

prestonvasquez Sep 10, 2025 •

edited

Loading