From 4197c9b1e9c4081a2003e3af78c115d9b3cc7412 Mon Sep 17 00:00:00 2001 From: Boomtime Date: Fri, 31 Jan 2014 10:28:38 +1100 Subject: [PATCH 1/6] Update orderby.txt cursor.sort() contains a warning about the 32MB limit on non-indexed sorting, this adds the same warning to $orderby - which is the underlying operator that powers sort(). The warning is largely a copy+paste with superficial name replacements. Signed-off-by: kay --- source/reference/operator/meta/orderby.txt | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/source/reference/operator/meta/orderby.txt b/source/reference/operator/meta/orderby.txt index 3339e098dd9..83407b94b51 100644 --- a/source/reference/operator/meta/orderby.txt +++ b/source/reference/operator/meta/orderby.txt @@ -35,3 +35,12 @@ $orderby large in-memory sort. The :method:`cursor.limit()` increases the speed and reduces the amount of memory required to return this query by way of an optimized algorithm. + +.. warning:: + + The sort function requires that the entire sort be able to + complete within 32 megabytes. When the sort option consumes more + than 32 megabytes, MongoDB will return an error. Use :operator:`$maxScan` + and/or :method:`~cursor.limit()`, or create an index on the field that you're + sorting to avoid this error. + From ec5f94a0c55cdf176c016e8be845e7aa4643b276 Mon Sep 17 00:00:00 2001 From: kay Date: Thu, 8 May 2014 15:09:41 -0400 Subject: [PATCH 2/6] tweak orderby size limit blurb --- source/reference/operator/meta/orderby.txt | 28 ++++++++++------------ 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/source/reference/operator/meta/orderby.txt b/source/reference/operator/meta/orderby.txt index 83407b94b51..3a7427c0ba4 100644 --- a/source/reference/operator/meta/orderby.txt +++ b/source/reference/operator/meta/orderby.txt @@ -29,18 +29,16 @@ $orderby ``-1``, as above) to sort in descending order or a positive value (e.g. ``1``) to sort in ascending order. - Unless you have an index for the specified key pattern, use - :operator:`$orderby` in conjunction with :operator:`$maxScan` and/or - :method:`cursor.limit()` to avoid requiring MongoDB to perform a - large in-memory sort. The :method:`cursor.limit()` increases the - speed and reduces the amount of memory required to return this query - by way of an optimized algorithm. - -.. warning:: - - The sort function requires that the entire sort be able to - complete within 32 megabytes. When the sort option consumes more - than 32 megabytes, MongoDB will return an error. Use :operator:`$maxScan` - and/or :method:`~cursor.limit()`, or create an index on the field that you're - sorting to avoid this error. - +Behavior +-------- + +The sort function requires that the entire sort be able to complete +within 32 megabytes. When the sort option consumes more than 32 +megabytes, MongoDB will return an error. + +To avoid this error, create an index to support the sort operation or +use :operator:`$orderby` in conjunction with :operator:`$maxScan` +and/or :method:`cursor.limit()`. The :method:`cursor.limit()` increases +the speed and reduces the amount of memory required to return this +query by way of an optimized algorithm. The specified limit must result +in a number of documents that fall within the 32 megabyte limit. From 78cc4c827648c938e63154e96808910056e39c23 Mon Sep 17 00:00:00 2001 From: joannac- Date: Tue, 28 Jan 2014 14:22:34 -0500 Subject: [PATCH 3/6] Reordered steps, added detail to step 2 Signed-off-by: kay --- source/tutorial/restore-sharded-cluster.txt | 27 ++++++++++++--------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/source/tutorial/restore-sharded-cluster.txt b/source/tutorial/restore-sharded-cluster.txt index 1954cecf848..bbf338e8fa3 100644 --- a/source/tutorial/restore-sharded-cluster.txt +++ b/source/tutorial/restore-sharded-cluster.txt @@ -28,21 +28,26 @@ Procedure #. Stop all :program:`mongos` and :program:`mongod` processes, including all shards *and* all config servers. -#. If shard hostnames have changed, you must manually update the - ``shards`` collection in the :ref:`config-database` to use the new - hostnames. Do the following: +Procedure +--------- + +#. Stop all :program:`mongos` and :program:`mongod` processes, + including all shards *and* all config servers. + +# If shard hostnames have *not* changed, skip to step 3. Otherwise: - a. Start the three :ref:`config servers ` by + #. Restore the config database on each config server + + b. Start the three :ref:`config servers ` by issuing commands similar to the following, using values appropriate to your configuration: - + .. code-block:: sh mongod --configsvr --dbpath /data/configdb --port 27019 - #. Restore the :ref:`config-database` on each config server. - - #. Start one :program:`mongos` instance. + #. Start one :program:`mongos` instance, making sure you have updated the + config string with the new configdb hostnames and ports #. Update the :ref:`config-database` collection named ``shards`` to reflect the new hostnames. @@ -59,12 +64,12 @@ Procedure - Data files for each :ref:`config server `, if you have not already done so in the previous step. -#. Restart all the :program:`mongos` instances. +#. Restart all the config servers :program:`mongod` instances. + +#. Restart all the :program:`mongos` instances, making sure to use the updated config string. #. Restart all the shard :program:`mongod` instances. -#. Restart all the config servers :program:`mongod` instances. - #. Connect to a :program:`mongos` instance from a :program:`mongo` shell and use the :method:`db.printShardingStatus()` method to ensure that the cluster is operational, as follows: From 173f5c3c25132ba5530526c9b8a976736e391bd1 Mon Sep 17 00:00:00 2001 From: kay Date: Thu, 8 May 2014 15:18:15 -0400 Subject: [PATCH 4/6] reorg restore sharded clusters steps --- source/tutorial/restore-sharded-cluster.txt | 46 +++++++++------------ 1 file changed, 19 insertions(+), 27 deletions(-) diff --git a/source/tutorial/restore-sharded-cluster.txt b/source/tutorial/restore-sharded-cluster.txt index bbf338e8fa3..70d25682bf1 100644 --- a/source/tutorial/restore-sharded-cluster.txt +++ b/source/tutorial/restore-sharded-cluster.txt @@ -28,30 +28,6 @@ Procedure #. Stop all :program:`mongos` and :program:`mongod` processes, including all shards *and* all config servers. -Procedure ---------- - -#. Stop all :program:`mongos` and :program:`mongod` processes, - including all shards *and* all config servers. - -# If shard hostnames have *not* changed, skip to step 3. Otherwise: - - #. Restore the config database on each config server - - b. Start the three :ref:`config servers ` by - issuing commands similar to the following, using values appropriate - to your configuration: - - .. code-block:: sh - - mongod --configsvr --dbpath /data/configdb --port 27019 - - #. Start one :program:`mongos` instance, making sure you have updated the - config string with the new configdb hostnames and ports - - #. Update the :ref:`config-database` collection named ``shards`` to reflect the - new hostnames. - #. Restore the following: - Data files for each server in each :term:`shard`. Because replica @@ -61,10 +37,26 @@ Procedure :ref:`backup-restore-snapshot` and :ref:`backup-restore-dump` sections for details on these procedures. - - Data files for each :ref:`config server `, - if you have not already done so in the previous step. + - Data files for each :ref:`config server `. + +#. Restart all the :ref:`config servers ` + :program:`mongod` instances by issuing command similar to the + following, using values appropriate + to your configuration: + + .. code-block:: sh + + mongod --configsvr --dbpath /data/configdb --port 27019 + +#. If shard hostnames **have changed**: + + a. Start one :program:`mongos` instance, using the updated + config string with the new ``configdb`` hostnames and ports. + + #. Update the ``shards`` collection in the :ref:`config-database` to + reflect the new hostnames. -#. Restart all the config servers :program:`mongod` instances. + #. Stop the :program:`mongos` instance. #. Restart all the :program:`mongos` instances, making sure to use the updated config string. From 13ce98153ce1c5ff0ce35c2e7e388bee39aadf8f Mon Sep 17 00:00:00 2001 From: kay Date: Thu, 8 May 2014 15:37:53 -0400 Subject: [PATCH 5/6] trivial: swap step 5 and 6 in restore sharded cluster tutorial --- source/tutorial/restore-sharded-cluster.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/tutorial/restore-sharded-cluster.txt b/source/tutorial/restore-sharded-cluster.txt index 70d25682bf1..4585ab28d7c 100644 --- a/source/tutorial/restore-sharded-cluster.txt +++ b/source/tutorial/restore-sharded-cluster.txt @@ -58,10 +58,10 @@ Procedure #. Stop the :program:`mongos` instance. -#. Restart all the :program:`mongos` instances, making sure to use the updated config string. - #. Restart all the shard :program:`mongod` instances. +#. Restart all the :program:`mongos` instances, making sure to use the updated config string. + #. Connect to a :program:`mongos` instance from a :program:`mongo` shell and use the :method:`db.printShardingStatus()` method to ensure that the cluster is operational, as follows: From 1e461aa4aba62e1a39f6245aaef4af765de1d528 Mon Sep 17 00:00:00 2001 From: achille Date: Thu, 8 May 2014 13:46:47 -0700 Subject: [PATCH 6/6] Disable balancer while indexes are inconsistent A chunk migration can cause an index build, so we should advise to disable the balancer to avoid performance issues. --- source/tutorial/build-indexes-on-replica-sets.txt | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/source/tutorial/build-indexes-on-replica-sets.txt b/source/tutorial/build-indexes-on-replica-sets.txt index f1946b714f1..28b8b663928 100644 --- a/source/tutorial/build-indexes-on-replica-sets.txt +++ b/source/tutorial/build-indexes-on-replica-sets.txt @@ -51,7 +51,15 @@ Procedure If you need to build an index in a :term:`sharded cluster`, repeat the following procedure for each replica set that provides each - :term:`shard`. + :term:`shard`. Also ensure the balancer has been disabled for all + collections on which you'll be creating an new index. This will prevent + an automatic index build as chunks are migrated. The balancer should be + enabled after the indexes have been updated on all shards. + +.. code-block:: sh + + sh.disableBalancing("dbName.collectionName"); + .. _tutorial-index-on-replica-sets-stop-one-member: