From 286c5f54de6c1b459f09904de645d786577ab74b Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Wed, 13 Feb 2013 20:57:16 -0500 Subject: [PATCH 1/5] DOCS-461 configure behavior of balancer --- source/core/sharded-cluster-internals.txt | 14 +- source/reference/command/moveChunk.txt | 17 +- source/sharding.txt | 1 + .../configure-sharded-cluster-balancer.txt | 180 ++++++++++++++++++ .../manage-chunks-in-sharded-cluster.txt | 7 +- .../manage-sharded-cluster-balancer.txt | 4 + 6 files changed, 205 insertions(+), 18 deletions(-) create mode 100644 source/tutorial/configure-sharded-cluster-balancer.txt diff --git a/source/core/sharded-cluster-internals.txt b/source/core/sharded-cluster-internals.txt index 47de4739b10..b900174ae37 100644 --- a/source/core/sharded-cluster-internals.txt +++ b/source/core/sharded-cluster-internals.txt @@ -448,6 +448,8 @@ when modifying chunk size: chunks must grow through insertion or updates until they reach the new size. +.. _sharding-shard-size: + Shard Size ~~~~~~~~~~ @@ -462,7 +464,11 @@ command. This will prevent the :term:`balancer` from migrating chunks to the shard when the value of :data:`mem.mapped ` exceeds the ``maxSize`` setting. -.. seealso:: :doc:`/administration/monitoring`. +.. seealso:: + + :ref:`sharded-cluster-config-max-shard-size` + + :doc:`/administration/monitoring`. .. _sharding-chunk-migration: @@ -506,9 +512,9 @@ All chunk migrations use the following procedure: no open cursors on the chunk, the source shard starts deleting its copy of documents from the migrated chunk. -When the ``_secondaryThrottle`` is ``true`` for :dbcommand:`moveChunk` -or the :term:`balancer`, MongoDB ensure that *one* :term:`secondary` -member has replicated changes before allowing new chunk migrations. +If enabled, the ``_secondaryThrottle`` setting causes the balancer to +wait for replication to secondaries. For more information, see +:ref:`sharded-cluster-config-secondary-throttle`. Detect Connections to :program:`mongos` Instances ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/source/reference/command/moveChunk.txt b/source/reference/command/moveChunk.txt index de253c6c7b4..59793bbe8b0 100644 --- a/source/reference/command/moveChunk.txt +++ b/source/reference/command/moveChunk.txt @@ -29,17 +29,12 @@ moveChunk :param to: The identifier of the shard, that you want to migrate the chunk to. - :param _secondaryThrottle: Optional. Set to ``false`` by - default. Provides :ref:`write concern ` - support for chunk - migrations. - - If you set ``_secondaryThrottle`` to ``true``, during chunk - migrations when a :term:`shard` hosted by a :term:`replica set`, - the :program:`mongod` will wait until the :term:`secondary` members - replicate the migration operations continuing to migrate chunk - data. You may also configure ``_secondaryThrottle`` in the balancer - configuration. + :param _secondaryThrottle: Optional. Set to ``false`` by default. If + set to ``true``, the balancer waits for + replication to :ref:`secondaries + ` before migrating chunks. For + details, see + :ref:`sharded-cluster-config-secondary-throttle`. Use the :method:`sh.moveChunk()` helper in the :program:`mongo` shell to migrate chunks manually. diff --git a/source/sharding.txt b/source/sharding.txt index eea981b4677..cac149adde2 100644 --- a/source/sharding.txt +++ b/source/sharding.txt @@ -54,6 +54,7 @@ Sharded Cluster Maintenance and Administration tutorial/manage-sharded-cluster-config-server tutorial/manage-chunks-in-sharded-cluster + tutorial/configure-sharded-cluster-balancer tutorial/manage-sharded-cluster-balancer tutorial/remove-shards-from-cluster diff --git a/source/tutorial/configure-sharded-cluster-balancer.txt b/source/tutorial/configure-sharded-cluster-balancer.txt new file mode 100644 index 00000000000..2428c1f56ce --- /dev/null +++ b/source/tutorial/configure-sharded-cluster-balancer.txt @@ -0,0 +1,180 @@ +.. index:: balancing; configure +.. _sharding-balancing-configure: + +========================================================== +Configure Behavior of Balancer Process in Sharded Clusters +========================================================== + +.. default-domain:: mongodb + +This section describes the settings you can configure on the balancer. +For conceptual information about the balancer, see +:ref:`sharding-balancing` and :ref:`sharding-balancing-internals`. + +You configure balancer settings through parameters in database commands +or through fields in the :data:`~config.settings` collection in the +:ref:`config database `. + +.. index:: balancing; secondary throttle +.. index:: secondary throttle +.. _sharded-cluster-config-secondary-throttle: + +Require Replication before Chunk Migration (Secondary Throttle) +--------------------------------------------------------------- + +.. versionadded:: 2.2.1 + +You can configure the balancer to wait for replication to secondaries +before migrating chunks. You do so by enabling the balancer's +``_secondaryThrottle`` parameter. + +Secondary throttle can speed performance in cases where you have +migration-caused I/O peaks that do not cooperate with other workloads. + +.. above para is paraphrased from SERVER-7686 + +When enabled, secondary throttle puts a ``{ w : 2 }`` write concern on +deletes and on bulk clones, which means the balancer waits for those +operations to replicate to at least one secondary before migrating +chunks. + +.. BACKGROUND NOTES + Specifically, secondary throttle affects the first and fourth + phases (informal phases) of chunk migration. Migration can happen during + the second and third phases (the "steady state"): + 1) bulk clone data from shardA to shardB in the chunk range + 2) continue to copy over ongoing changes that occurred during the initial clone step + as well as current changes to that chunk range + 3) Stop writes, allow shardB to get final changes, commit migration to config server + 4) cleanup now-inactive data on shardA in chunk range (once all cursors are done) + +To enable secondary throttle, set ``_secondaryThrottle`` +to ``true`` by doing either of the following: + +- Issue the :dbcommand:`moveChunk` command with the + ``_secondaryThrottle`` parameter set to ``true``. + +- Enable the ``_secondaryThrottle`` setting directly in the + :data:`~config.settings` collection in the :ref:`config database + `. To do so, run the following commands from the + :program:`mongo` shell: + + .. code-block:: javascript + + use config + db.settings.update( { "_id" : "balancer" } , { $set : { "_secondaryThrottle" : true } } ) + +.. _sharded-cluster-config-no-auto-split: + +Prevent Auto-Splitting of Chunks +-------------------------------- + +.. versionadded:: 2.0.7 + +By default, :program:`mongos` instances automatically split chunks +during inserts or updates if the chunks exceed the default chunk size. +When chunk distribution becomes uneven, the balancer automatically +migrates chunks among shards. Automatic chunk migrations are crucial for +distributing data, but for deployments with large numbers of +:program:`mongos` instances, the automatic migration might affect the +performance of the cluster. + +You can turn off the auto-splitting of chunks by enabling +:setting:`noAutoSplit` for individual :program:`mongos` instances. + +.. note:: Turning off auto-splitting can lead to an imbalanced + distribution of data in the sharded cluster. + +To turn off auto-splitting, do one of the following: + +- When staring a :program:`mongos`, include the :option:`--noAutoSplit + ` command-line option. + +- In the configuration file for a given :program:`mongos`, include the + :setting:`noAutoSplit` setting. + +Because any :program:`mongos` in a cluster can create a split, to +totally disable splitting in a cluster you must set +:setting:`noAutoSplit` on all :program:`mongos`. + +.. warning:: + + With :setting:`noAutoSplit` enabled, the data in your sharded cluster + may become imbalanced over time. Enable with caution. + +.. _sharded-cluster-config-balancing-window: + +Schedule a Window of Time for Balancing to Occur +------------------------------------------------ + +You can schedule a window of time during which the balancer is allowed +to migrate chunks. See :ref:`sharding-schedule-balancing-window` and +:ref:`sharding-balancing-remove-window`. + +.. _sharded-cluster-config-default-chunk-size: + +Change the Default Chunk Size +----------------------------- + +The default chunk size for a sharded cluster affects how often chunks +are split and migrated. For details, see :ref:`sharding-chunk-size`. + +To modify the default chunk size for a sharded cluster, see +:ref:`sharding-balancing-modify-chunk-size`. + +.. _sharded-cluster-config-max-shard-size: + +Change the Maximum Size for a Given Shard +----------------------------------------- + +The ``maxSize`` field in the :data:`~config.shards` collection in the +:ref:`config database ` sets the maximum size for a +shard, allowing you to control disk use and affect whether the balancer +will migrate chunks to a shard. By default, ``maxSize`` is not +specified, allowing shards to consume the total amount of available +space on their machines if necessary. You can set ``maxSize`` both when +adding a shard and once a shard is running. + +.. seealso:: :ref:`sharding-shard-size` + +Set Maximum Size When Adding a Shard +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When adding a shard using the :dbcommand:`addShard` command, set the +``maxSize`` parameter to the maximum size in megabytes. For example, the +following command run in the :program:`mongo` shell adds a shard with a +maximum size of 125 megabytes: + +.. code-block:: javascript + + db.runCommand( { addshard : "example.net:34008", maxSize : 125 } ) + +Set Maximum Size on a Running Shard +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To set ``maxSize`` on an existing shard, insert or update the +``maxSize`` field in the :data:`~config.shards` collection in the +:ref:`config database `. Set the ``maxSize`` in +megabytes. + +.. example:: Assume you have the following shard without a ``maxSize`` field: + + .. code-block:: javascript + + { "_id" : "shard0000", "host" : "example.net:34001" } + + Run the following sequence of commands in the :program:`mongo` shell + to insert a ``maxSize`` of 125 megabytes:: + + .. code-block:: javascript + + use config + db.shards.update( { _id : "shard0000" }, { $set : { maxSize : 125 } }, { upsert: true } ) + + To later increase the ``maxSize`` setting to 250 megabytes, run the + following: + + .. code-block:: javascript + + use config + db.shards.update( { _id : "shard0000" }, { $set : { maxSize : 250 } } ) diff --git a/source/tutorial/manage-chunks-in-sharded-cluster.txt b/source/tutorial/manage-chunks-in-sharded-cluster.txt index 9f606877224..39fd7473712 100644 --- a/source/tutorial/manage-chunks-in-sharded-cluster.txt +++ b/source/tutorial/manage-chunks-in-sharded-cluster.txt @@ -293,9 +293,10 @@ to pre-splitting. .. versionadded:: 2.2 :dbcommand:`moveChunk` command has the: ``_secondaryThrottle`` - parameter. When set to ``true``, MongoDB ensures that - :term:`secondary` members have replicated operations before allowing - new chunk migrations. + parameter. When set to ``true``, MongoDB ensures replication to + :term:`secondaries ` before allowing + new chunk migrations. For more information, see + :ref:`sharded-cluster-config-secondary-throttle`. .. warning:: diff --git a/source/tutorial/manage-sharded-cluster-balancer.txt b/source/tutorial/manage-sharded-cluster-balancer.txt index 2ce2c6e9a1c..b9972a178ec 100644 --- a/source/tutorial/manage-sharded-cluster-balancer.txt +++ b/source/tutorial/manage-sharded-cluster-balancer.txt @@ -22,6 +22,10 @@ This page includes the following: - :ref:`sharding-balancing-disable-temporally` +.. seealso:: + + :ref:`sharding-balancing-configure` + .. _sharding-balancing-check-lock: Check the Balancer Lock From fa7377d785bbc167db09639fa5c019c133cd21eb Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Thu, 14 Feb 2013 15:47:57 -0500 Subject: [PATCH 2/5] DOCS-461 review edits --- .../configure-sharded-cluster-balancer.txt | 155 +++++++----------- 1 file changed, 63 insertions(+), 92 deletions(-) diff --git a/source/tutorial/configure-sharded-cluster-balancer.txt b/source/tutorial/configure-sharded-cluster-balancer.txt index 2428c1f56ce..9e787e797b7 100644 --- a/source/tutorial/configure-sharded-cluster-balancer.txt +++ b/source/tutorial/configure-sharded-cluster-balancer.txt @@ -7,14 +7,17 @@ Configure Behavior of Balancer Process in Sharded Clusters .. default-domain:: mongodb -This section describes the settings you can configure on the balancer. +The balancer runs on a single :program:`mongos` instance and distributes +chunks evenly throughout a sharded cluster. In most deployments, you do +not need to configure the balancer. The balancer automatically +distributes chunks in an optimal manner. However, administrators might +need to modify balancer behavior depending on application or operational +requirements. Should a situation arise where modifying balancer behavior +is necessary, this page describes settings that can be changed. + For conceptual information about the balancer, see :ref:`sharding-balancing` and :ref:`sharding-balancing-internals`. -You configure balancer settings through parameters in database commands -or through fields in the :data:`~config.settings` collection in the -:ref:`config database `. - .. index:: balancing; secondary throttle .. index:: secondary throttle .. _sharded-cluster-config-secondary-throttle: @@ -25,16 +28,14 @@ Require Replication before Chunk Migration (Secondary Throttle) .. versionadded:: 2.2.1 You can configure the balancer to wait for replication to secondaries -before migrating chunks. You do so by enabling the balancer's -``_secondaryThrottle`` parameter. - -Secondary throttle can speed performance in cases where you have -migration-caused I/O peaks that do not cooperate with other workloads. - -.. above para is paraphrased from SERVER-7686 +during migrations. You do so by enabling the balancer's +``_secondaryThrottle`` parameter, which reduces throughput (i.e., +"throttles") in order to decrease the load on secondaries. You might do +this, for example, if you have migration-caused I/O peaks that impact +other workloads When enabled, secondary throttle puts a ``{ w : 2 }`` write concern on -deletes and on bulk clones, which means the balancer waits for those +deletes and on copies, which means the balancer waits for those operations to replicate to at least one secondary before migrating chunks. @@ -42,65 +43,25 @@ chunks. Specifically, secondary throttle affects the first and fourth phases (informal phases) of chunk migration. Migration can happen during the second and third phases (the "steady state"): - 1) bulk clone data from shardA to shardB in the chunk range - 2) continue to copy over ongoing changes that occurred during the initial clone step + 1) copies the documents in the chunk from shardA to shardB + 2) continues to copy over ongoing changes that occurred during the initial copy step, as well as current changes to that chunk range 3) Stop writes, allow shardB to get final changes, commit migration to config server 4) cleanup now-inactive data on shardA in chunk range (once all cursors are done) -To enable secondary throttle, set ``_secondaryThrottle`` -to ``true`` by doing either of the following: - -- Issue the :dbcommand:`moveChunk` command with the - ``_secondaryThrottle`` parameter set to ``true``. - -- Enable the ``_secondaryThrottle`` setting directly in the - :data:`~config.settings` collection in the :ref:`config database - `. To do so, run the following commands from the - :program:`mongo` shell: - - .. code-block:: javascript - - use config - db.settings.update( { "_id" : "balancer" } , { $set : { "_secondaryThrottle" : true } } ) - -.. _sharded-cluster-config-no-auto-split: - -Prevent Auto-Splitting of Chunks --------------------------------- - -.. versionadded:: 2.0.7 - -By default, :program:`mongos` instances automatically split chunks -during inserts or updates if the chunks exceed the default chunk size. -When chunk distribution becomes uneven, the balancer automatically -migrates chunks among shards. Automatic chunk migrations are crucial for -distributing data, but for deployments with large numbers of -:program:`mongos` instances, the automatic migration might affect the -performance of the cluster. +You enable ``_secondaryThrottle`` directly in the +:data:`settings <~config.settings>` collection in the :ref:`config database +` by running the following commands from the +:program:`mongo` shell: -You can turn off the auto-splitting of chunks by enabling -:setting:`noAutoSplit` for individual :program:`mongos` instances. - -.. note:: Turning off auto-splitting can lead to an imbalanced - distribution of data in the sharded cluster. - -To turn off auto-splitting, do one of the following: - -- When staring a :program:`mongos`, include the :option:`--noAutoSplit - ` command-line option. - -- In the configuration file for a given :program:`mongos`, include the - :setting:`noAutoSplit` setting. - -Because any :program:`mongos` in a cluster can create a split, to -totally disable splitting in a cluster you must set -:setting:`noAutoSplit` on all :program:`mongos`. +.. code-block:: javascript -.. warning:: + use config + db.settings.update( { "_id" : "balancer" } , { $set : { "_secondaryThrottle" : true } } ) - With :setting:`noAutoSplit` enabled, the data in your sharded cluster - may become imbalanced over time. Enable with caution. +You also can enable secondary throttle when issuing the +:dbcommand:`moveChunk` command by setting ``_secondaryThrottle`` to +``true``. For more information, see :dbcommand:`moveChunk`. .. _sharded-cluster-config-balancing-window: @@ -108,63 +69,73 @@ Schedule a Window of Time for Balancing to Occur ------------------------------------------------ You can schedule a window of time during which the balancer is allowed -to migrate chunks. See :ref:`sharding-schedule-balancing-window` and -:ref:`sharding-balancing-remove-window`. +to migrate chunks, as described in the following procedures: + +- :ref:`sharding-schedule-balancing-window` + +- :ref:`sharding-balancing-remove-window`. + +The configured time is evaluated relative to the time zone of each +individual :program:`mongos` instance in the sharded cluster. .. _sharded-cluster-config-default-chunk-size: -Change the Default Chunk Size ------------------------------ +Configure Default Chunk Size +---------------------------- -The default chunk size for a sharded cluster affects how often chunks -are split and migrated. For details, see :ref:`sharding-chunk-size`. +The default chunk size for a sharded cluster is 64 megabytes. In most +situations, the default size is optimal for splitting and migrating +chunks. For information on how chunk size affects deployments, see +details, see :ref:`sharding-chunk-size`. -To modify the default chunk size for a sharded cluster, see -:ref:`sharding-balancing-modify-chunk-size`. +Changing the default chunk size affects chunks that are processes during +migrations and auto-splits but does not retroactively affect all chunks. + +To configure default chunk size, see :ref:`sharding-balancing-modify-chunk-size`. .. _sharded-cluster-config-max-shard-size: -Change the Maximum Size for a Given Shard ------------------------------------------ +Change the Maximum Storage Size for a Given Shard +------------------------------------------------- The ``maxSize`` field in the :data:`~config.shards` collection in the :ref:`config database ` sets the maximum size for a -shard, allowing you to control disk use and affect whether the balancer -will migrate chunks to a shard. By default, ``maxSize`` is not -specified, allowing shards to consume the total amount of available -space on their machines if necessary. You can set ``maxSize`` both when -adding a shard and once a shard is running. +shard, allowing you to control whether the balancer will migrate chunks +to a shard. If :data:`dataSize <~dbStats.dataSize>` is above a shard's +``maxSize``, the balancer will not move chunks to the shard. The +balancer also will not move chunks off the shard. The ``maxSize`` value +only affects the balancer's selection of destination shards. -.. seealso:: :ref:`sharding-shard-size` +By default, ``maxSize`` is not specified, allowing shards to consume the +total amount of available space on their machines if necessary. -Set Maximum Size When Adding a Shard -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +You can set ``maxSize`` both when adding a shard and once a shard is +running. -When adding a shard using the :dbcommand:`addShard` command, set the -``maxSize`` parameter to the maximum size in megabytes. For example, the -following command run in the :program:`mongo` shell adds a shard with a -maximum size of 125 megabytes: +To set ``maxSize`` when adding a shard, set the :dbcommand:`addShard` +command's ``maxSize`` parameter to the maximum size in megabytes. For +example, the following command run in the :program:`mongo` shell adds a +shard with a maximum size of 125 megabytes: .. code-block:: javascript db.runCommand( { addshard : "example.net:34008", maxSize : 125 } ) -Set Maximum Size on a Running Shard -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - To set ``maxSize`` on an existing shard, insert or update the ``maxSize`` field in the :data:`~config.shards` collection in the :ref:`config database `. Set the ``maxSize`` in megabytes. -.. example:: Assume you have the following shard without a ``maxSize`` field: +.. example:: + + Assume you have the following shard without a ``maxSize`` field: .. code-block:: javascript { "_id" : "shard0000", "host" : "example.net:34001" } Run the following sequence of commands in the :program:`mongo` shell - to insert a ``maxSize`` of 125 megabytes:: + to insert a ``maxSize`` of 125 megabytes: .. code-block:: javascript From 03f4010f7606c521ba2d1c33cda05dba130cc5d7 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Thu, 14 Feb 2013 18:39:13 -0500 Subject: [PATCH 3/5] DOCS-461 review edits II --- source/reference/command/moveChunk.txt | 4 +- .../configure-sharded-cluster-balancer.txt | 101 +++++++++--------- 2 files changed, 53 insertions(+), 52 deletions(-) diff --git a/source/reference/command/moveChunk.txt b/source/reference/command/moveChunk.txt index 59793bbe8b0..ade16f90778 100644 --- a/source/reference/command/moveChunk.txt +++ b/source/reference/command/moveChunk.txt @@ -32,8 +32,8 @@ moveChunk :param _secondaryThrottle: Optional. Set to ``false`` by default. If set to ``true``, the balancer waits for replication to :ref:`secondaries - ` before migrating chunks. For - details, see + ` while copying and deleting + data during migrations. For details, see :ref:`sharded-cluster-config-secondary-throttle`. Use the :method:`sh.moveChunk()` helper in the :program:`mongo` diff --git a/source/tutorial/configure-sharded-cluster-balancer.txt b/source/tutorial/configure-sharded-cluster-balancer.txt index 9e787e797b7..1135a5e22ad 100644 --- a/source/tutorial/configure-sharded-cluster-balancer.txt +++ b/source/tutorial/configure-sharded-cluster-balancer.txt @@ -18,51 +18,6 @@ is necessary, this page describes settings that can be changed. For conceptual information about the balancer, see :ref:`sharding-balancing` and :ref:`sharding-balancing-internals`. -.. index:: balancing; secondary throttle -.. index:: secondary throttle -.. _sharded-cluster-config-secondary-throttle: - -Require Replication before Chunk Migration (Secondary Throttle) ---------------------------------------------------------------- - -.. versionadded:: 2.2.1 - -You can configure the balancer to wait for replication to secondaries -during migrations. You do so by enabling the balancer's -``_secondaryThrottle`` parameter, which reduces throughput (i.e., -"throttles") in order to decrease the load on secondaries. You might do -this, for example, if you have migration-caused I/O peaks that impact -other workloads - -When enabled, secondary throttle puts a ``{ w : 2 }`` write concern on -deletes and on copies, which means the balancer waits for those -operations to replicate to at least one secondary before migrating -chunks. - -.. BACKGROUND NOTES - Specifically, secondary throttle affects the first and fourth - phases (informal phases) of chunk migration. Migration can happen during - the second and third phases (the "steady state"): - 1) copies the documents in the chunk from shardA to shardB - 2) continues to copy over ongoing changes that occurred during the initial copy step, - as well as current changes to that chunk range - 3) Stop writes, allow shardB to get final changes, commit migration to config server - 4) cleanup now-inactive data on shardA in chunk range (once all cursors are done) - -You enable ``_secondaryThrottle`` directly in the -:data:`settings <~config.settings>` collection in the :ref:`config database -` by running the following commands from the -:program:`mongo` shell: - -.. code-block:: javascript - - use config - db.settings.update( { "_id" : "balancer" } , { $set : { "_secondaryThrottle" : true } } ) - -You also can enable secondary throttle when issuing the -:dbcommand:`moveChunk` command by setting ``_secondaryThrottle`` to -``true``. For more information, see :dbcommand:`moveChunk`. - .. _sharded-cluster-config-balancing-window: Schedule a Window of Time for Balancing to Occur @@ -84,7 +39,7 @@ Configure Default Chunk Size ---------------------------- The default chunk size for a sharded cluster is 64 megabytes. In most -situations, the default size is optimal for splitting and migrating +situations, the default size is appropriate for splitting and migrating chunks. For information on how chunk size affects deployments, see details, see :ref:`sharding-chunk-size`. @@ -102,9 +57,10 @@ The ``maxSize`` field in the :data:`~config.shards` collection in the :ref:`config database ` sets the maximum size for a shard, allowing you to control whether the balancer will migrate chunks to a shard. If :data:`dataSize <~dbStats.dataSize>` is above a shard's -``maxSize``, the balancer will not move chunks to the shard. The -balancer also will not move chunks off the shard. The ``maxSize`` value -only affects the balancer's selection of destination shards. +``maxSize``, the balancer will not move chunks to the shard. Also, the +balancer will not move chunks off an overloaded shard. This must happen +manually. The ``maxSize`` value only affects the balancer's selection of +destination shards. By default, ``maxSize`` is not specified, allowing shards to consume the total amount of available space on their machines if necessary. @@ -140,7 +96,7 @@ megabytes. .. code-block:: javascript use config - db.shards.update( { _id : "shard0000" }, { $set : { maxSize : 125 } }, { upsert: true } ) + db.shards.update( { _id : "shard0000" }, { $set : { maxSize : 125 } } ) To later increase the ``maxSize`` setting to 250 megabytes, run the following: @@ -149,3 +105,48 @@ megabytes. use config db.shards.update( { _id : "shard0000" }, { $set : { maxSize : 250 } } ) + +.. index:: balancing; secondary throttle +.. index:: secondary throttle +.. _sharded-cluster-config-secondary-throttle: + +Require Replication before Chunk Migration (Secondary Throttle) +--------------------------------------------------------------- + +.. versionadded:: 2.2.1 + +You can configure the balancer to wait for replication to secondaries +during migrations. You do so by enabling the balancer's +``_secondaryThrottle`` parameter, which reduces throughput (i.e., +"throttles") in order to decrease the load on secondaries. You might do +this, for example, if you have migration-caused I/O peaks that impact +other workloads + +When enabled, secondary throttle puts a ``{ w : 2 }`` write concern on +deletes and on copies, which means the balancer waits for those +operations to replicate to at least one secondary before migrating +chunks. + +.. BACKGROUND NOTES + Specifically, secondary throttle affects the first and fourth + phases (informal phases) of chunk migration. Migration can happen during + the second and third phases (the "steady state"): + 1) copies the documents in the chunk from shardA to shardB + 2) continues to copy over ongoing changes that occurred during the initial copy step, + as well as current changes to that chunk range + 3) Stop writes, allow shardB to get final changes, commit migration to config server + 4) cleanup now-inactive data on shardA in chunk range (once all cursors are done) + +You enable ``_secondaryThrottle`` directly in the +:data:`settings <~config.settings>` collection in the :ref:`config database +` by running the following commands from the +:program:`mongo` shell: + +.. code-block:: javascript + + use config + db.settings.update( { "_id" : "balancer" } , { $set : { "_secondaryThrottle" : true } , { upsert : true } } ) + +You also can enable secondary throttle when issuing the +:dbcommand:`moveChunk` command by setting ``_secondaryThrottle`` to +``true``. For more information, see :dbcommand:`moveChunk`. From 44a44d417f2713c45e5f3d6a132903d0d25b50e3 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Thu, 14 Feb 2013 18:39:52 -0500 Subject: [PATCH 4/5] DOCS-461 review edits III --- source/reference/command/moveChunk.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/reference/command/moveChunk.txt b/source/reference/command/moveChunk.txt index ade16f90778..1e5cb8b669c 100644 --- a/source/reference/command/moveChunk.txt +++ b/source/reference/command/moveChunk.txt @@ -31,8 +31,8 @@ moveChunk :param _secondaryThrottle: Optional. Set to ``false`` by default. If set to ``true``, the balancer waits for - replication to :ref:`secondaries - ` while copying and deleting + replication to :term:`secondaries ` + while copying and deleting data during migrations. For details, see :ref:`sharded-cluster-config-secondary-throttle`. From fc77202b44142fd5dc22853c8d5563125db60cba Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Fri, 15 Feb 2013 09:51:53 -0500 Subject: [PATCH 5/5] DOCS-461 edit to link targets --- source/tutorial/configure-sharded-cluster-balancer.txt | 1 - source/tutorial/manage-sharded-cluster-balancer.txt | 3 +-- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/source/tutorial/configure-sharded-cluster-balancer.txt b/source/tutorial/configure-sharded-cluster-balancer.txt index 1135a5e22ad..cd2e9495b01 100644 --- a/source/tutorial/configure-sharded-cluster-balancer.txt +++ b/source/tutorial/configure-sharded-cluster-balancer.txt @@ -1,5 +1,4 @@ .. index:: balancing; configure -.. _sharding-balancing-configure: ========================================================== Configure Behavior of Balancer Process in Sharded Clusters diff --git a/source/tutorial/manage-sharded-cluster-balancer.txt b/source/tutorial/manage-sharded-cluster-balancer.txt index b9972a178ec..454e340805b 100644 --- a/source/tutorial/manage-sharded-cluster-balancer.txt +++ b/source/tutorial/manage-sharded-cluster-balancer.txt @@ -1,5 +1,4 @@ .. index:: balancing; operations -.. _sharding-balancing-operations: =============================== Manage Sharded Cluster Balancer @@ -24,7 +23,7 @@ This page includes the following: .. seealso:: - :ref:`sharding-balancing-configure` + :doc:`/tutorial/configure-sharded-cluster-balancer` .. _sharding-balancing-check-lock: