From f5abe3ba0fab07afc7afc1525dc9a9d4f4c8dc59 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Fri, 12 Oct 2012 13:16:09 -0400 Subject: [PATCH 1/6] DOCS-303 migrated the sharding config schema page --- source/core/sharding-internals.txt | 146 +++++++++++++++++++++++++++-- 1 file changed, 136 insertions(+), 10 deletions(-) diff --git a/source/core/sharding-internals.txt b/source/core/sharding-internals.txt index ca86fe18c5b..16bec398192 100644 --- a/source/core/sharding-internals.txt +++ b/source/core/sharding-internals.txt @@ -21,7 +21,7 @@ Shard Keys ---------- Shard keys are the field in a collection that MongoDB uses to -distribute :term:`documents ` within a shard cluster. See the +distribute :term:`documents ` within a sharding cluster. See the :ref:`overview of shard keys ` for an introduction to these topics. @@ -169,7 +169,7 @@ compound shard key. The data may become more splitable with a compound shard key. .. see:: ":ref:`sharding-mongos`" for more information on query - operations in the context of shard clusters. + operations in the context of sharding clusters. .. [#shard-key-index] In many ways, you can think of the shard key a cluster-wide unique index. However, be aware that sharded systems @@ -402,7 +402,7 @@ than two.* The specification of the balancing window is relative to the local time zone of all individual :program:`mongos` instances in the - shard cluster. + sharding cluster. .. index:: sharding; chunk size .. _sharding-chunk-size: @@ -511,30 +511,156 @@ member has replicated changes before allowing new chunk migrations. Config Database --------------- -The ``config`` database contains sharding configuration information. To -start the ``config`` database from the :program:`mongo` shell, issue the -following command: +The ``config`` database contains information about your sharding +configuration and stores the information in a set of collections +specific to sharding. + +.. important:: Back up the ``config`` database before performing + any maintenance on the config server. + +To start the ``config`` database, issue the following command from the +:program:`mongo` shell: .. code-block:: javascript use config -The config database includes the following collections used by sharding: +The config database includes the following collections specific to sharding: .. data:: chunks + This collection lists the chunks in the sharding cluster. Each chunk + is listed as a separate document. The following is the document for a + chunk with the ``_id`` ``mydb.foo-animal_\"cat\"``. Among other + information, the document shows the chunk's minimum and maximum + values (in the ``min`` and ``max`` fields) and the name of the shard + to which the chunk belongs (in the ``shard`` field): + + .. code-block:: javascript + + { + + "_id" : "mydb.foo-a_\"cat\"", + + "lastmod" : Timestamp(1000, 3), + + "lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc"), + + "ns" : "mydb.foo", + + "min" : { + + "animal" : "cat" + + }, + + "max" : { + + "animal" : "dog" + + }, + + "shard" : "shard0004" + + } + +.. data:: collections + + This lists the collections in the sharding cluster. Each collection is + listed as a separate document. The following is the document for the + ``foo`` collection in the ``mydb`` database: + + .. code-block:: javascript + + { + + "_id" : "mydb.foo", + + "lastmod" : ISODate("1970-01-16T15:00:58.107Z"), + + "dropped" : false, + + "key" : { + + "a" : 1 + + }, + + "unique" : false, + + "lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc") + + } + .. data:: databases -.. data:: lockpings + This collection lists the databases in the sharding cluster and whether + or not each is partitioned. Each database is listed as a separate + document. If a database is partitioned, the ``primary`` key gives the + name of the :term:`primary shard`. + + .. code-block:: + + { "_id" : "admin", "partitioned" : false, "primary" : "config" } + { "_id" : "mydb", "partitioned" : true, "primary" : "shard0000" } + +.. TODO Is data:: lockpings a shard-specific collection? -.. data:: locks +.. TODO Is data:: locks a shard-specific collection? .. data:: mongos + This collection lists all the :program:`mongos` sharding routers + affiliated with the sharding cluster. Each :program:`mongos` is + listed as a separate document. A :program:`mongos` pings the cluster + every 30 seconds to verify to the cluster the :program:`mongos` is + active. This collection is used for reporting. + + The following document shows the status of the :program:`mongos` + running on ``example.com:30000``. + + .. code-block:: javascript + + { "_id" : "example.com:30000", "ping" : ISODate("2012-10-12T17:08:13.538Z"), "up" : 13699, "waiting" : true } + .. data:: settings + This collection lists the following sharding configuration settings: + + - Chunk size. To change chunk size, see :ref:`sharding-balancing-modify-chunk-size`. + + - Balancer status. To change status, see :ref:`sharding-balancing-disable-temporally`. + + The following is an example ``settings`` collection: + + .. code-block:: javascript + + { "_id" : "chunksize", "value" : 64 } + + { "_id" : "balancer", "stopped" : false } + .. data:: shards -.. data:: system.indexes + This collection lists the shards you have created. Each shard is + listed as a separate document. If the shard is a replica set, the + ``host`` field displays the name of the replica set, then a slash, then + the hostname, as shown here: + + code-block:: javascript + + { "_id" : "shard0000", "host" : "shard1/localhost:30000" } .. data:: version + + This collection contains the current metadata version number. This + collection contains only one document: + + To access the ``version`` collection you must use the + :method:`db.getCollection()` method. For example, to display the + collection's document: + + .. code-block:: javascript + + mongos> db.getCollection("version").find() + + { "_id" : 1, "version" : 3 } From e0d25a8540f2a989ccfffe407187d91a0787a81b Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Fri, 12 Oct 2012 14:08:14 -0400 Subject: [PATCH 2/6] DOCS-303 corrected formattingerrors --- source/core/sharding-internals.txt | 52 +++++++++++++++--------------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/source/core/sharding-internals.txt b/source/core/sharding-internals.txt index 16bec398192..a3475d5d9f8 100644 --- a/source/core/sharding-internals.txt +++ b/source/core/sharding-internals.txt @@ -538,31 +538,31 @@ The config database includes the following collections specific to sharding: .. code-block:: javascript - { + { - "_id" : "mydb.foo-a_\"cat\"", + "_id" : "mydb.foo-a_\"cat\"", - "lastmod" : Timestamp(1000, 3), + "lastmod" : Timestamp(1000, 3), - "lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc"), + "lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc"), - "ns" : "mydb.foo", + "ns" : "mydb.foo", - "min" : { + "min" : { - "animal" : "cat" + "animal" : "cat" - }, + }, - "max" : { + "max" : { - "animal" : "dog" + "animal" : "dog" - }, + }, - "shard" : "shard0004" + "shard" : "shard0004" - } + } .. data:: collections @@ -572,25 +572,25 @@ The config database includes the following collections specific to sharding: .. code-block:: javascript - { + { - "_id" : "mydb.foo", + "_id" : "mydb.foo", - "lastmod" : ISODate("1970-01-16T15:00:58.107Z"), + "lastmod" : ISODate("1970-01-16T15:00:58.107Z"), - "dropped" : false, + "dropped" : false, - "key" : { + "key" : { - "a" : 1 + "a" : 1 - }, + }, - "unique" : false, + "unique" : false, - "lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc") + "lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc") - } + } .. data:: databases @@ -635,9 +635,9 @@ The config database includes the following collections specific to sharding: .. code-block:: javascript - { "_id" : "chunksize", "value" : 64 } + { "_id" : "chunksize", "value" : 64 } - { "_id" : "balancer", "stopped" : false } + { "_id" : "balancer", "stopped" : false } .. data:: shards @@ -646,7 +646,7 @@ The config database includes the following collections specific to sharding: ``host`` field displays the name of the replica set, then a slash, then the hostname, as shown here: - code-block:: javascript + .. code-block:: javascript { "_id" : "shard0000", "host" : "shard1/localhost:30000" } From 0cbfb25675ef7e903700c91d3ba71c5dcf20ea63 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Fri, 12 Oct 2012 14:12:11 -0400 Subject: [PATCH 3/6] DOCS-303 corrected formatting error --- source/core/sharding-internals.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/core/sharding-internals.txt b/source/core/sharding-internals.txt index a3475d5d9f8..d53ed12e477 100644 --- a/source/core/sharding-internals.txt +++ b/source/core/sharding-internals.txt @@ -599,7 +599,7 @@ The config database includes the following collections specific to sharding: document. If a database is partitioned, the ``primary`` key gives the name of the :term:`primary shard`. - .. code-block:: + .. code-block:: javascript { "_id" : "admin", "partitioned" : false, "primary" : "config" } { "_id" : "mydb", "partitioned" : true, "primary" : "shard0000" } From c24dd8235910094ccd01c1b93cd71cee202e5c33 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Fri, 12 Oct 2012 14:17:25 -0400 Subject: [PATCH 4/6] term not in glossary --- source/core/sharding.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/core/sharding.txt b/source/core/sharding.txt index 580bb17e335..8154017d043 100644 --- a/source/core/sharding.txt +++ b/source/core/sharding.txt @@ -462,7 +462,7 @@ Sharded Query Response Process To route a query to a :term:`cluster `, :program:`mongos` uses the following process: -#. Determine the list of :term:`shards` that must receive the query. +#. Determine the list of :term:`shards ` that must receive the query. In some cases, when the :term:`shard key` or a prefix of the shard key is a part of the query, the :program:`mongos` can route the From a53e0b16fca55dca7091ba606bec08612e35876a Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Fri, 12 Oct 2012 14:23:25 -0400 Subject: [PATCH 5/6] review edits --- source/core/sharding-internals.txt | 37 ++++++------------------------ 1 file changed, 7 insertions(+), 30 deletions(-) diff --git a/source/core/sharding-internals.txt b/source/core/sharding-internals.txt index d53ed12e477..5af0b7547cb 100644 --- a/source/core/sharding-internals.txt +++ b/source/core/sharding-internals.txt @@ -21,7 +21,7 @@ Shard Keys ---------- Shard keys are the field in a collection that MongoDB uses to -distribute :term:`documents ` within a sharding cluster. See the +distribute :term:`documents ` within a sharded cluster. See the :ref:`overview of shard keys ` for an introduction to these topics. @@ -169,7 +169,7 @@ compound shard key. The data may become more splitable with a compound shard key. .. see:: ":ref:`sharding-mongos`" for more information on query - operations in the context of sharding clusters. + operations in the context of sharded clusters. .. [#shard-key-index] In many ways, you can think of the shard key a cluster-wide unique index. However, be aware that sharded systems @@ -402,7 +402,7 @@ than two.* The specification of the balancing window is relative to the local time zone of all individual :program:`mongos` instances in the - sharding cluster. + sharded cluster. .. index:: sharding; chunk size .. _sharding-chunk-size: @@ -529,7 +529,7 @@ The config database includes the following collections specific to sharding: .. data:: chunks - This collection lists the chunks in the sharding cluster. Each chunk + This collection lists the chunks in the sharded cluster. Each chunk is listed as a separate document. The following is the document for a chunk with the ``_id`` ``mydb.foo-animal_\"cat\"``. Among other information, the document shows the chunk's minimum and maximum @@ -539,62 +539,41 @@ The config database includes the following collections specific to sharding: .. code-block:: javascript { - "_id" : "mydb.foo-a_\"cat\"", - "lastmod" : Timestamp(1000, 3), - "lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc"), - "ns" : "mydb.foo", - "min" : { - "animal" : "cat" - }, - "max" : { - "animal" : "dog" - }, - "shard" : "shard0004" - } .. data:: collections - This lists the collections in the sharding cluster. Each collection is + This lists the collections in the sharded cluster. Each collection is listed as a separate document. The following is the document for the ``foo`` collection in the ``mydb`` database: .. code-block:: javascript { - "_id" : "mydb.foo", - "lastmod" : ISODate("1970-01-16T15:00:58.107Z"), - "dropped" : false, - "key" : { - "a" : 1 - }, - "unique" : false, - "lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc") - } .. data:: databases - This collection lists the databases in the sharding cluster and whether + This collection lists the databases in the sharded cluster and whether or not each is partitioned. Each database is listed as a separate document. If a database is partitioned, the ``primary`` key gives the name of the :term:`primary shard`. @@ -611,7 +590,7 @@ The config database includes the following collections specific to sharding: .. data:: mongos This collection lists all the :program:`mongos` sharding routers - affiliated with the sharding cluster. Each :program:`mongos` is + affiliated with the sharded cluster. Each :program:`mongos` is listed as a separate document. A :program:`mongos` pings the cluster every 30 seconds to verify to the cluster the :program:`mongos` is active. This collection is used for reporting. @@ -636,7 +615,6 @@ The config database includes the following collections specific to sharding: .. code-block:: javascript { "_id" : "chunksize", "value" : 64 } - { "_id" : "balancer", "stopped" : false } .. data:: shards @@ -662,5 +640,4 @@ The config database includes the following collections specific to sharding: .. code-block:: javascript mongos> db.getCollection("version").find() - { "_id" : 1, "version" : 3 } From 3e3814d4b0632b94e551f41a8cf30ae85cffc968 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Fri, 12 Oct 2012 15:19:47 -0400 Subject: [PATCH 6/6] DOCS-303 review edits --- source/core/sharding-internals.txt | 64 +++++++++++++++++++----------- 1 file changed, 41 insertions(+), 23 deletions(-) diff --git a/source/core/sharding-internals.txt b/source/core/sharding-internals.txt index 5af0b7547cb..bcc450cacfd 100644 --- a/source/core/sharding-internals.txt +++ b/source/core/sharding-internals.txt @@ -487,13 +487,13 @@ All chunk migrations use the following procedure: the source shard for the chunk. In this operation the balancer passes the name of the destination shard to the source shard. -#. The source initaties the move with an internal +#. The source initiaties the move with an internal :dbcommand:`moveChunk` command with the destination shard. #. The destination shard begins requesting documents in the chunk, and begins receiving these chunks. -#. After receving the final document in the chunk, the destination +#. After receiving the final document in the chunk, the destination shard initiates a synchronization process to ensure that all changes to the documents in the chunk on the source shard during the migration process exist on the destination shard. @@ -513,7 +513,7 @@ Config Database The ``config`` database contains information about your sharding configuration and stores the information in a set of collections -specific to sharding. +used by sharding. .. important:: Back up the ``config`` database before performing any maintenance on the config server. @@ -525,13 +525,14 @@ To start the ``config`` database, issue the following command from the use config -The config database includes the following collections specific to sharding: +The ``config`` database holds the following collections that support +sharded cluster operations: -.. data:: chunks +.. data:: chunks - This collection lists the chunks in the sharded cluster. Each chunk - is listed as a separate document. The following is the document for a - chunk with the ``_id`` ``mydb.foo-animal_\"cat\"``. Among other + The :data:`chunks` collection represents each chunk in the sharded cluster + in a separate document. The following is a document for a + chunk with an ``_id`` value of ``mydb.foo-animal_\"cat\"``. Among other information, the document shows the chunk's minimum and maximum values (in the ``min`` and ``max`` fields) and the name of the shard to which the chunk belongs (in the ``shard`` field): @@ -554,9 +555,10 @@ The config database includes the following collections specific to sharding: .. data:: collections - This lists the collections in the sharded cluster. Each collection is - listed as a separate document. The following is the document for the - ``foo`` collection in the ``mydb`` database: + The :data:`collections` collection represents each sharded collection + in the cluster in a separate document. Given a collection named ``foo`` + in the ``mydb`` database, the document in the :data:`collections` + collection would resemble: .. code-block:: javascript @@ -573,7 +575,7 @@ The config database includes the following collections specific to sharding: .. data:: databases - This collection lists the databases in the sharded cluster and whether + The :data:`databases` collection holds the databases in the sharded cluster and whether or not each is partitioned. Each database is listed as a separate document. If a database is partitioned, the ``primary`` key gives the name of the :term:`primary shard`. @@ -583,17 +585,28 @@ The config database includes the following collections specific to sharding: { "_id" : "admin", "partitioned" : false, "primary" : "config" } { "_id" : "mydb", "partitioned" : true, "primary" : "shard0000" } -.. TODO Is data:: lockpings a shard-specific collection? +.. data:: lockpings -.. TODO Is data:: locks a shard-specific collection? + The :data:`lockpings` collection keeps track of the active components + in the sharded cluster. Given a cluster with a :program:`mongos` + running on ``example.com:30000``, the document in the + :data:`lockpings` collection would resemble: + + .. code-block:: javascript + + { "_id" : "example.com:30000:1350047994:16807", "ping" : ISODate("2012-10-12T18:32:54.892Z") } + +.. data:: locks + + The :data:`locks` collection is used for distributed locking between + :program:`mongos` instances and shards. .. data:: mongos - This collection lists all the :program:`mongos` sharding routers - affiliated with the sharded cluster. Each :program:`mongos` is - listed as a separate document. A :program:`mongos` pings the cluster + The :data:`mongos` collection represents each :program:`mongos` + affiliated with the sharded cluster. A :program:`mongos` sends pings to all members of the cluster every 30 seconds to verify to the cluster the :program:`mongos` is - active. This collection is used for reporting. + active. The ``ping`` field shows the time of the last ping. This collection is used for reporting. The following document shows the status of the :program:`mongos` running on ``example.com:30000``. @@ -604,7 +617,7 @@ The config database includes the following collections specific to sharding: .. data:: settings - This collection lists the following sharding configuration settings: + The :data:`settings` collection holds the following sharding configuration settings: - Chunk size. To change chunk size, see :ref:`sharding-balancing-modify-chunk-size`. @@ -619,8 +632,8 @@ The config database includes the following collections specific to sharding: .. data:: shards - This collection lists the shards you have created. Each shard is - listed as a separate document. If the shard is a replica set, the + The :data:`shards` collection represents each shard in the cluster + in a separate document. If the shard is a replica set, the ``host`` field displays the name of the replica set, then a slash, then the hostname, as shown here: @@ -628,12 +641,17 @@ The config database includes the following collections specific to sharding: { "_id" : "shard0000", "host" : "shard1/localhost:30000" } +.. data:: system.indexes + + The :data:`system.indexes` collection contains all the indexes in the + shard. For more information on indexes, see :doc:`/indexes`. + .. data:: version - This collection contains the current metadata version number. This + The :data:`version` collection holds the current metadata version number. This collection contains only one document: - To access the ``version`` collection you must use the + To access the :data:`version` collection you must use the :method:`db.getCollection()` method. For example, to display the collection's document: