From a837eb758c73bc7d2045b999e5efd88eb944ae99 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Tue, 16 Apr 2013 16:34:28 -0400 Subject: [PATCH] DOCS-1395 config server tutorials --- bin/htaccess.yaml | 43 +++ source/administration/sharded-clusters.txt | 6 +- source/faq/sharding.txt | 6 +- source/tutorial.txt | 5 + .../backup-sharded-cluster-metadata.txt | 38 +++ source/tutorial/deploy-config-servers.txt | 46 ++++ .../manage-sharded-cluster-config-server.txt | 246 ------------------ ...onfig-servers-with-different-hostnames.txt | 53 ++++ ...rate-config-servers-with-same-hostname.txt | 40 +++ source/tutorial/replace-config-server.txt | 59 +++++ 10 files changed, 292 insertions(+), 250 deletions(-) create mode 100644 source/tutorial/backup-sharded-cluster-metadata.txt create mode 100644 source/tutorial/deploy-config-servers.txt delete mode 100644 source/tutorial/manage-sharded-cluster-config-server.txt create mode 100644 source/tutorial/migrate-config-servers-with-different-hostnames.txt create mode 100644 source/tutorial/migrate-config-servers-with-same-hostname.txt create mode 100644 source/tutorial/replace-config-server.txt diff --git a/bin/htaccess.yaml b/bin/htaccess.yaml index 611b78bf8ef..aaa59e84282 100644 --- a/bin/htaccess.yaml +++ b/bin/htaccess.yaml @@ -1155,6 +1155,49 @@ redirect-path: '/tutorial/configure-replica-set-tag-sets' url-base: '/reference/replica-configuration' type: 'redirect' code: 301 +outputs: + - 'v2.2' +--- +redirect-path: '/tutorial/manage-sharded-cluster-config-server' +url-base: '/administration/sharded-clusters' +type: 'redirect' +code: 301 +outputs: + - 'master' + - 'after-v2.2' +--- +redirect-path: '/tutorial/deploy-config-servers' +url-base: '/tutorial/manage-sharded-cluster-config-server' +type: 'redirect' +code: 301 +outputs: + - 'v2.2' +--- +redirect-path: '/tutorial/migrate-config-servers-with-same-hostname' +url-base: '/tutorial/manage-sharded-cluster-config-server' +type: 'redirect' +code: 301 +outputs: + - 'v2.2' +--- +redirect-path: '/tutorial/migrate-config-servers-with-different-hostnames' +url-base: '/tutorial/manage-sharded-cluster-config-server' +type: 'redirect' +code: 301 +outputs: + - 'v2.2' +--- +redirect-path: '/tutorial/replace-config-server' +url-base: '/tutorial/manage-sharded-cluster-config-server' +type: 'redirect' +code: 301 +outputs: + - 'v2.2' +--- +redirect-path: '/tutorial/backup-sharded-cluster-metadata' +url-base: '/tutorial/manage-sharded-cluster-config-server' +type: 'redirect' +code: 301 outputs: - 'v2.2' ... diff --git a/source/administration/sharded-clusters.txt b/source/administration/sharded-clusters.txt index 7c677a92a28..460ea9aef0c 100644 --- a/source/administration/sharded-clusters.txt +++ b/source/administration/sharded-clusters.txt @@ -32,7 +32,11 @@ Sharded Cluster Maintenance and Administration :maxdepth: 1 /tutorial/administer-shard-tags - /tutorial/manage-sharded-cluster-config-server + /tutorial/deploy-config-servers + /tutorial/migrate-config-servers-with-same-hostname + /tutorial/migrate-config-servers-with-different-hostnames + /tutorial/replace-config-server + /tutorial/backup-sharded-cluster-metadata /tutorial/manage-chunks-in-sharded-cluster /tutorial/configure-sharded-cluster-balancer /tutorial/manage-sharded-cluster-balancer diff --git a/source/faq/sharding.txt b/source/faq/sharding.txt index 2603f928ff9..67df2c92be9 100644 --- a/source/faq/sharding.txt +++ b/source/faq/sharding.txt @@ -61,8 +61,7 @@ is to: - restore the dumped data into MongoDB. See :dbcommand:`shardCollection`, :method:`sh.shardCollection()`, -:doc:`/administration/sharded-clusters`, the :ref:`Shard Key -` section in the +the :ref:`Shard Key ` section in the :doc:`/core/sharded-cluster-internals` document, :doc:`/tutorial/deploy-shard-cluster`, and :issue:`SERVER-4000` for more information. @@ -247,7 +246,8 @@ the data will reside only on the new shard. What is the process for moving, renaming, or changing the number of config servers? ----------------------------------------------------------------------------------- -.. see:: :ref:`sharding-procedure-config-server` which describes this process. +See :doc:`/administration/sharded-clusters` for information on +migrating and replacing config servers. When do the ``mongos`` servers detect config server changes? ------------------------------------------------------------ diff --git a/source/tutorial.txt b/source/tutorial.txt index 408f9899439..ba5253cc276 100644 --- a/source/tutorial.txt +++ b/source/tutorial.txt @@ -66,6 +66,11 @@ Sharding - :doc:`/tutorial/convert-replica-set-to-replicated-shard-cluster` - :doc:`/tutorial/add-shards-to-shard-cluster` - :doc:`/tutorial/remove-shards-from-cluster` +- :doc:`/tutorial/deploy-config-servers` +- :doc:`/tutorial/migrate-config-servers-with-same-hostname` +- :doc:`/tutorial/migrate-config-servers-with-different-hostnames` +- :doc:`/tutorial/replace-config-server` +- :doc:`/tutorial/backup-sharded-cluster-metadata` - :doc:`/tutorial/backup-small-sharded-cluster-with-mongodump` - :doc:`/tutorial/backup-sharded-cluster-with-filesystem-snapshots` - :doc:`/tutorial/backup-sharded-cluster-with-database-dumps` diff --git a/source/tutorial/backup-sharded-cluster-metadata.txt b/source/tutorial/backup-sharded-cluster-metadata.txt new file mode 100644 index 00000000000..6c097598234 --- /dev/null +++ b/source/tutorial/backup-sharded-cluster-metadata.txt @@ -0,0 +1,38 @@ +======================= +Backup Cluster Metadata +======================= + +.. default-domain:: mongodb + +This procedure shuts down the :program:`mongod` instance of a +:ref:`config server ` in order to create a +backup of a :doc:`sharded cluster's ` +metadata. The cluster's config servers store all of the cluster's +metadata, most importantly the mapping from :term:`chunks ` to +:term:`shards `. + +When you perform this procedure, the cluster remains operational +[#read-only]_. + +#. Disable the cluster balancer process temporarily. See + :ref:`sharding-balancing-disable-temporally` for more information. + +#. Shut down one of the config databases. + +#. Create a full copy of the data files (i.e. the path specified by + the :setting:`dbpath` option for the config instance.) + +#. Restart the original configuration server. + +#. Re-enable the balancer to allow the cluster to resume normal + balancing operations. See the + :ref:`sharding-balancing-disable-temporally` section for more + information on managing the balancer process. + +.. seealso:: :doc:`/core/backups`. + +.. [#read-only] While one of the three config servers is unavailable, + the cluster cannot split any chunks nor can it migrate chunks + between shards. Your application will be able to write data to the + cluster. The :ref:`sharding-config-server` section of the + documentation provides more information on this topic. diff --git a/source/tutorial/deploy-config-servers.txt b/source/tutorial/deploy-config-servers.txt new file mode 100644 index 00000000000..b3f95724273 --- /dev/null +++ b/source/tutorial/deploy-config-servers.txt @@ -0,0 +1,46 @@ +====================================================== +Deploy Three Config Servers for Production Deployments +====================================================== + +.. default-domain:: mongodb + +This procedure converts a test deployment with only one +:ref:`config server ` to a production deployment +with three config servers. + +For redundancy, all production :doc:`sharded clusters +` should deploy three config servers on three +different machines. Use a single config server only for testing +deployments, never for production deployments. When you shift to +production, upgrade immediately to three config servers. + +To convert a test deployment with one config server to a production +deployment with three config servers: + +#. Shut down all existing MongoDB processes in the cluster. This + includes: + + - all :program:`mongod` instances or :term:`replica sets ` + that provide your shards. + + - all :program:`mongos` instances in your cluster. + +#. Copy the entire :setting:`dbpath` file system tree from the + existing config server to the two machines that will provide the + additional config servers. These commands, issued on the system + with the existing :ref:`config-database`, ``mongo-config0.example.net`` may + resemble the following: + + .. code-block:: sh + + rsync -az /data/configdb mongo-config1.example.net:/data/configdb + rsync -az /data/configdb mongo-config2.example.net:/data/configdb + +#. Start all three config servers, using the same invocation that you + used for the single config server. + + .. code-block:: sh + + mongod --configsvr + +#. Restart all shard :program:`mongod` and :program:`mongos` processes. diff --git a/source/tutorial/manage-sharded-cluster-config-server.txt b/source/tutorial/manage-sharded-cluster-config-server.txt deleted file mode 100644 index 5500830526a..00000000000 --- a/source/tutorial/manage-sharded-cluster-config-server.txt +++ /dev/null @@ -1,246 +0,0 @@ -.. index:: config servers; operations -.. _sharding-procedure-config-server: - -========================= -Manage the Config Servers -========================= - -.. default-domain:: mongodb - -:ref:`Config servers ` store all cluster metadata, most importantly, -the mapping from :term:`chunks ` to :term:`shards `. -This section provides an overview of the basic -procedures to migrate, replace, and maintain these servers. - -This page includes the following: - -- :ref:`sharding-config-server-deploy-three` - -- :ref:`sharding-process-config-server-migrate-same-hostname` - -- :ref:`sharding-process-config-server-migrate-different-hostname` - -- :ref:`sharding-config-server-replace` - -- :ref:`sharding-config-server-backup` - -.. _sharding-config-server-deploy-three: - -Deploy Three Config Servers for Production Deployments ------------------------------------------------------- - -For redundancy, all production :term:`sharded clusters ` -should deploy three config servers processes on three different -machines. - -Do not use only a single config server for production deployments. -Only use a single config server deployments for testing. You should -upgrade to three config servers immediately if you are shifting to -production. The following process shows how to convert a test -deployment with only one config server to production deployment with -three config servers. - -#. Shut down all existing MongoDB processes in the cluster. This - includes: - - - all :program:`mongod` instances or :term:`replica sets ` - that provide your shards. - - - all :program:`mongos` instances in your cluster. - -#. Copy the entire :setting:`dbpath` file system tree from the - existing config server to the two machines that will provide the - additional config servers. These commands, issued on the system - with the existing :ref:`config-database`, ``mongo-config0.example.net`` may - resemble the following: - - .. code-block:: sh - - rsync -az /data/configdb mongo-config1.example.net:/data/configdb - rsync -az /data/configdb mongo-config2.example.net:/data/configdb - -#. Start all three config servers, using the same invocation that you - used for the single config server. - - .. code-block:: sh - - mongod --configsvr - -#. Restart all shard :program:`mongod` and :program:`mongos` processes. - -.. _sharding-process-config-server-migrate-same-hostname: - -Migrate Config Servers with the Same Hostname ---------------------------------------------- - -Use this process when you need to migrate a config server to a new -system but the new system will be accessible using the same host -name. - -#. Shut down the config server that you're moving. - - This will render all config data for your cluster :ref:`read only - `. - -#. Change the DNS entry that points to the system that provided the old - config server, so that the *same* hostname points to the new - system. - - How you do this depends on how you organize your DNS and - hostname resolution services. - -#. Move the entire :setting:`dbpath` file system tree from the old - config server to the new config server. This command, issued on the - old config server system, may resemble the following: - - .. code-block:: sh - - rsync -az /data/configdb mongo-config0.example.net:/data/configdb - -#. Start the config instance on the new system. The default invocation - is: - - .. code-block:: sh - - mongod --configsvr - -When you start the third config server, your cluster will become -writable and it will be able to create new splits and migrate chunks -as needed. - -.. _sharding-process-config-server-migrate-different-hostname: - -Migrate Config Servers with Different Hostnames ------------------------------------------------ - -Use this process when you need to migrate a :ref:`config-database` to a new -server and it *will not* be accessible via the same hostname. If -possible, avoid changing the hostname so that you can use the -:ref:`previous procedure `. - -#. Disable the cluster balancer process temporarily. See - :ref:`sharding-balancing-disable-temporally` for more information. - -#. Shut down the :ref:`config server ` you're moving. - - This will render all config data for your cluster "read only:" - - .. code-block:: sh - - rsync -az /data/configdb mongodb.config2.example.net:/data/configdb - -#. Start the config instance on the new system. The default invocation - is: - - .. code-block:: sh - - mongod --configsvr - -#. Shut down all existing MongoDB processes. This includes: - - - all :program:`mongod` instances or :term:`replica sets ` - that provide your shards. - - - the :program:`mongod` instances that provide your existing - :ref:`config databases `. - - - all :program:`mongos` instances in your cluster. - -#. Restart all :program:`mongod` processes that provide the shard - servers. - -#. Update the :option:`--configdb ` parameter (or - :setting:`configdb`) for all :program:`mongos` instances and - restart all :program:`mongos` instances. - -#. Re-enable the balancer to allow the cluster to resume normal - balancing operations. See the - :ref:`sharding-balancing-disable-temporally` section for more - information on managing the balancer process. - -.. _sharding-config-server-replace: - -Replace a Config Server ------------------------ - -Use this procedure only if you need to replace one of your config -servers after it becomes inoperable (e.g. hardware failure.) This -process assumes that the hostname of the instance will not change. If -you must change the hostname of the instance, use the process for -:ref:`migrating a config server to a different hostname -`. - -#. Disable the cluster balancer process temporarily. See - :ref:`sharding-balancing-disable-temporally` for more information. - -#. Provision a new system, with the same hostname as the previous - host. - - You will have to ensure that the new system has the same IP address - and hostname as the system it's replacing *or* you will need to - modify the DNS records and wait for them to propagate. - -#. Shut down *one* (and only one) of the existing config servers. Copy - all this host's :setting:`dbpath` file system tree from the current system - to the system that will provide the new config server. This - command, issued on the system with the data files, may resemble the - following: - - .. code-block:: sh - - rsync -az /data/configdb mongodb.config2.example.net:/data/configdb - -#. Restart the config server process that you used in the previous - step to copy the data files to the new config server instance. - -#. Start the new config server instance. The default invocation is: - - .. code-block:: sh - - mongod --configsvr - -#. Re-enable the balancer to allow the cluster to resume normal - balancing operations. See the - :ref:`sharding-balancing-disable-temporally` section for more - information on managing the balancer process. - -.. note:: - - In the course of this procedure *never* remove a config server from - the :setting:`configdb` parameter on any of the :program:`mongos` - instances. If you need to change the name of a config server, - always make sure that all :program:`mongos` instances have three - config servers specified in the :setting:`configdb` setting at all - times. - -.. _sharding-config-server-backup: - -Backup Cluster Metadata ------------------------ - -The cluster will remain operational [#read-only]_ without one -of the config database's :program:`mongod` instances, creating a backup -of the cluster metadata from the config database is straight forward: - -#. Disable the cluster balancer process temporarily. See - :ref:`sharding-balancing-disable-temporally` for more information. - -#. Shut down one of the :term:`config databases `. - -#. Create a full copy of the data files (i.e. the path specified by - the :setting:`dbpath` option for the config instance.) - -#. Restart the original configuration server. - -#. Re-enable the balancer to allow the cluster to resume normal - balancing operations. See the - :ref:`sharding-balancing-disable-temporally` section for more - information on managing the balancer process. - -.. seealso:: :doc:`/core/backups`. - -.. [#read-only] While one of the three config servers is unavailable, - the cluster cannot split any chunks nor can it migrate chunks - between shards. Your application will be able to write data to the - cluster. The :ref:`sharding-config-server` section of the - documentation provides more information on this topic. diff --git a/source/tutorial/migrate-config-servers-with-different-hostnames.txt b/source/tutorial/migrate-config-servers-with-different-hostnames.txt new file mode 100644 index 00000000000..08aa0428931 --- /dev/null +++ b/source/tutorial/migrate-config-servers-with-different-hostnames.txt @@ -0,0 +1,53 @@ +=============================================== +Migrate Config Servers with Different Hostnames +=============================================== + +.. default-domain:: mongodb + +This procedure migrates a :ref:`config server ` +in a :doc:`sharded cluster ` +to a new server that uses a different hostname. Use this procedure only +if the config server *will not* be accessible via the same hostname. +If possible, avoid changing the hostname so that you can instead use the +the procedure to :doc:`migrate a config server and use the same hostname +`. + +#. Disable the cluster balancer process temporarily. See + :ref:`sharding-balancing-disable-temporally` for more information. + +#. Shut down the :ref:`config server ` you are moving. + + This will render all config data for your cluster "read only:" + + .. code-block:: sh + + rsync -az /data/configdb mongodb.config2.example.net:/data/configdb + +#. Start the config instance on the new system. The default invocation + is: + + .. code-block:: sh + + mongod --configsvr + +#. Shut down all existing MongoDB processes. This includes: + + - all :program:`mongod` instances or :term:`replica sets ` + that provide your shards. + + - the :program:`mongod` instances that provide your existing + :ref:`config databases `. + + - all :program:`mongos` instances in your cluster. + +#. Restart all :program:`mongod` processes that provide the shard + servers. + +#. Update the :option:`--configdb ` parameter (or + :setting:`configdb`) for all :program:`mongos` instances and + restart all :program:`mongos` instances. + +#. Re-enable the balancer to allow the cluster to resume normal + balancing operations. See the + :ref:`sharding-balancing-disable-temporally` section for more + information on managing the balancer process. diff --git a/source/tutorial/migrate-config-servers-with-same-hostname.txt b/source/tutorial/migrate-config-servers-with-same-hostname.txt new file mode 100644 index 00000000000..1f43a6d1972 --- /dev/null +++ b/source/tutorial/migrate-config-servers-with-same-hostname.txt @@ -0,0 +1,40 @@ +============================================= +Migrate Config Servers with the Same Hostname +============================================= + +.. default-domain:: mongodb + +This procedure migrates a :ref:`config server ` +in a :doc:`sharded cluster ` +to a new system that uses *the same* hostname. + +#. Shut down the config server that you are moving. + + This will render all config data for your cluster :ref:`read only + `. + +#. Change the DNS entry that points to the system that provided the old + config server, so that the *same* hostname points to the new + system. + + How you do this depends on how you organize your DNS and + hostname resolution services. + +#. Move the entire :setting:`dbpath` file system tree from the old + config server to the new config server. This command, issued on the + old config server system, may resemble the following: + + .. code-block:: sh + + rsync -az /data/configdb mongo-config0.example.net:/data/configdb + +#. Start the config instance on the new system. The default invocation + is: + + .. code-block:: sh + + mongod --configsvr + +When you start the third config server, your cluster will become +writable and it will be able to create new splits and migrate chunks +as needed. diff --git a/source/tutorial/replace-config-server.txt b/source/tutorial/replace-config-server.txt new file mode 100644 index 00000000000..000956d8713 --- /dev/null +++ b/source/tutorial/replace-config-server.txt @@ -0,0 +1,59 @@ +======================= +Replace a Config Server +======================= + +.. default-domain:: mongodb + +This procedure replaces an inoperable +:ref:`config server ` in a +:doc:`sharded cluster `. Use this procedure only +to replace a config server that has become inoperable (e.g. hardware +failure). + +This process assumes that the hostname of the instance will not change. +If you must change the hostname of the instance, use the procedure to +:doc:`migrate a config server and use a new hostname +`. + +#. Disable the cluster balancer process temporarily. See + :ref:`sharding-balancing-disable-temporally` for more information. + +#. Provision a new system, with the same hostname as the previous + host. + + You will have to ensure that the new system has the same IP address + and hostname as the system it's replacing *or* you will need to + modify the DNS records and wait for them to propagate. + +#. Shut down *one* (and only one) of the existing config servers. Copy + all this host's :setting:`dbpath` file system tree from the current system + to the system that will provide the new config server. This + command, issued on the system with the data files, may resemble the + following: + + .. code-block:: sh + + rsync -az /data/configdb mongodb.config2.example.net:/data/configdb + +#. Restart the config server process that you used in the previous + step to copy the data files to the new config server instance. + +#. Start the new config server instance. The default invocation is: + + .. code-block:: sh + + mongod --configsvr + +#. Re-enable the balancer to allow the cluster to resume normal + balancing operations. See the + :ref:`sharding-balancing-disable-temporally` section for more + information on managing the balancer process. + +.. note:: + + In the course of this procedure *never* remove a config server from + the :setting:`configdb` parameter on any of the :program:`mongos` + instances. If you need to change the name of a config server, + always make sure that all :program:`mongos` instances have three + config servers specified in the :setting:`configdb` setting at all + times.