From 2244b448ec0ff1fe868582780306f143ca8151b6 Mon Sep 17 00:00:00 2001 From: Greg Studer Date: Wed, 27 Feb 2013 11:07:52 -0500 Subject: [PATCH] Update source/release-notes/2.4-upgrade.txt Added section about forcing the upgradeLock and retrying for local mongos crashes during upgrade. --- source/release-notes/2.4-upgrade.txt | 49 ++++++++++++++++++++++++---- 1 file changed, 42 insertions(+), 7 deletions(-) diff --git a/source/release-notes/2.4-upgrade.txt b/source/release-notes/2.4-upgrade.txt index 4045a8411ee..c29469a184d 100644 --- a/source/release-notes/2.4-upgrade.txt +++ b/source/release-notes/2.4-upgrade.txt @@ -25,7 +25,7 @@ following procedures. 2. Stop all processes in the cluster. Upgrade all processes to a 2.4-series release of MongoDB, and start all processes at the same - 2.time. + time. .. _upgrade-cluster-upgrade: @@ -65,6 +65,11 @@ procedure. 1. Turn off the :ref:`balancer ` in the :term:`sharded cluster`, as described in :ref:`sharding-balancing-disable-temporally`. + + If you wish, *after* turning off the balancer you may make a backup + copy of the config database by using :program:`mongodump` or other + tools. The upgrade process also creates its own backup collections + which may be removed after the upgrade is verified complete. #. Ensure there are no version 2.0 :program:`mongod` or :program:`mongos` processes still active in the sharded @@ -94,22 +99,52 @@ procedure. The upgrade will prevent any chunk moves or splits from occurring during the upgrade process. If there are very many sharded - collections, acquiring the locks for all collections may take + collections or there are stale locks held by other failed processes, + acquiring the locks for all collections can take seconds or minutes. See the log for progress updates. #. When the :program:`mongos` process starts successfully, the upgrade is complete. If the :program:`mongos` process fails to start, check the log for more information. - If a network interruption occurred and prevented changes, see - :ref:`upgrade-cluster-resync`. + If a network problem occurred or the :program:`mongos` program is somehow + terminated during the upgrade, the upgrade process is generally retryable + if not in the short critical section. + + If on retry the upgrade process waits on the upgrade lock, this means a + previous upgrade process may still be active or has ended abnormally. After + 15 minutes of no remote activity the upgrade lock will be forced, but if + you can verify the upgrade process is no longer active you may force this lock + manually: + + .. code-block:: sh + + mongo <2.2 mongos port> + + .. code-block:: javascript + + db.getMongo().getCollection("config.locks").findOne({ _id : "upgradeLock" }) + + If the host mentioned in the process field of this lock is known to be offline, + you may then run: + + .. code-block:: javascript + + db.getMongo().getCollection("config.locks").update({ _id : "upgradeLock" }, { $set : { state : 0 } }) + + It is safer to wait for the mongos to verify inactivity if you have any doubt + about whether another upgrade is occurring. + + If a network interruption occurred and prevented changes during the critical + section, you will get a message on retry to follow the instructions + here :ref:`upgrade-cluster-resync`. #. :ref:`Re-enable the balancer `. You can now perform operations that modify cluster metadata. - + #. Upgrade and restart other :program:`mongos` processes in the - sharded cluster, without the :option:`--upgrade ` + sharded cluster, *without* the :option:`--upgrade ` option. Once you have upgraded, *do not* introduce version 2.0 MongoDB @@ -138,7 +173,7 @@ To resync the config servers: 1. Turn off the :ref:`balancer ` in the sharded cluster and stop all metadata operations. If you are in the middle of an upgrade process (:ref:`upgrade-cluster-upgrade`), you - have already disabled the balancer. . + have already disabled the balancer. #. Shut down two of the three config servers, preferably the last two listed in the :setting:`configdb` string. For example, if your :setting:`configdb`