Skip to content

DOCSP-7679 Change Initial Sync Semantics #5139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 26, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions source/core/replica-set-elections.txt
Original file line number Diff line number Diff line change
Expand Up @@ -150,14 +150,15 @@ member votes in an election.
- .. include:: /includes/fact-rs-non-voting-priority-restriction.rst

- .. include:: /includes/fact-rs-nonzero-priority-vote-restriction.rst

- Only voting members in the following states are eligible to vote:

- :replstate:`PRIMARY`

- :replstate:`SECONDARY`

- :replstate:`STARTUP2`
- :replstate:`STARTUP2` (unless the member was newly added to the
replica set)

- :replstate:`RECOVERING`

Expand Down
27 changes: 27 additions & 0 deletions source/includes/extracts-initial-sync-semantics.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
ref: initial-sync-semantics
content: |
Starting in MongoDB 5.0, a newly added secondary does not count as a
voting member and cannot be elected until it has reached the
:replstate:`SECONDARY` state.

When a new voting node is added to a replica set,
:dbcommand:`replSetReconfig` will internally add a ``newlyAdded``
field to the node's configuration. Nodes with the ``newlyAdded`` field
do not count towards the current number of voting nodes. When initial
sync completes and the node reaches :replstate:`SECONDARY` state, the
``newlyAdded`` field is automatically removed.

.. note::

- Configurations that attempt to add a field named ``newlyAdded``
will error even if run with ``{ force: true }``.

- If an existing node has a ``newlyAdded`` field, using
:method:`rs.reconfig()` to change the configuration will not
remove the ``newlyAdded`` field. The ``newlyAdded`` field will be
appended to the user provided configuration.

- :dbcommand:`replSetGetConfig` will remove any ``newlyAdded`` fields
from its output. If you would like to see any ``newlyAdded``
fields, you can query the :data:`local.system.replset` collection
directly.
12 changes: 9 additions & 3 deletions source/includes/replica-states.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

* - 1
- :replstate:`PRIMARY`

- The member in state :doc:`primary </core/replica-set-primary>`
is the only member that can accept write operations. Eligible to
vote.
Expand All @@ -38,8 +38,14 @@

* - 5
- :replstate:`STARTUP2`
- The member has joined the set and is running an initial sync. Eligible to
vote.
- The member has joined the set and is running an initial sync.
Eligible to vote.

.. note::

Starting in MongoDB 5.0, if the member was newly added to the
replica set, it is not eligible to vote and cannot be elected
during the initial sync process.

* - 6
- :replstate:`UNKNOWN`
Expand Down
4 changes: 2 additions & 2 deletions source/includes/steps-replace-disabled-config-server.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ ref: add-to-replica-set
pre: |
Connect a :binary:`~bin.mongo` shell to the primary of the config server
replica set and use :method:`rs.add()` to add the new member.
.. tip::

.. warning::

.. include:: /includes/tip-repl-set-add-members.rst

Expand Down
19 changes: 9 additions & 10 deletions source/includes/tip-repl-set-add-members.rst
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
When a newly added secondary has its :rsconf:`~members[n].votes` and
:rsconf:`~members[n].priority` settings greater than zero, during
its initial sync, the secondary still counts as a voting member even
though it cannot serve reads nor become primary because its data is
not yet consistent.

This can lead to a case where a majority of the voting members are
Before MongoDB 5.0, a newly added secondary still counts as a voting
member even though it can neither serve reads nor become primary until
its data is consistent. If you are running a MongoDB version earlier
than 5.0 and add a secondary with its :rsconf:`~members[n].votes`
and :rsconf:`~members[n].priority` settings greater than zero, this can
lead to a case where a majority of the voting members are
online but no primary can be elected. To avoid such situations,
consider adding the new secondary initially with
:rsconf:`priority :0 <members[n].priority>` and :rsconf:`votes :0
<members[n].votes>`. Then, once the member has transitioned into
:replstate:`SECONDARY` state, use :method:`rs.reconfig()` to update its
priority and votes.
<members[n].votes>`. Then, run :method:`rs.status()` to ensure the
member has transitioned into :replstate:`SECONDARY` state. Finally, use
:method:`rs.reconfig()` to update its priority and votes.
5 changes: 5 additions & 0 deletions source/reference/command/replSetReconfig.txt
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,11 @@ Reconfiguration Waits Until a Majority of Members Install the Replica Configurat

.. include:: /includes/extracts/replSetReconfig-majority.rst

Automatic Reconfiguration for New Voting Replica Set Members
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. include:: /includes/extracts/initial-sync-semantics.rst

Access Control
~~~~~~~~~~~~~~

Expand Down
46 changes: 8 additions & 38 deletions source/reference/method/rs.add.txt
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ which will disconnect the shell (such as adding a new member with
a higher priority than the current primary). In such cases, the :binary:`~bin.mongo`
shell may display an error even if the operation succeeds.

.. tip::
.. warning::

.. include:: /includes/tip-repl-set-add-members.rst

Expand Down Expand Up @@ -133,48 +133,18 @@ to a new replica set, you can call the :method:`rs.add()` method with:
Add a Secondary to an Existing Replica Set
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. tip::

.. include:: /includes/tip-repl-set-add-members.rst

To add a new secondary member with default vote and priority settings
Add a new secondary member with default vote and priority settings
to an existing replica set:

#. Add the member initially as a :ref:`non-voting
<replica-set-non-voting-members>`, :doc:`priority 0
</core/replica-set-priority-0-member>` member:

.. cssclass:: copyable-code

.. code-block:: javascript

rs.add( { host: "mongodbd4.example.net:27017", priority: 0, votes: 0 } )

#. Ensure that the new member has reached :replstate:`SECONDARY` state.
To check the state of the replica set members, run
:method:`rs.status()`:

.. cssclass:: copyable-code
.. code-block:: javascript

rs.status()

#. Reconfigure the replica set to update the votes and priority of the
new member:

.. code-block:: javascript

var cfg = rs.conf();
.. cssclass:: copyable-code

cfg.members[n].priority = 1; // Substitute the correct array index for the new member
cfg.members[n].votes = 1; // Substitute the correct array index for the new member
.. code-block:: javascript

rs.reconfig(cfg)
rs.add( { host: "mongodbd4.example.net:27017" } )

where ``n`` is the array index of the new member in the
:rsconf:`members` array.
.. warning::

.. include:: /includes/warning-rs-reconfig.rst
.. include:: /includes/tip-repl-set-add-members.rst

.. _rs-add-priority-0:

Expand Down Expand Up @@ -212,7 +182,7 @@ the host ``mongodb3.example.net`` and accessible on the default port
.. cssclass:: copyable-code

.. code-block:: javascript

rs.add( { host: "mongodb3.example.net:27017", arbiterOnly: true } )

- Host name
Expand Down
4 changes: 4 additions & 0 deletions source/reference/method/rs.reconfig.txt
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,10 @@ modify this timeout using the
Replace ``<hostname>`` and ``<port>`` with those of the removed
member.

Automatic Reconfiguration for New Voting Replica Set Members
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. include:: /includes/extracts/initial-sync-semantics.rst

.. _replica-set-reconfiguration-usage:

Expand Down
6 changes: 5 additions & 1 deletion source/reference/replica-configuration.txt
Original file line number Diff line number Diff line change
Expand Up @@ -419,9 +419,13 @@ Replica Set Configuration Fields
If you need more than 7 members in one replica set, set
:rsconf:`members[n].votes` to ``0`` for the
additional non-voting members.

.. include:: /includes/fact-rs-non-voting-priority-restriction.rst

Starting in MongoDB 5.0, a newly added secondary does not count as
a voting member and cannot be elected until it has reached
:replstate:`SECONDARY` state.

Non-voting members cannot acknowledge write operations
issued with a :writeconcern:`"majority"` write concern.

Expand Down
20 changes: 14 additions & 6 deletions source/reference/replica-states.txt
Original file line number Diff line number Diff line change
Expand Up @@ -52,13 +52,21 @@ Other States

.. replstate:: STARTUP2

Each data-bearing member of a replica set enters the :replstate:`STARTUP2` state as
soon as :binary:`~bin.mongod` finishes loading that member's
configuration, at which time it becomes an active member of the replica set and is eligible to vote.
Each data-bearing member of a replica set enters the
:replstate:`STARTUP2` state as soon as :binary:`~bin.mongod` finishes
loading that member's configuration, at which time it becomes an
active member of the replica set and is eligible to vote.

The member then decides whether or not to undertake an initial sync.
If a member begins an initial sync, the member remains in :replstate:`STARTUP2` until all
data is copied and all indexes are built. Afterwards, the member transitions to
:replstate:`RECOVERING`.
If a member begins an initial sync, the member remains in
:replstate:`STARTUP2` until all data is copied and all indexes are
built. Afterwards, the member transitions to :replstate:`RECOVERING`.

.. note::

Starting in MongoDB 5.0, if the member was newly added to the
replica set, it is not eligible to vote and cannot be elected
during the initial sync process.

.. replstate:: RECOVERING

Expand Down
5 changes: 5 additions & 0 deletions source/release-notes/5.0-compatibility.txt
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,11 @@ operations to the oplog when running as a
:term:`standalone instance <standalone>` should only be done with
guidance from MongoDB Support.

Automatic Reconfiguration for New Voting Replica Set Members
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. include:: /includes/extracts/initial-sync-semantics.rst

Projection Compatibility Changes
--------------------------------

Expand Down
34 changes: 3 additions & 31 deletions source/tutorial/expand-replica-set.txt
Original file line number Diff line number Diff line change
Expand Up @@ -154,43 +154,15 @@ Add a Member to an Existing Replica Set
not know which member is the primary, log into any member of the
replica set and issue the :method:`db.isMaster()` command.

#. Use :method:`rs.add()` to add the new member to the replica set.
#. Use :method:`rs.add()` to add the new member to the replica set.
Pass the :rsconf:`member configuration document <members>` to the
method. For example, to add a member at host
``mongodb3.example.net``, issue the following command:

.. code-block:: javascript

rs.add( { host: "mongodb3.example.net:27017", priority: 0, votes: 0 } )
rs.add( { host: "mongodb3.example.net:27017" } )

.. tip::
.. warning::

.. include:: /includes/tip-repl-set-add-members.rst

#. Ensure that the new member has reached :replstate:`SECONDARY` state.
To check the state of the replica set members, run
:method:`rs.status()`:

.. cssclass:: copyable-code
.. code-block:: javascript

rs.status()

#. Once the newly added member has transitioned into
:replstate:`SECONDARY` state, use :method:`rs.reconfig()` to update
the newly added member's :rsconf:`~members[n].priority` and
:rsconf:`~members[n].votes` if needed.

For example, if :method:`rs.conf()` returns the configuration
document for ``mongodb3.example.net:27017`` as the fifth element in
the :rsconf:`members` array, to update its priority and votes to
``1``, use the following sequence of operations:

.. code-block:: javascript

var cfg = rs.conf();
cfg.members[4].priority = 1
cfg.members[4].votes = 1
rs.reconfig(cfg)

.. include:: /includes/warning-rs-reconfig.rst
15 changes: 12 additions & 3 deletions source/tutorial/resync-replica-set-member.txt
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,11 @@ MongoDB provides two options for performing an initial sync:
Procedures
----------

.. note::

To prevent changing the write quorum, never rotate more than one
replica set member at a time.

.. _replica-set-auto-resync-stale-member:

Automatically Sync a Member
Expand Down Expand Up @@ -89,7 +94,8 @@ perform an initial sync by restarting the instance without the contents of the
:setting:`~storage.dbPath` directory.

4. :doc:`Start the mongod process </tutorial/manage-mongodb-processes>`,
specifying :ref:`cli-mongod-replica-set` as appropriate.
specifying :ref:`cli-mongod-replica-set` with a new
:rsconf:`members[n]._id`.

At this point, the :binary:`~bin.mongod` performs an initial sync. The length of
the initial sync process depends on the size of the database and the network
Expand Down Expand Up @@ -127,5 +133,8 @@ Sync the Member
```````````````

After you have copied the data files from the "seed" source, start the
:binary:`~bin.mongod` instance and allow it to apply all operations from
the oplog until it reflects the current state of the replica set.
:binary:`~bin.mongod` instance with a new :rsconf:`members[n]._id` and
allow it to apply all operations from the oplog until it reflects the
current state of the replica set. To see the current status of the
replica set, use :method:`rs.printSecondaryReplicationInfo()` or
:method:`rs.status()`.
Loading