Skip to content

DOCS-14598: Address eMRC=true PSA cache pressure #5528

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 22, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config/redirects
Original file line number Diff line number Diff line change
Expand Up @@ -1821,6 +1821,7 @@ raw: /manual/core/wildcard -> ${base}/manual/core/index-wildcard/
[v3.6-v4.4]: /${version}/core/sharding-change-shard-key-value.txt -> ${base}/${version}/core/sharding-shard-key.txt
[v3.6-v4.4]: /${version}/core/sharding-set-missing-shard-key-fields.txt -> ${base}/${version}/core/sharding-shard-key.txt
[v3.6-v4.4]: /${version}/core/sharding-find-shard-key.txt -> ${base}/${version}/core/sharding-shard-key.txt
[v5.0-*]: /${version}/reference/read-concern-majority/#disable-read-concern-majority -> ${base}/${version}/reference/read-concern-majority/#primary-secondary-arbiter-replica-sets

#
# Upgrade / Downgrade redirects
Expand Down
5 changes: 5 additions & 0 deletions source/administration/replica-set-maintenance.txt
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,10 @@ replica sets.
primary-secondary-arbiter (PSA) replica set or on a replica set that
is changing to a PSA architecture.

:doc:`/tutorial/mitigate-psa-performance-issues`
Reduce cache pressure and increased write traffic for a deployment
that has a three-member primary-secondary-arbiter (PSA) architecture.

.. toctree::
:titlesonly:
:hidden:
Expand All @@ -72,3 +76,4 @@ replica sets.
/tutorial/configure-replica-set-secondary-sync-target
/tutorial/rename-unsharded-replica-set
/tutorial/modify-psa-replica-set-safely
/tutorial/mitigate-psa-performance-issues
5 changes: 5 additions & 0 deletions source/core/replica-set-arbiter.txt
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ number of votes to break a tie:

.. end-content-even-votes-example

Performance Issues with PSA replica sets
----------------------------------------

.. include:: /includes/fact-psa-performance-issues.rst

Replica Set Protocol Version and Arbiter
-----------------------------------------

Expand Down
5 changes: 5 additions & 0 deletions source/includes/fact-psa-performance-issues.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
If you are using a three-member primary-secondary-arbiter (PSA)
architecture, the write concern :writeconcern:`"majority"` can cause
performance issues if a secondary is unavailable or lagging. See
:ref:`performance-issues-psa` for advice on how to mitigate these
issues.
4 changes: 4 additions & 0 deletions source/reference/configuration-options.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4165,6 +4165,10 @@ LDAP Parameters
and is always set to ``true``. In earlier versions of MongoDB,
:setting:`~replication.enableMajorityReadConcern` was configurable.

.. warning::

.. include:: /includes/fact-psa-performance-issues.rst


``sharding`` Options
~~~~~~~~~~~~~~~~~~~~
Expand Down
13 changes: 9 additions & 4 deletions source/reference/program/mongod.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1837,12 +1837,17 @@ Replication Options
*Default*: true

Configures support for :readconcern:`"majority"` read concern.
Starting in MongoDB 5.0,
:option:`--enableMajorityReadConcern` cannot be changed
and is always set to ``true``. In earlier versions of MongoDB,

Starting in MongoDB 5.0,
:option:`--enableMajorityReadConcern` cannot be changed
and is always set to ``true``. In earlier versions of MongoDB,
:option:`--enableMajorityReadConcern` was configurable.

.. warning::

.. include:: /includes/fact-psa-performance-issues.rst


Sharded Cluster Options
~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
20 changes: 14 additions & 6 deletions source/reference/read-concern-majority.txt
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,10 @@ Availability
Read concern :readconcern:`"majority"` is available for use with or
without causally consistent sessions and transactions.

.. warning::

.. include:: /includes/fact-psa-performance-issues.rst

Example
-------

Expand Down Expand Up @@ -130,11 +134,15 @@ Read Your Own Writes

.. include:: /includes/fact-read-own-writes.rst

.. _disable-read-concern-majority:

Disable Read Concern Majority
-----------------------------

*For 3-Member Primary-Secondary-Arbiter Architecture*
.. _disable-read-concern-majority:


.. If you change this headline please also change the redirect that
.. links from #disable-read-concern-majority to this headline.

Primary-Secondary-Arbiter Replica Sets
--------------------------------------

.. include:: /includes/fact-eMRC-always-true-in-5.0.rst

.. include:: /includes/fact-psa-performance-issues.rst
2 changes: 2 additions & 0 deletions source/release-notes/5.0-compatibility.txt
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,8 @@ Replica Sets

.. include:: /includes/fact-eMRC-always-true-in-5.0.rst

.. include:: /includes/fact-psa-performance-issues.rst

``secondaryDelaySecs`` Configuration Setting
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
2 changes: 2 additions & 0 deletions source/release-notes/5.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,8 @@ limited to an integer between ``0`` and ``255`` inclusive.

.. include:: /includes/fact-eMRC-always-true-in-5.0.rst

.. include:: /includes/fact-psa-performance-issues.rst

Enhanced Thread Pool Timeout Control
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
5 changes: 5 additions & 0 deletions source/tutorial/add-replica-set-arbiter.txt
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,11 @@ server or a monitoring host.
Considerations
--------------

Primary-Secondary-Arbiter Replica Sets
--------------------------------------

.. include:: /includes/fact-psa-performance-issues.rst

Replica Set Protocol Version
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
79 changes: 79 additions & 0 deletions source/tutorial/mitigate-psa-performance-issues.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
.. _performance-issues-psa:

================================================
Mitigate Performance Issues with PSA Replica Set
================================================

.. default-domain:: mongodb

.. contents:: On this page
:local:
:backlinks: none
:depth: 1
:class: singlecol

Overview
--------

In a three-member replica set with a primary-secondary-arbiter (PSA)
architecture or a sharded cluster with three-member PSA shards, a
data-bearing node that is down or lagged can lead to performance issues.

If one data-bearing node goes down, the other node becomes the primary.
Writes with :writeconcern:`w:1 <\<number\>>` continue to succeed in this
state but writes with write concern :writeconcern:`"majority"` cannot
succeed and the commit point starts to lag. If your PSA replica set
contains a lagged secondary and your replica set requires two nodes to
majority commit a change, your commit point also lags.

With a lagged commit point, two things can affect your cluster
performance:

- The storage engine keeps **all** changes that happen after the commit
point on disk to retain a :term:`durable` history. The extra I/O from
these writes tends to increase over time. This can greatly impact
write performance and increase cache pressure.
- MongoDB allows the :ref:`oplog <replica-set-oplog>` to grow past its
configured size limit to avoid deleting the :data:`majority commit
point <replSetGetStatus.optimes.lastCommittedOpTime>`.

To reduce the cache pressure and increased write traffic, set
:rsconf:`votes: 0 <members[n].votes>` and :rsconf:`priority: 0
<members[n].priority>` for the node that is unavailable or lagging. For
write operations issued with "majority", only voting members are
considered to determine the number of nodes needed to perform a majority
commit. Setting the configuration of the node to :rsconf:`votes: 0
<members[n].votes>` reduces the number of nodes required to commit a
write with write concern :writeconcern:`"majority"` from two to one and
allows these writes to succeed.

Once the secondary is caught up, you can use the
:method:`rs.reconfigForPSASet()` method to set :rsconf:`votes
<members[n].votes>` back to ``1``.

.. note::

In earlier versions of MongoDB,
:setting:`~replication.enableMajorityReadConcern` and
:option:`--enableMajorityReadConcern` were configurable allowing you
to disable the default read concern :readconcern:`"majority"` which
had a similar effect.

Procedure
---------

To reduce the cache pressure and increased write traffic for a
deployment with a three-member primary-secondary-arbiter (PSA)
architecture, set ``{ votes: 0, priority: 0 }`` for the secondary that
is unavailable or lagging:

.. code-block:: javascript

cfg = rs.conf();
cfg["members"][<array_index>]["votes"] = 0;
cfg["members"][<array_index>]["priority"] = 0;
rs.reconfig(cfg);


If you want to change the configuration of the secondary later, use the
:method:`rs.reconfigForPSASet()` method.