DOCSP33213 fsync/fsyncUnlock - Backport to v4.4 (#5060)

kennethdyer · Kenneth P. J. Dyer · ajhuh-mdb · web-flow · commit b958e33793e3 · 2023-10-13T11:23:24.000-04:00
* DOCSP-33213 fsync/fsyncUnlock Changes (#4802) * DOCSP-33213 Updates to fsync/fsyncUnlock for Self-Managed Backups of Sharded Clusters * Adds 7.1 versionadded * Reworks Larger Deployment notes * Documents new feature on fsyncUnlock and mongosh helpers * Fixes build issue * Adds lock timeout * Minor edits for cluster coverage * Minor edits for cluster coverage * Adds include for lock/unlock * Adjusts fsync text * Fixes tables * Adds notice for lock/unlock * Minor edit * Major edits to db.fsyncLock to bring it in sync * Minor edit * fsyncUnlock method edits * Minor edits * Minor edits * Adds release note * Fixes build issue * Fixes build issue * Fixes build issue * Fixes build issue * Fixes build issue * Fixes per Ali Co-authored-by: Alison Huh <112565127+ajhuh-mdb@users.noreply.github.com> * Fixes per Ali * Fixes per Ali Co-authored-by: Alison Huh <112565127+ajhuh-mdb@users.noreply.github.com> * Fixes per Nandini * Fixes per Nandini * Fixes per Nandini * Fixes per Nandini * Fixes per Nandini --------- Co-authored-by: Kenneth P. J. Dyer <kenneth.dyer@monodb.com> Co-authored-by: Alison Huh <112565127+ajhuh-mdb@users.noreply.github.com> * fixes build issue * Fixes per Jeff --------- Co-authored-by: Kenneth P. J. Dyer <kenneth.dyer@monodb.com> Co-authored-by: Alison Huh <112565127+ajhuh-mdb@users.noreply.github.com>
diff --git a/source/core/backups.txt b/source/core/backups.txt
@@ -1,3 +1,5 @@
+.. _backup-methods:
+
 ======================
 MongoDB Backup Methods
 ======================
diff --git a/source/includes/fsync-lock-command.rst b/source/includes/fsync-lock-command.rst
@@ -0,0 +1,8 @@
+.. important::
+
+   Servers maintain an fsync lock count.  The :dbcommand:`fsync` command with
+   the ``lock`` field set to ``true`` increments the lock count while the
+   :dbcommand:`fsyncUnlock` command decrements it. To enable writes on a locked
+   server or cluster, call the :dbcommand:`fsyncUnlock` command until the lock
+   count reaches zero.
+   
diff --git a/source/includes/fsync-lock-method.rst b/source/includes/fsync-lock-method.rst
@@ -0,0 +1,6 @@
+
+Servers maintain an fsync lock count.  The :method:`~db.fsyncLock` method
+increments the lock count while the :method:`~db.fsyncUnlock` method decrements
+it. To unlock writes on a server or cluster, call the :method:`~db.fsyncUnlock`
+method until the lock count reaches zero.
+   
diff --git a/source/includes/release-notes/fsync-fsyncUnlock.rst b/source/includes/release-notes/fsync-fsyncUnlock.rst
@@ -0,0 +1,13 @@
+
+Starting in MongoDB 4.4.25, the :dbcommand:`fsync` and :dbcommand:`fsyncUnlock`
+commands can perform fsync operations sharded clusters.
+
+When run on :program:`mongos` with the ``lock`` field set to ``true``, the
+:dbcommand:`fsync` command flushes writes from the storage layer to disk and
+locks each shard, preventing additional writes. The :dbcommand:`fsyncUnlock`
+command can then be used to unlock the cluster. 
+
+This feature enables self-managed backups of sharded clusters using
+:program:`mongodump`.
+
+
diff --git a/source/reference/command/fsync.txt b/source/reference/command/fsync.txt
@@ -19,70 +19,81 @@ Definition
 
 .. dbcommand:: fsync
 
-   Forces the :binary:`~bin.mongod` process to flush all pending writes
-   from the storage layer to disk and locks the *entire*
-   :binary:`~bin.mongod` instance to prevent additional writes until the
-   user releases the lock with a corresponding
-   :dbcommand:`fsyncUnlock`. Optionally, you can use :dbcommand:`fsync`
-   to lock the :binary:`~bin.mongod` instance and block write operations
-   for the purpose of capturing backups.
-
-   As applications write data, MongoDB records the data in the storage
-   layer and then writes the data to disk within the :setting:`~storage.syncPeriodSecs`
-   interval, which is 60 seconds by default. Run :dbcommand:`fsync` when
-   you want to flush writes to disk ahead of that interval.
+   Flushes all pending writes from the storage layer to disk.  When the ``lock``
+   field is set to ``true``, it sets a lock on the server or cluster to prevent
+   additional writes until the lock is released.
    
 
+   .. versionadded:: 4.4.25
+
+      When the ``fsync`` command runs on :program:`mongos`, it performs the
+      fsync operation on each shard in the cluster.
+
+
+   As applications write data, MongoDB records the data in the storage layer
+   and then writes the data to disk within the
+   :setting:`~storage.syncPeriodSecs` interval, which is 60 seconds by default.
+   Run ``fsync`` when you want to flush writes to disk ahead of that interval.
+
+   .. include:: /includes/fsync-lock-command
+
+   Use this command to block writes when you want to perform backup
+   operations.
+
+   .. |method| replace:: :method:`db.fsyncLock` helper method
+   .. include:: /includes/fact-dbcommand-tip
+ 
+
    The :dbcommand:`fsync` command has the following syntax:
 
    .. code-block:: javascript
 
-      { fsync: 1, lock: <Boolean>, comment: <any> }
+      db.adminCommand(
+         { 
+           fsync: 1, 
+           lock: <Boolean>, 
+           fsyncLockAcquisitionTimeout: <integer>,
+           comment: <any> 
+         }
+      )
 
    The :dbcommand:`fsync` command has the following fields:
 
 
    .. list-table::
       :header-rows: 1
-      :widths: 20 20 80
-   
+      :widths: 20 20 60 
+    
       * - Field
-   
         - Type
-   
         - Description
-   
+    
       * - ``fsync``
+        - integer
+        - Enter "1" to apply :dbcommand:`fsync`.
    
+      * - ``fsyncLockAcquisitionTimeoutMillis``     
         - integer
+        - Optional. Specifies the amount of time in milliseconds to wait to
+          acquire locks.  If the lock acquisition operation times out, the
+          command returns a failed response.
    
-        - Enter "1" to apply :dbcommand:`fsync`.
-           
-      * - ``lock``
+          Default: ``90000``
    
-        - boolean
+          .. versionadded:: 4.4.25
    
-        - Optional. Takes a lock on the :binary:`~bin.mongod` instance and blocks all
-          write operations. Each :dbcommand:`fsync` with ``lock`` operation
+      * - ``lock``
+        - boolean
+        - Optional. Takes a lock on the server or cluster and blocks all
+          write operations. Each ``fsync`` with ``lock`` operation
           takes a lock.
           
-          
       * - ``comment``
-
         - any
-
         - .. include:: /includes/extracts/comment-content.rst
        
           .. versionadded:: 4.4
-
-
-   To run the :dbcommand:`fsync` command, use the
-   :method:`db.adminCommand()` method:
-
-   .. code-block:: javascript
-
-      db.adminCommand( { fsync: 1, ... } )
-
+    
 Considerations
 --------------
 
@@ -91,61 +102,60 @@ Considerations
 Impact on Larger Deployments
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-An :dbcommand:`fsync` lock is only possible on *individual*
-:binary:`~bin.mongod` instances of a
-sharded cluster, not on the entire cluster. To back up an entire sharded
-cluster, please see :doc:`/administration/backup-sharded-clusters` for
-more information.
+.. versionadded:: 4.4.25
 
-Alternatives with Journaling
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+When the ``fsync`` command runs on :program:`mongos`, it performs the fsync
+operation on the entire cluster.  By setting the ``lock`` field to ``true``,
+it sets a lock on the cluster, preventing additional writes. 
 
-If your :binary:`~bin.mongod` has :term:`journaling <journal>` enabled,
-please use :ref:`file system or volume/block level snapshot tool <backup-with-journaling>` to create a
-backup of the data set and the journal together as a single unit.
+To take a usable self-managed backup, before locking a sharded cluster:
 
+- Ensure that no chunk migration, resharding, or DDL operations are active.
 
-``fsync`` with ``lock: true``
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+- Stop the balancer to prevent additional chunk migrations from starting.
 
-.. versionchanged:: 3.4
+Alternatives with Journaling
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-   The ``{ fsync: 1, lock: true }`` command now returns a ``lockCount``
-   in the return document.
+If your :program:`mongod` has :term:`journaling <journal>` enabled, use
+:ref:`a file system or volume/block level snapshot tool <backup-with-journaling>`
+to create a backup of the data set and the journal together as a single unit.
 
-After ``{ fsync: 1, lock: true }`` runs on a :binary:`~bin.mongod`, all
-write operations will block. The :binary:`~bin.mongo` shell provides a
-helper method :method:`db.fsyncLock()`.
 
+Lock Count
+~~~~~~~~~~
 
-.. note::
+The ``fsync`` command returns a document includes a ``lockCount`` field. When
+run on :program:`mongod`, the count indicates the number of fsync locks set on
+the server.  
 
-   The ``{ fsync: 1, lock: true }`` operation maintain a lock count.
-   Each ``{ fsync: 1, lock: true }`` operation increments the lock
-   count.
+When run on a sharded cluster, :program:`mongos` sends the fsync operation to
+each shard and returns the results, which includes the ``lockCount`` for each.
 
-   To unlock a :binary:`~bin.mongod` instance for writes, the lock count
-   must be zero. That is, for a given number of ``{ fsync: 1, lock:
-   true }`` operation, you must issue a corresponding number of unlock
-   operations in order to unlock the instance for writes. To unlock,
-   see :method:`db.fsyncUnlock()`.
+.. note::
+
+   If the ``lockCount`` field is non-zero, all writes are blocked on the server
+   and cluster. To reduce the lock count, use the :dbcommand:`fsyncUnlock`
+   command.
 
 Examples
 --------
 
-Lock ``mongod`` Instance
-~~~~~~~~~~~~~~~~~~~~~~~~
+Fsync Lock
+~~~~~~~~~~
 
 .. note::
 
    .. include:: /includes/extracts/wt-fsync-lock-compatibility-command.rst
 
-The primary use of :dbcommand:`fsync` is to lock the :binary:`~bin.mongod`
-instance in order to back up the files within :binary:`~bin.mongod`\ 's :setting:`~storage.dbPath`.
-The operation flushes all data to the storage layer and
-blocks all write operations until you unlock the :binary:`~bin.mongod` instance.
+The ``fsync`` command can lock an individual :program:`mongod` instance or a
+sharded cluster through :program:`mongos`.  When run with the ``lock`` field
+set to ``true``, the fsync operation flushes all data to the storage layer and
+blocks all additional write operations until you unlock the instance or
+cluster.
 
-To lock the database, use the ``lock`` field set to ``true``:
+To lock the database, use the ``fsync`` command to set the ``lock`` field
+to ``true``:
 
 .. code-block:: javascript
 
@@ -163,37 +173,38 @@ operation and the ``lockCount``:
       "ok" : 1
    }
 
-You may continue to perform read operations on a :binary:`~bin.mongod` instance that has a
-:dbcommand:`fsync` lock. However, after the first write operation all
-subsequent read operations wait until you unlock the :binary:`~bin.mongod` instance.
+When locked, write operations are blocked.  Separate connections may continue
+read operations until the first attempt at a write operation, then they also
+wait until the sever or cluster is unlocked.
+
 
 .. important::
 
-   The ``{ fsync: 1, lock: true }`` operation maintain a lock count.
+   The fsync lock operation maintains a lock count.
 
-   To unlock a :binary:`~bin.mongod` instance for writes, the lock count
-   must be zero. That is, for a given number of ``{ fsync: 1, lock:
-   true }`` operation, you must issue a corresponding number of unlock
-   operations in order to unlock the instance for writes.
+   To unlock a server or cluster for writes, the lock count
+   must be zero. That is, for the given number of times you perform an fsync
+   lock, you must issue a corresponding number of unlock operations to unlock
+   the server or cluster for writes.
 
-Unlock ``mongod`` Instance
-~~~~~~~~~~~~~~~~~~~~~~~~~~
+Fsync Unlock 
+~~~~~~~~~~~~
 
-To unlock the :binary:`~bin.mongod`, use :method:`db.fsyncUnlock()`:
+To unlock a server of cluster, use the :dbcommand:`fsyncUnlock` command:
 
 .. code-block:: javascript
 
-   db.fsyncUnlock();
+   db.adminCommnad( { fsyncUnlock: 1 } )
 
-Repeat the :method:`db.fsyncUnlock()` to reduce the lock count to zero
-to unlock the instance for writes.
+Repeat this command as many times as needed to reduce the lock count to zero.
+Once the lock count reaches zero, the server or cluster can resume writes.
 
 Check Lock Status
 ~~~~~~~~~~~~~~~~~
 
 To check the state of the fsync lock, use :method:`db.currentOp()`. Use
-the following JavaScript function in the shell to test if :binary:`~bin.mongod` instance is
-currently locked:
+the following JavaScript function in the shell to test if the server or
+cluster is currently locked:
 
 .. code-block:: javascript
 
@@ -212,5 +223,6 @@ call it, with the following syntax:
 
    serverIsLocked()
 
-This function will return ``true`` if the :binary:`~bin.mongod` instance is
-currently locked and ``false`` if the :binary:`~bin.mongod` is not locked.
+This function will return ``true`` if the server or cluster is
+currently locked and ``false`` if the server or cluster is not locked.
+
diff --git a/source/reference/command/fsyncUnlock.txt b/source/reference/command/fsyncUnlock.txt
@@ -19,29 +19,30 @@ Definition
 
 .. dbcommand:: fsyncUnlock
 
-   Reduces the lock taken by :dbcommand:`fsync` (with the lock option)
-   on a :binary:`~bin.mongod` instance by 1.
+   Reduces the lock count on the server or cluster.  To enable write operations,
+   the lock count must be zero.
+   
 
-   .. important::
+   .. versionadded:: 4.4.25
 
-      The :dbcommand:`fsync` ``lock`` and :dbcommand:`fsyncUnlock`
-      operations maintain a lock count. Each :dbcommand:`fsync` ``lock``
-      operation increments the lock count, and :dbcommand:`fsyncUnlock`
-      decrements the lock count.
+      When the ``fsyncUnlock`` command runs on :program:`mongos`, it
+      reduces the lock count for each shard in the cluster.
 
-      To unlock a :binary:`~bin.mongod` instance for writes, the lock count
-      must be zero. That is, for a given number of :dbcommand:`fsync`
-      ``lock`` operations, you must issue a corresponding number of
-      :dbcommand:`fsyncUnlock` operations to unlock the instance for
-      writes.
+   Use this command to unblock writes after you finish a backup operation.
+
+   .. include:: /includes/fsync-lock-command
 
    :dbcommand:`fsyncUnlock` is an administrative operation. Typically
    you will use :dbcommand:`fsyncUnlock` following a database
    :doc:`backup operation </core/backups>`.
 
+   .. |method| replace:: :method:`db.fsyncUnlock` helper method
+   .. include:: /includes/fact-dbcommand-tip
+ 
    To run the :dbcommand:`fsyncUnlock` command, use the
    :method:`db.adminCommand()` method:
 
+
    .. code-block:: javascript
 
       db.adminCommand( { fsyncUnlock: 1, comment: <any> } )
@@ -54,25 +55,19 @@ Definition
    .. list-table::
       :header-rows: 1
       :widths: 30 70
-
+    
       * - Field
-
         - Description
-
+    
       * - ``info``
         - Information on the status of the operation
-
+    
       * - ``lockCount`` (*New in version 3.4*)
         - The number of locks remaining on the instance after the operation.
-
+    
       * - ``ok``
         - The status code.
-
-   .. tip::
-
-      The :binary:`~bin.mongo` shell provides the helper method
-      :method:`db.fsyncUnlock()`.
-
+ 
 Examples
 --------
 
diff --git a/source/reference/method/db.fsyncLock.txt b/source/reference/method/db.fsyncLock.txt
diff --git a/source/reference/method/db.fsyncUnlock.txt b/source/reference/method/db.fsyncUnlock.txt

Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,5 @@`
	`1`	`+.. _backup-methods:`
	`2`	`+`
`1`	`3`	`======================`
`2`	`4`	`MongoDB Backup Methods`
`3`	`5`	`======================`