diff --git a/config/sphinx_local.yaml b/config/sphinx_local.yaml index 57e519e89fa..9b848908c95 100644 --- a/config/sphinx_local.yaml +++ b/config/sphinx_local.yaml @@ -38,6 +38,7 @@ theme: - /storage - /administration/production-checklist - /contributors/getting-started + - /core/journaling sidebars: '**': - 'pagenav.html' diff --git a/source/core/journaling.txt b/source/core/journaling.txt index 5a8c4bf866f..9e56d5aec6d 100644 --- a/source/core/journaling.txt +++ b/source/core/journaling.txt @@ -253,3 +253,10 @@ there is no separate journal. Write operations with a write concern of :writeconcern:`j: true ` are immediately acknowledged. .. seealso:: :ref:`In-Memory Storage Engine: Durability ` + +.. class:: hidden + + .. toctree:: + :titlesonly: + + /tutorial/manage-journaling diff --git a/source/includes/extracts-manage-journaling.yaml b/source/includes/extracts-manage-journaling.yaml new file mode 100644 index 00000000000..081e7ae73f7 --- /dev/null +++ b/source/includes/extracts-manage-journaling.yaml @@ -0,0 +1,186 @@ +ref: journaling-intro +content: | + MongoDB uses *write ahead logging* to an on-disk :term:`journal` to + guarantee :doc:`write operation ` durability. + The MMAPv1 storage engine also requires the `journal` in order to + provide crash resiliency. + + The WiredTiger storage engine does not require journaling to guarantee + a consistent state after a crash. The database will be restored to the + last consistent :ref:`checkpoint ` + during recovery. However, if MongoDB exits unexpectedly in between + checkpoints, journaling is required to recover writes that occurred + after the last checkpoint. + + With journaling enabled, if :program:`mongod` stops unexpectedly, the + program can recover everything written to the journal. MongoDB will + re-apply the write operations on restart and maintain a consistent + state. By default, the greatest extent of lost writes, i.e., those not + made to the journal, are those made in the last 100 milliseconds, plus + the time it takes to perform the actual journal writes. See + :setting:`~storage.journal.commitIntervalMs` for more information on + the default. +--- +ref: journaling-enable-journaling +content: | + Enable Journaling + ~~~~~~~~~~~~~~~~~ + + .. versionchanged:: 2.0 + For 64-bit builds of :program:`mongod`, journaling is enabled by + default. + + To enable journaling, start :program:`mongod` with the + :option:`--journal ` command line option. +--- +ref: journaling-disable-journaling +content: | + Disable Journaling + ~~~~~~~~~~~~~~~~~~ + + .. warning:: + + Do not disable journaling on production systems. When using the + MMAPv1 storage engine *without* a journal, if your + :program:`mongod` instance stops without shutting down cleanly + unexpectedly for any reason, (e.g. power failure) and you are not + running with journaling, then you must recover from an unaffected + :term:`replica set` member or backup, as described in :doc:`repair + `. + + To disable journaling, start :program:`mongod` with the + :option:`--nojournal ` command line option. +--- +ref: journaling-get-commit-acknowledgment +content: | + Get Commit Acknowledgment + ~~~~~~~~~~~~~~~~~~~~~~~~~ + + You can get commit acknowledgment with the :ref:`write-concern` and + the :writeconcern:`j` option. For details, see + :ref:`write-concern-operation`. +--- +ref: journaling-avoid-preallocation-lag-mmapv1 +content: | + Avoid Preallocation Lag for MMAPv1 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + With the :doc:`MMAPv1 storage engine `, MongoDB may + preallocate journal files if the :program:`mongod` process determines + that it is more efficient to preallocate journal files than create new + journal files as needed. + + Depending on your filesystem, you might experience a preallocation lag + the first time you start a :program:`mongod` instance with journaling + enabled. The amount of time required to pre-allocate files might last + several minutes; during this time, you will not be able to connect to + the database. This is a one-time preallocation and does not occur with + future invocations. + + To avoid :ref:`preallocation lag `, you can + preallocate files in the journal directory by copying them from + another instance of :program:`mongod`. + + Preallocated files do not contain data. It is safe to later remove + them. But if you restart :program:`mongod` with journaling, + :program:`mongod` will create them again. + + .. example:: The following sequence preallocates journal files for an + instance of :program:`mongod` running on port ``27017`` with a + database path of ``/data/db``. + + For demonstration purposes, the sequence starts by creating a set + of journal files in the usual way. + + 1. Create a temporary directory into which to create a set of + journal files: + + .. code-block:: sh + + mkdir ~/tmpDbpath + + #. Create a set of journal files by staring a :program:`mongod` + instance that uses the temporary directory: + + .. code-block:: sh + + mongod --port 10000 --dbpath ~/tmpDbpath --journal + + #. When you see the following log output, indicating + :program:`mongod` has the files, press CONTROL+C to stop the + :program:`mongod` instance: + + .. code-block:: sh + + [initandlisten] waiting for connections on port 10000 + + #. Preallocate journal files for the new instance of + :program:`mongod` by moving the journal files from the data + directory of the existing instance to the data directory of the + new instance: + + .. code-block:: sh + + mv ~/tmpDbpath/journal /data/db/ + + #. Start the new :program:`mongod` instance: + + .. code-block:: sh + + mongod --port 27017 --dbpath /data/db --journal +--- +ref: journaling-monitor-journal-status +content: | + Monitor Journal Status + ~~~~~~~~~~~~~~~~~~~~~~ + + Use the following commands and methods to monitor journal status: + + - :dbcommand:`serverStatus` + + The :dbcommand:`serverStatus` command returns database status + information that is useful for assessing performance. + + - :dbcommand:`journalLatencyTest` + + Use :dbcommand:`journalLatencyTest` to measure how long it takes on + your volume to write to the disk in an append-only fashion. You can + run this command on an idle system to get a baseline sync time for + journaling. You can also run this command on a busy system to see + the sync time on a busy system, which may be higher if the journal + directory is on the same volume as the data files. + + The :dbcommand:`journalLatencyTest` command also provides a way to + check if your disk drive is buffering writes in its local cache. If + the number is very low (i.e., less than 2 milliseconds) and the + drive is non-SSD, the drive is probably buffering writes. In that + case, enable cache write-through for the device in your operating + system, unless you have a disk controller card with battery backed + RAM. +--- +ref: journaling-change-group-commit-interval-mmapv1 +content: | + Change the Group Commit Interval for MMAPv1 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + For the :doc:`MMAPv1 storage engine `, you can set the + group commit interval using the :option:`--journalCommitInterval + ` command line option. The allowed + range is ``2`` to ``300`` milliseconds. + + Lower values increase the durability of the journal at the expense of + disk performance. +--- +ref: journaling-recover-data-unexpected-shutdown +content: | + Recover Data After Unexpected Shutdown + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + On a restart after a crash, MongoDB replays all journal files in the + journal directory before the server becomes available. If MongoDB must + replay journal files, :program:`mongod` notes these events in the log + output. + + There is no reason to run :dbcommand:`repairDatabase` in these + situations. +... diff --git a/source/includes/toc-administration-reference.yaml b/source/includes/toc-administration-reference.yaml index ef7ced989f0..1f499d64bb3 100644 --- a/source/includes/toc-administration-reference.yaml +++ b/source/includes/toc-administration-reference.yaml @@ -21,12 +21,6 @@ description: | Provides an example and a high level overview of the output of the :dbcommand:`serverStatus` command. --- -file: /core/journaling -description: | - Describes the internal operation of MongoDB's journaling facility - and outlines how the journal allows MongoDB to provide provides - durability and crash resiliency. ---- file: /reference/exit-codes description: | Lists the unique codes returned by :program:`mongos` and diff --git a/source/includes/toc-administration-routine.yaml b/source/includes/toc-administration-routine.yaml index 67ffe896121..632a3eee529 100644 --- a/source/includes/toc-administration-routine.yaml +++ b/source/includes/toc-administration-routine.yaml @@ -26,10 +26,10 @@ file: /tutorial/rotate-log-files description: | Archive the current log files and start new ones. --- -file: /tutorial/manage-journaling +file: /tutorial/admin-manage-journaling description: | Describes the procedures for configuring and managing MongoDB's - journaling system which allows MongoDB to provide crash resiliency + journaling system, which allows MongoDB to provide crash resiliency and durability. --- file: /tutorial/store-javascript-function-on-server diff --git a/source/tutorial/admin-manage-journaling.txt b/source/tutorial/admin-manage-journaling.txt new file mode 100644 index 00000000000..28b08e866cc --- /dev/null +++ b/source/tutorial/admin-manage-journaling.txt @@ -0,0 +1,30 @@ +================= +Manage Journaling +================= + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +.. include:: /includes/extracts/journaling-intro.rst + +Procedures +---------- + +.. include:: /includes/extracts/journaling-enable-journaling.rst + +.. include:: /includes/extracts/journaling-disable-journaling.rst + +.. include:: /includes/extracts/journaling-get-commit-acknowledgment.rst + +.. include:: /includes/extracts/journaling-avoid-preallocation-lag-mmapv1.rst + +.. include:: /includes/extracts/journaling-monitor-journal-status.rst + +.. include:: /includes/extracts/journaling-change-group-commit-interval-mmapv1.rst + +.. include:: /includes/extracts/journaling-recover-data-unexpected-shutdown.rst diff --git a/source/tutorial/manage-journaling.txt b/source/tutorial/manage-journaling.txt index c026685ecae..c8f7cf09618 100644 --- a/source/tutorial/manage-journaling.txt +++ b/source/tutorial/manage-journaling.txt @@ -10,174 +10,25 @@ Manage Journaling :depth: 1 :class: singlecol -MongoDB uses *write ahead logging* to an on-disk :term:`journal` to -guarantee :doc:`write operation ` durability. -The MMAPv1 storage engine also requires the `journal` in order to provide -crash resiliency. - -The WiredTiger storage engine does not require journaling to guarantee a -consistent state after a crash. The database will be restored to the last -consistent :ref:`checkpoint ` during recovery. -However, if MongoDB exits unexpectedly in between checkpoints, journaling is -required to recover writes that occurred after the last checkpoint. - -With journaling enabled, if :program:`mongod` stops unexpectedly, the program -can recover everything written to the journal. MongoDB will re-apply the write -operations on restart and maintain a consistent state. By default, the greatest -extent of lost writes, i.e., those not made to the journal, are those made in -the last 100 milliseconds, plus the time it takes to perform the actual journal -writes. See :setting:`~storage.journal.commitIntervalMs` for more information -on the default. +.. include:: /includes/extracts/journaling-intro.rst Procedures ---------- -Enable Journaling -~~~~~~~~~~~~~~~~~ - -.. versionchanged:: 2.0 - For 64-bit builds of :program:`mongod`, journaling is enabled by default. - -To enable journaling, start :program:`mongod` with the -:option:`--journal ` command line option. - -Disable Journaling -~~~~~~~~~~~~~~~~~~ +.. include:: /includes/extracts/journaling-enable-journaling.rst -.. warning:: +.. include:: /includes/extracts/journaling-disable-journaling.rst - Do not disable journaling on production systems. When using the - MMAPv1 storage engine *without* a journal, if your - :program:`mongod` instance stops without shutting down cleanly - unexpectedly for any reason, (e.g. power failure) and you are - not running with journaling, then you must recover from an - unaffected :term:`replica set` member or backup, as described in - :doc:`repair `. - -To disable journaling, start :program:`mongod` with the -:option:`--nojournal ` command line option. - -Get Commit Acknowledgment -~~~~~~~~~~~~~~~~~~~~~~~~~ - -You can get commit acknowledgment with the -:ref:`write-concern` and the ``j`` option. For details, see -:ref:`write-concern-operation`. +.. include:: /includes/extracts/journaling-get-commit-acknowledgment.rst .. _journaling-avoid-preallocation-lag: -Avoid Preallocation Lag for MMAPv1 -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -With the :doc:`MMAPv1 storage engine `, MongoDB may -preallocate journal files if the :program:`mongod` process determines -that it is more efficient to preallocate journal files than create new -journal files as needed. - -Depending on your filesystem, you might experience a preallocation lag -the first time you start a :program:`mongod` instance with journaling -enabled. The amount of time required to pre-allocate files might last -several minutes; during this time, you will not be able to connect to -the database. This is a one-time preallocation and does not occur with -future invocations. - -To avoid :ref:`preallocation lag `, you can -preallocate files in the journal directory by copying them from another -instance of :program:`mongod`. - -Preallocated files do not contain data. It is safe to later remove them. -But if you restart :program:`mongod` with journaling, :program:`mongod` -will create them again. - -.. example:: The following sequence preallocates journal files for an - instance of :program:`mongod` running on port ``27017`` with a database - path of ``/data/db``. - - For demonstration purposes, the sequence starts by creating a set of - journal files in the usual way. - - 1. Create a temporary directory into which to create a set of journal - files: - - .. code-block:: sh - - mkdir ~/tmpDbpath - - #. Create a set of journal files by staring a :program:`mongod` - instance that uses the temporary directory: - - .. code-block:: sh - - mongod --port 10000 --dbpath ~/tmpDbpath --journal +.. include:: /includes/extracts/journaling-avoid-preallocation-lag-mmapv1.rst - #. When you see the following log output, indicating - :program:`mongod` has the files, press CONTROL+C to stop the - :program:`mongod` instance: - - .. code-block:: sh - - [initandlisten] waiting for connections on port 10000 - - #. Preallocate journal files for the new instance of - :program:`mongod` by moving the journal files from the data directory - of the existing instance to the data directory of the new instance: - - .. code-block:: sh - - mv ~/tmpDbpath/journal /data/db/ - - #. Start the new :program:`mongod` instance: - - .. code-block:: sh - - mongod --port 27017 --dbpath /data/db --journal - -Monitor Journal Status -~~~~~~~~~~~~~~~~~~~~~~ - -Use the following commands and methods to monitor journal status: - -- :dbcommand:`serverStatus` - - The :dbcommand:`serverStatus` command returns database status - information that is useful for assessing performance. - -- :dbcommand:`journalLatencyTest` - - Use :dbcommand:`journalLatencyTest` to measure how long it takes on - your volume to write to the disk in an append-only fashion. You can - run this command on an idle system to get a baseline sync time for - journaling. You can also run this command on a busy system to see the - sync time on a busy system, which may be higher if the journal - directory is on the same volume as the data files. - - The :dbcommand:`journalLatencyTest` command also provides a way to - check if your disk drive is buffering writes in its local cache. If - the number is very low (i.e., less than 2 milliseconds) and the drive - is non-SSD, the drive is probably buffering writes. In that case, - enable cache write-through for the device in your operating system, - unless you have a disk controller card with battery backed RAM. +.. include:: /includes/extracts/journaling-monitor-journal-status.rst .. _journaling-journal-commit-interval: -Change the Group Commit Interval for MMAPv1 -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -For the :doc:`MMAPv1 storage engine `, you can set the -group commit interval using the :option:`--journalCommitInterval -` command line option. The allowed -range is ``2`` to ``300`` milliseconds. - -Lower values increase the durability of the journal at the expense of -disk performance. - -Recover Data After Unexpected Shutdown -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -On a restart after a crash, MongoDB replays all journal files in the -journal directory before the server becomes available. If MongoDB must -replay journal files, :program:`mongod` notes these events in the log -output. +.. include:: /includes/extracts/journaling-change-group-commit-interval-mmapv1.rst -There is no reason to run :dbcommand:`repairDatabase` in these -situations. +.. include:: /includes/extracts/journaling-recover-data-unexpected-shutdown.rst