From 008081062f15d468a6e5e3af5ad682ddbd3210ef Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Wed, 12 Dec 2012 23:19:13 -0500 Subject: [PATCH 1/8] DOCS-814 journaling, first draft --- source/administration/journaling.txt | 327 +++++++++++++++++++++++++++ source/faq/journaling.txt | 61 +++++ 2 files changed, 388 insertions(+) create mode 100644 source/administration/journaling.txt create mode 100644 source/faq/journaling.txt diff --git a/source/administration/journaling.txt b/source/administration/journaling.txt new file mode 100644 index 00000000000..806cd3a441d --- /dev/null +++ b/source/administration/journaling.txt @@ -0,0 +1,327 @@ +========== +Journaling +========== + +.. default-domain:: mongodb + +:term:`Journaling ` ensures durability of data by storing +:doc:`write operations ` in an on-disk +``journal`` prior to applying them to the database. The ``journal`` +ensures write operations can be re-applied in the event of a crash. + +Journaling also provides early :doc:`read access +` to modified data by applying write operations +to mapped data files in RAM. The data is available for reads even before +MongoDB writes the data to the on-disk data files (i.e., databases). + +Beginning with MongoDB version 2.0, journaling is enabled by default for +64-bit platforms. + +How Journaling Works +-------------------- + +When journaling is enabled, MongoDB stores and applies :doc:`write +operations ` to numerous locations before +the changes are reflected in the your databases. + +This section explains this process in detail. + +.. _journaling-configuring-storage: + +Storage Locations used in Journaling +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Journaling adds three storage locations to MongoDB. + +The ``shared view`` stores modified data for upload to the MongoDB +databases. The ``shared view`` is the only location with direct access +to the MongoDB data files. When you start a :program:`mongod` with +journaling enabled, MongoDB asks the operating system to map your +existing on-disk data files to the ``shared view`` memory location. The +operating system maps the files but does not load them. MongoDB later +loads data files to ``shared view`` as needed. + +The ``private view`` stores data for use in :doc:`read operations +`. The ``private view`` is mapped to the ``shared view`` +and is the first place MongoDB applies new :doc:`write operations +`, mean read operations get the most up-to-date +data. Keep in mind that because the ``private view`` is a second mapping +of data files, journaling often doubles the amount of virtual memory +:program:`mongod` uses. + +The ``journal`` is an on-disk location that stores new write operations +after they have been applied to the ``private cache`` but before they +have been applied to the databases. The ``journal`` provides durability. +If the :program:`mongod` instance were to crash without having applied +the writes to the databases, the ``journal`` could replay the writes to +the ``shared view`` for eventual upload to the databases. + +.. _journaling-record-write-operation: + +How Journaling Records Write Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As users perform :doc:`write operations `, +MongoDB writes the data to the ``private view`` in RAM, making it +immediately available for :doc:`read operations +`. + +MongoDB then copies the write operations in batches from the ``private +view`` to the ``journal``, which stores the operations on disk to ensure +durability. When writing to the ``journal``, MongoDB adds a write operation as +an entry on the ``journal``'s forward pointer. Each entry on the pointer +describes which bytes the write operation changed in the data files. +(The ``journal`` also has a behind pointer, discussed later in this +section.) + +MongoDB copies the write operations to the journal in batches +called group commits. By default, MongoDB performs a group commit every +100 milliseconds, which means a series of operations over 100 +milliseconds are committed as a single batch. This is done to achieve +high performance. + +MongoDB next applies the ``journal``'s write operations to the ``shared +view``. At this point, the ``shared view`` becomes inconsistent with the data files +on disk (i.e., the MongoDB databases). + +At default intervals of 60 seconds, MongoDB asks the operating system to +flush the ``shared view`` to disk. This brings the data files up-to-date +with the latest write operations. + +When write operations are flushed to the databases, MongoDB removes the +write operations from the ``journal``'s behind pointer. The behind pointer +is always far back from advanced pointer. + +As part of journaling, MongoDB routinely asks the operating system to +remap the ``shared view`` to the ``private view``, for consistency. + +.. note:: The interaction between the ``shared view`` and the on-disk + data files is not dissimilar to how MongoDB works *without* + journaling, which is that MongoDB asks the operating system to flush + in-memory changes back to the data files every 60 seconds. + +What Journaling Stores +~~~~~~~~~~~~~~~~~~~~~~ + +Journaling stores: + +- documents +- indexes +- meta data on collections and databases +- journals, which are information about the information stored + +.. _journaling-journal-files: + +Journal Files +~~~~~~~~~~~~~ + +With journaling enabled, MongoDB creates a ``journal`` directory within +your database directory. The ``journal`` directory holds journal files, +which contain write-ahead redo logs. The directory also holds a +last-sequence-number file. A clean shutdown removes all the files in the +``journal`` directory. + +Journal files are append-only files and are named with the ``j._`` +prefix. When a journal file reaches 1 gigabyte, a new file is created. +Files that no longer are needed are automatically deleted. Unless your +write-bytes-per-second rate is extremely high, the directory should +contain only two or three journal files. + +To limit the size of journal files to 128 megabytes per file, use the +:option:`--smallfiles` command line option when starting +:program:`mongod`. + +To speed the frequent sequential writes that occur to the current +journal file, you can symbolically link the ``journal`` directory to a +dedicated hard drive before starting :program:`mongod`. + +In some cases, you might experience a preallocation lag the first time +you start a :program:`mongod` instance with journaling enabled. MongoDB +may determine that it is faster to preallocate journal files than to +create them as needed. This would be the case if it is faster on your +file system to write to files of predefined sizes than to append files. +If MongoDB preallocates the files, you might experience a several +minutes delay on first startup of :program:`mongod`. You will not be +able to connect to the database until the preallocation completes. This +is a one-time preallocation and does not occur with future invocations. +Check the logs to see if MongoDB is preallocating. The logs will display +the standard "waiting for connections on port" message when complete. + +To avoid this lag, see :ref:`journaling-avoid-preallocation-lag`. + +Configuration and Setup +----------------------- + +Enable Journaling +~~~~~~~~~~~~~~~~~ + +Beginning with version 2.0, journaling is enabled by default for 64-bit +platforms. + +To enable journaling, start :program:`mongod` with the +:option:`--journal` command line option. + +If :program:`mongod` decides to preallocate the files, it will not start +listening on port 27017 until this process completes, which can take a +few minutes. This means that your applications and the shell will not be +able to connect to the database immediately on initial startup. Check +the logs to see if MongoDB is busy preallocating. + +Disable Journaling +~~~~~~~~~~~~~~~~~~ + +To disable journaling, start :program:`mongod` with the +:option:`--nojournal ` command line option. + +It is OK to disable journaling after running with journaling. Simply +shut down :program:`mongod` cleanly and restart with +:option:`--nojournal `. + +Get Commit Acknowledgement +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can wait for group commit acknowledgement with the getLastError +Command. In versions before 1.9.0 using getLastError + fsync would do +this, in newer versions the "j" option has been specifically created for +this purpose. + +In version 1.9.2+ the group commit delay is shortened when a commit +acknowledgement (getLastError + j) is pending; this can be as little as +1/3 of the normal group commit interval. + +.. _journaling-avoid-preallocation-lag: + +Avoid Preallocation Lag +~~~~~~~~~~~~~~~~~~~~~~~ + +To avoid preallocation lag, you can preallocate files in the ``journal`` +directory by copying them from another instance of :program:`mongod`. +(For details on preallocation lag, see :ref:`journaling-journal-files`.) + +.. example:: The following sequence of commands preallocates journal + files for an instance of :program:`mongod` running on port ``27017`` + with a database path of ``/data/db``. + + .. code-block:: sh + + $ mkdir ~/tmpDbpath + $ mongod --port 10000 --dbpath ~/tmpDbpath --journal + # startup messages + # . + # . + # . + # wait for prealloc to finish + Thu Mar 17 10:02:52 [initandlisten] preallocating a journal file + ~/tmpDbpath/journal/prealloc.0 + Thu Mar 17 10:03:03 [initandlisten] preallocating a journal file + ~/tmpDbpath/journal/prealloc.1 + Thu Mar 17 10:03:14 [initandlisten] preallocating a journal file + ~/tmpDbpath/journal/prealloc.2 + Thu Mar 17 10:03:25 [initandlisten] flushing directory + ~/tmpDbpath/journal + Thu Mar 17 10:03:25 [initandlisten] flushing directory + ~/tmpDbpath/journal + Thu Mar 17 10:03:25 [initandlisten] waiting for connections on port + 10000 + Thu Mar 17 10:03:25 [websvr] web admin interface listening on port 11000 + # then Ctrl-C to kill this instance + ^C + $ mv ~/tmpDbpath/journal /data/db/ + $ # restart mongod on port 27017 with --journal + +Preallocated files do not contain data. It is safe to remove them. But +if you restart :program:`mongod` with journaling, :program:`mongod` will +create them again. + +Change the Group Commit Interval +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Beginning with version 1.9.2, you can set the group commit interval +using the :option:`--journalCommitInterval ` command line option. The allowed range is ``2`` to +``300`` milliseconds. + +Monitor journal Status +~~~~~~~~~~~~~~~~~~~~~~ + +serverStatus command + +The serverStatus command now includes some statistics regarding +journaling. + +journalLatencyTest Command + +You can use the journalLatencyTest command to measure how long it takes +on your volume to write to the disk (including fsyncing the data) in an +append-only fashion. + +> use admin + +> db.runCommand("journalLatencyTest") + +You can run this command on an idle system to get a baseline sync time +for journaling. In addition, it is safe to run this command on a busy +system to see the sync time on a busy system (which may be higher if the +journal directory is on the same volume as the data files). + +In version 1.9.2+ you can set the group commit interval, using +--journalCommitInterval command-line option, to between 2 and 300 +milliseconds (default is 100ms). The actual interval are the maximum +of this setting and your disk latency as measured above. + +journalLatencyTest is also a good way to check if your disk drive is +buffering writes in its local cache. If the number is very low (e.g., +less than 2ms) and the drive is non-ssd, the drive is probably buffering +writes. In that case, you will want to enable cache write-through for +the device in your operating system. (Unless you have a disk controller +card with battery backed ram, then this is a good thing.) + +Command-line Options +-------------------- + +- `--master`: The :term:`master` mode. + +- :option:`--oplogSize`: This takes an argument and specifies the size + limit in MB for the oplog. + +- `--slave`: The :term:`slave` mode. + +- `--source`: This takes an argument and specifies the master as + . + +- `--only`: This takes an argument and specifies a single database to + replicate. + +Recovery +-------- + +On a restart after a crash, journal files in journal are replayed +before the server goes online. This is indicated in the log output. +You do not need to run a repair. + +With journaling if you want a dataset to reside entirely in RAM, you +need twice as much RAM available as the dataset size to be able to store +the ``shared view`` and ``private view``. + +Recommendations +~~~~~~~~~~~~~~~ + +Recommend to set (or at least check) for a low read ahead value +for the data disks, say 40 blocks. + +– And 0 for non-spinning disks + +Recommend to use a separate disk for the journal entries, with a +slightly higher read ahead, say 100 blocks + +– Writes are always at the end of the journal + +– Deletes are always at the beginning of the journal + +Include checking the read ahead values in onboarding interviews + +Set the read ahead values in the templates we distribute + +Be aware of the issue for sudden performance breakdown tickets + +– Beware of resident memory estimates when diagnosing RAM usage diff --git a/source/faq/journaling.txt b/source/faq/journaling.txt new file mode 100644 index 00000000000..c4e8a46e512 --- /dev/null +++ b/source/faq/journaling.txt @@ -0,0 +1,61 @@ +=============== +FAQ: Journaling +=============== + +.. default-domain:: mongodb + +This document addresses common questions regarding MongoDB journaling. + +If you don't find the answer you're looking for, check +the :doc:`complete list of FAQs ` or post your question to the +`MongoDB User Mailing List `_. + +.. contents:: Frequently Asked Questions: + :backlinks: none + :local: + +If I am using replication, can some members use journaling and others not? +-------------------------------------------------------------------------- +Yes. It is OK to use journaling on some replica set members and not others. + +Can I use the journaling feature to perform safe hot backups? +------------------------------------------------------------- + +Yes, see Backups with Journaling Enabled. + +32 bit nuances? +--------------- + +There is extra memory mapped file activity with journaling. This will +further constrain the limited db size of 32 bit builds. Thus, for now +journaling by default is disabled on 32 bit systems. + +When did the --journal option change from --dur? +------------------------------------------------ + +In 1.8 the option was renamed to --journal, but the old name is still +accepted for backwards compatibility; please change to --journal if you +are using the old option. + +Will the journal replay have problems if entries are incomplete (like the failure happened in the middle of one)? +----------------------------------------------------------------------------------------------------------------- + +Each journal (group) write is consistent and won't be replayed during +recovery unless it is complete. + +How many times is data written to disk when replication and journaling are both on? +----------------------------------------------------------------------------------- + +In v1.8, for an insert, four times. The object is written to the main +collection, and also the oplog collection (so that is twice). Both of +those writes are journaled as a single mini-transaction in the journal +file (the files in /data/db/journal). Thus 4 times total. + +There is an open item in to reduce this by having the journal be +compressed. This will reduce from 4x to probably ~2.5x. + +The above applies to collection data and inserts which is the worst case +scenario. Index updates are written to the index and the journal, but +not the oplog, so they should be 2X today not 4X. Likewise updates with +things like $set, $addToSet, $inc, etc. are compactly logged all around +so those are generally small. From 095827190e6537e3f16fca3c2eab663e03992b1f Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Thu, 13 Dec 2012 13:39:16 -0500 Subject: [PATCH 2/8] minor --- source/administration/journaling.txt | 62 ++++++++++++++-------------- 1 file changed, 31 insertions(+), 31 deletions(-) diff --git a/source/administration/journaling.txt b/source/administration/journaling.txt index 806cd3a441d..567be2ae364 100644 --- a/source/administration/journaling.txt +++ b/source/administration/journaling.txt @@ -6,23 +6,24 @@ Journaling :term:`Journaling ` ensures durability of data by storing :doc:`write operations ` in an on-disk -``journal`` prior to applying them to the database. The ``journal`` +journal prior to applying them to the data files. The journal ensures write operations can be re-applied in the event of a crash. -Journaling also provides early :doc:`read access -` to modified data by applying write operations -to mapped data files in RAM. The data is available for reads even before -MongoDB writes the data to the on-disk data files (i.e., databases). +Journaling ensures that :program:`mongodb` is crash resistent. Without a +journal, if :program:`mongodb` exits unexpectedly, the operators must assume +the data are in an inconsistent state and should resync from a clean +secondary. -Beginning with MongoDB version 2.0, journaling is enabled by default for -64-bit platforms. +.. versionchanged:: 2.0 + + Journaling is enabled by default for 64-bit platforms. How Journaling Works -------------------- -When journaling is enabled, MongoDB stores and applies :doc:`write -operations ` to numerous locations before -the changes are reflected in the your databases. +When running with journaling, MongoDB stores and applies :doc:`write +operations ` in memory and in the journal before +the changes are in the data files. This section explains this process in detail. @@ -34,9 +35,9 @@ Storage Locations used in Journaling Journaling adds three storage locations to MongoDB. The ``shared view`` stores modified data for upload to the MongoDB -databases. The ``shared view`` is the only location with direct access -to the MongoDB data files. When you start a :program:`mongod` with -journaling enabled, MongoDB asks the operating system to map your +data files. The ``shared view`` is the only location with direct access +to the MongoDB data files. When running with journaling, :program:`mongod` +asks the operating system to map your existing on-disk data files to the ``shared view`` memory location. The operating system maps the files but does not load them. MongoDB later loads data files to ``shared view`` as needed. @@ -49,12 +50,12 @@ data. Keep in mind that because the ``private view`` is a second mapping of data files, journaling often doubles the amount of virtual memory :program:`mongod` uses. -The ``journal`` is an on-disk location that stores new write operations +The journal is an on-disk location that stores new write operations after they have been applied to the ``private cache`` but before they -have been applied to the databases. The ``journal`` provides durability. +have been applied to the data files. The journal provides durability. If the :program:`mongod` instance were to crash without having applied -the writes to the databases, the ``journal`` could replay the writes to -the ``shared view`` for eventual upload to the databases. +the writes to the data files, the journal could replay the writes to +the ``shared view`` for eventual upload to the data files. .. _journaling-record-write-operation: @@ -67,11 +68,11 @@ immediately available for :doc:`read operations `. MongoDB then copies the write operations in batches from the ``private -view`` to the ``journal``, which stores the operations on disk to ensure -durability. When writing to the ``journal``, MongoDB adds a write operation as -an entry on the ``journal``'s forward pointer. Each entry on the pointer +view`` to the journal, which stores the operations on disk to ensure +durability. When writing to the journal, MongoDB adds a write operation as +an entry on the journal's forward pointer. Each entry on the pointer describes which bytes the write operation changed in the data files. -(The ``journal`` also has a behind pointer, discussed later in this +(The journal also has a behind pointer, discussed later in this section.) MongoDB copies the write operations to the journal in batches @@ -80,16 +81,15 @@ called group commits. By default, MongoDB performs a group commit every milliseconds are committed as a single batch. This is done to achieve high performance. -MongoDB next applies the ``journal``'s write operations to the ``shared -view``. At this point, the ``shared view`` becomes inconsistent with the data files -on disk (i.e., the MongoDB databases). +MongoDB next applies the journal's write operations to the ``shared +view``. At this point, the ``shared view`` becomes inconsistent with the data files. At default intervals of 60 seconds, MongoDB asks the operating system to flush the ``shared view`` to disk. This brings the data files up-to-date with the latest write operations. -When write operations are flushed to the databases, MongoDB removes the -write operations from the ``journal``'s behind pointer. The behind pointer +When write operations are flushed to the data files, MongoDB removes the +write operations from the journal's behind pointer. The behind pointer is always far back from advanced pointer. As part of journaling, MongoDB routinely asks the operating system to @@ -115,11 +115,11 @@ Journaling stores: Journal Files ~~~~~~~~~~~~~ -With journaling enabled, MongoDB creates a ``journal`` directory within -your database directory. The ``journal`` directory holds journal files, +With journaling enabled, MongoDB creates a journal directory within +your database directory. The journal directory holds journal files, which contain write-ahead redo logs. The directory also holds a last-sequence-number file. A clean shutdown removes all the files in the -``journal`` directory. +journal directory. Journal files are append-only files and are named with the ``j._`` prefix. When a journal file reaches 1 gigabyte, a new file is created. @@ -132,7 +132,7 @@ To limit the size of journal files to 128 megabytes per file, use the :program:`mongod`. To speed the frequent sequential writes that occur to the current -journal file, you can symbolically link the ``journal`` directory to a +journal file, you can symbolically link the journal directory to a dedicated hard drive before starting :program:`mongod`. In some cases, you might experience a preallocation lag the first time @@ -194,7 +194,7 @@ acknowledgement (getLastError + j) is pending; this can be as little as Avoid Preallocation Lag ~~~~~~~~~~~~~~~~~~~~~~~ -To avoid preallocation lag, you can preallocate files in the ``journal`` +To avoid preallocation lag, you can preallocate files in the journal directory by copying them from another instance of :program:`mongod`. (For details on preallocation lag, see :ref:`journaling-journal-files`.) From 709a87961463fb7a8c72432bdc7b19fd25b19510 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Mon, 17 Dec 2012 15:32:49 -0500 Subject: [PATCH 3/8] DOCS-814 minor edits --- source/administration/journaling.txt | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/source/administration/journaling.txt b/source/administration/journaling.txt index 567be2ae364..2997ea2e1af 100644 --- a/source/administration/journaling.txt +++ b/source/administration/journaling.txt @@ -9,14 +9,23 @@ Journaling journal prior to applying them to the data files. The journal ensures write operations can be re-applied in the event of a crash. -Journaling ensures that :program:`mongodb` is crash resistent. Without a -journal, if :program:`mongodb` exits unexpectedly, the operators must assume -the data are in an inconsistent state and should resync from a clean -secondary. +Journaling ensures that :program:`mongodb` is crash resilient. *Without* +a journal, if :program:`mongodb` exits unexpectedly, you must assume +your data is in an inconsistent state and must either run +:doc:`repair ` +or :ref:`resync ` from a clean member +of the replica set. + +With journaling, if :program:`mongodb` stops unexpectedly, the program +can recover everything written to the journal, and the data is in a +consistent state. By default, the greatest extent of lost writes, i.e., +those not made to the journal, is no more than the last 100 +milliseconds. -.. versionchanged:: 2.0 +.. versionchanged:: 2.0 Journaling is enabled by default for 64-bit platforms. + For other platforms, see :setting:`journal`. How Journaling Works -------------------- From c083828bbfad8526800b1d903e6d7b4f0f60f63c Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Mon, 17 Dec 2012 15:58:24 -0500 Subject: [PATCH 4/8] DOCS-814 edits --- source/administration/journaling.txt | 260 +++++++++++++-------------- 1 file changed, 122 insertions(+), 138 deletions(-) diff --git a/source/administration/journaling.txt b/source/administration/journaling.txt index 2997ea2e1af..d32555b994d 100644 --- a/source/administration/journaling.txt +++ b/source/administration/journaling.txt @@ -22,142 +22,10 @@ consistent state. By default, the greatest extent of lost writes, i.e., those not made to the journal, is no more than the last 100 milliseconds. - .. versionchanged:: 2.0 Journaling is enabled by default for 64-bit platforms. For other platforms, see :setting:`journal`. -How Journaling Works --------------------- - -When running with journaling, MongoDB stores and applies :doc:`write -operations ` in memory and in the journal before -the changes are in the data files. - -This section explains this process in detail. - -.. _journaling-configuring-storage: - -Storage Locations used in Journaling -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Journaling adds three storage locations to MongoDB. - -The ``shared view`` stores modified data for upload to the MongoDB -data files. The ``shared view`` is the only location with direct access -to the MongoDB data files. When running with journaling, :program:`mongod` -asks the operating system to map your -existing on-disk data files to the ``shared view`` memory location. The -operating system maps the files but does not load them. MongoDB later -loads data files to ``shared view`` as needed. - -The ``private view`` stores data for use in :doc:`read operations -`. The ``private view`` is mapped to the ``shared view`` -and is the first place MongoDB applies new :doc:`write operations -`, mean read operations get the most up-to-date -data. Keep in mind that because the ``private view`` is a second mapping -of data files, journaling often doubles the amount of virtual memory -:program:`mongod` uses. - -The journal is an on-disk location that stores new write operations -after they have been applied to the ``private cache`` but before they -have been applied to the data files. The journal provides durability. -If the :program:`mongod` instance were to crash without having applied -the writes to the data files, the journal could replay the writes to -the ``shared view`` for eventual upload to the data files. - -.. _journaling-record-write-operation: - -How Journaling Records Write Operations -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -As users perform :doc:`write operations `, -MongoDB writes the data to the ``private view`` in RAM, making it -immediately available for :doc:`read operations -`. - -MongoDB then copies the write operations in batches from the ``private -view`` to the journal, which stores the operations on disk to ensure -durability. When writing to the journal, MongoDB adds a write operation as -an entry on the journal's forward pointer. Each entry on the pointer -describes which bytes the write operation changed in the data files. -(The journal also has a behind pointer, discussed later in this -section.) - -MongoDB copies the write operations to the journal in batches -called group commits. By default, MongoDB performs a group commit every -100 milliseconds, which means a series of operations over 100 -milliseconds are committed as a single batch. This is done to achieve -high performance. - -MongoDB next applies the journal's write operations to the ``shared -view``. At this point, the ``shared view`` becomes inconsistent with the data files. - -At default intervals of 60 seconds, MongoDB asks the operating system to -flush the ``shared view`` to disk. This brings the data files up-to-date -with the latest write operations. - -When write operations are flushed to the data files, MongoDB removes the -write operations from the journal's behind pointer. The behind pointer -is always far back from advanced pointer. - -As part of journaling, MongoDB routinely asks the operating system to -remap the ``shared view`` to the ``private view``, for consistency. - -.. note:: The interaction between the ``shared view`` and the on-disk - data files is not dissimilar to how MongoDB works *without* - journaling, which is that MongoDB asks the operating system to flush - in-memory changes back to the data files every 60 seconds. - -What Journaling Stores -~~~~~~~~~~~~~~~~~~~~~~ - -Journaling stores: - -- documents -- indexes -- meta data on collections and databases -- journals, which are information about the information stored - -.. _journaling-journal-files: - -Journal Files -~~~~~~~~~~~~~ - -With journaling enabled, MongoDB creates a journal directory within -your database directory. The journal directory holds journal files, -which contain write-ahead redo logs. The directory also holds a -last-sequence-number file. A clean shutdown removes all the files in the -journal directory. - -Journal files are append-only files and are named with the ``j._`` -prefix. When a journal file reaches 1 gigabyte, a new file is created. -Files that no longer are needed are automatically deleted. Unless your -write-bytes-per-second rate is extremely high, the directory should -contain only two or three journal files. - -To limit the size of journal files to 128 megabytes per file, use the -:option:`--smallfiles` command line option when starting -:program:`mongod`. - -To speed the frequent sequential writes that occur to the current -journal file, you can symbolically link the journal directory to a -dedicated hard drive before starting :program:`mongod`. - -In some cases, you might experience a preallocation lag the first time -you start a :program:`mongod` instance with journaling enabled. MongoDB -may determine that it is faster to preallocate journal files than to -create them as needed. This would be the case if it is faster on your -file system to write to files of predefined sizes than to append files. -If MongoDB preallocates the files, you might experience a several -minutes delay on first startup of :program:`mongod`. You will not be -able to connect to the database until the preallocation completes. This -is a one-time preallocation and does not occur with future invocations. -Check the logs to see if MongoDB is preallocating. The logs will display -the standard "waiting for connections on port" message when complete. - -To avoid this lag, see :ref:`journaling-avoid-preallocation-lag`. - Configuration and Setup ----------------------- @@ -172,9 +40,8 @@ To enable journaling, start :program:`mongod` with the If :program:`mongod` decides to preallocate the files, it will not start listening on port 27017 until this process completes, which can take a -few minutes. This means that your applications and the shell will not be -able to connect to the database immediately on initial startup. Check -the logs to see if MongoDB is busy preallocating. +few minutes. Your applications and the shell will not be able to connect +to the database until the process completes. Disable Journaling ~~~~~~~~~~~~~~~~~~ @@ -182,9 +49,8 @@ Disable Journaling To disable journaling, start :program:`mongod` with the :option:`--nojournal ` command line option. -It is OK to disable journaling after running with journaling. Simply -shut down :program:`mongod` cleanly and restart with -:option:`--nojournal `. +To disable journaling, shut down :program:`mongod` cleanly and restart +with :option:`--nojournal `. Get Commit Acknowledgement ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -285,6 +151,40 @@ writes. In that case, you will want to enable cache write-through for the device in your operating system. (Unless you have a disk controller card with battery backed ram, then this is a good thing.) +.. _journaling-journal-files: + +Journal Files +~~~~~~~~~~~~~ + +With journaling enabled, MongoDB creates a journal directory within +your database directory. The journal directory holds journal files, +which contain write-ahead redo logs. The directory also holds a +last-sequence-number file. A clean shutdown removes all the files in the +journal directory. + +Journal files are append-only files and are named with the ``j._`` +prefix. When a journal file reaches 1 gigabyte, a new file is created. +Files that no longer are needed are automatically deleted. Unless you +write many bytes of data per-second, the journal directory should +contain only two or three journal files. + +To limit the size of journal files to 128 megabytes per file, use the +``--smallfiles`` command line option when starting :program:`mongod`. + +To speed the frequent sequential writes that occur to the current +journal file, ensure that the journal directory is on a different system. + +Depending on your file system, you might experience a preallocation lag +the first time you start a :program:`mongod` instance with journaling +enabled. MongoDB preallocates journal files if it is faster on your file +system to predefine file size. Preallocation lag might last several +minutes, during which you will not be able to connect to the database. +This is a one-time preallocation and does not occur with future +invocations. + +To avoid preallocation lag, see +:ref:`journaling-avoid-preallocation-lag`. + Command-line Options -------------------- @@ -334,3 +234,87 @@ Set the read ahead values in the templates we distribute Be aware of the issue for sudden performance breakdown tickets – Beware of resident memory estimates when diagnosing RAM usage + +How Journaling Works +-------------------- + +When running with journaling, MongoDB stores and applies :doc:`write +operations ` in memory and in the journal before +the changes are in the data files. + +This section explains this process in detail. + +.. _journaling-configuring-storage: + +Storage Views used in Journaling +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Journaling adds three storage views to MongoDB. + +The ``shared view`` stores modified data for upload to the MongoDB +data files. The ``shared view`` is the only view with direct access +to the MongoDB data files. When running with journaling, :program:`mongod` +asks the operating system to map your existing on-disk data files to the +``shared view`` memory view. The operating system maps the files but +does not load them. MongoDB later loads data files to ``shared view`` as +needed. + +The ``private view`` stores data for use in :doc:`read operations +`. The ``private view`` is mapped to the ``shared view`` +and is the first place MongoDB applies new :doc:`write operations +`, mean read operations get the most up-to-date +data. Keep in mind that because the ``private view`` is a second mapping +of data files, journaling often doubles the amount of virtual memory +:program:`mongod` uses. + +The journal is an on-disk view that stores new write operations +after they have been applied to the ``private cache`` but before they +have been applied to the data files. The journal provides durability. +If the :program:`mongod` instance were to crash without having applied +the writes to the data files, the journal could replay the writes to +the ``shared view`` for eventual upload to the data files. + +.. _journaling-record-write-operation: + +How Journaling Records Write Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MongoDB copies the write operations to the journal in batches +called group commits. By default, MongoDB performs a group commit every +100 milliseconds, which means a series of operations over 100 +milliseconds are committed as a single batch. This is done to achieve +high performance. + +Journaling stores raw operations that allow MongoDB to reconstruct the +following: + +- document insertion/updates +- index modifications +- changes to the namespace files + +As users perform :doc:`write operations `, +MongoDB writes the data first to the ``private view`` in RAM. MongoDB +then copies the write operations in batches from the ``private view`` to +the journal, which stores the operations on disk to ensure durability. +MongoDB adds write operations as entries on the journal's forward +pointer. Each entry on the pointer describes which bytes the write +operation changed in the data files. + +MongoDB next applies the journal's write operations to the ``shared +view``. At this point, the ``shared view`` becomes inconsistent with the data files. + +At default intervals of 60 seconds, MongoDB asks the operating system to +flush the ``shared view`` to disk. This brings the data files up-to-date +with the latest write operations. + +When write operations are flushed to the data files, MongoDB removes the +write operations from the journal's behind pointer. The behind pointer +is always far back from advanced pointer. + +As part of journaling, MongoDB routinely asks the operating system to +remap the ``shared view`` to the ``private view``, for consistency. + +.. note:: The interaction between the ``shared view`` and the on-disk + data files is similar to how MongoDB works *without* + journaling, which is that MongoDB asks the operating system to flush + in-memory changes back to the data files every 60 seconds. From f165da4a84d510c4b55e5087d0862241fc56c442 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Mon, 17 Dec 2012 16:07:21 -0500 Subject: [PATCH 5/8] DOCS-814 edits --- source/administration/journaling.txt | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/source/administration/journaling.txt b/source/administration/journaling.txt index d32555b994d..388b1dd4542 100644 --- a/source/administration/journaling.txt +++ b/source/administration/journaling.txt @@ -55,14 +55,9 @@ with :option:`--nojournal `. Get Commit Acknowledgement ~~~~~~~~~~~~~~~~~~~~~~~~~~ -You can wait for group commit acknowledgement with the getLastError -Command. In versions before 1.9.0 using getLastError + fsync would do -this, in newer versions the "j" option has been specifically created for -this purpose. - -In version 1.9.2+ the group commit delay is shortened when a commit -acknowledgement (getLastError + j) is pending; this can be as little as -1/3 of the normal group commit interval. +You can wait for group commit acknowledgement with the +:dbcommand:`getLastError` command and the ``j`` option. For details, see +:ref:`write-concern-operation`. .. _journaling-avoid-preallocation-lag: From f0e788e95bc66b6cfcaaf26a7bbbb51ed39515dd Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Mon, 17 Dec 2012 20:31:10 -0500 Subject: [PATCH 6/8] DOCS-814 journaling --- source/administration/journaling.txt | 229 ++++++++++++--------------- source/faq/journaling.txt | 61 ------- 2 files changed, 98 insertions(+), 192 deletions(-) delete mode 100644 source/faq/journaling.txt diff --git a/source/administration/journaling.txt b/source/administration/journaling.txt index 388b1dd4542..03848659bc5 100644 --- a/source/administration/journaling.txt +++ b/source/administration/journaling.txt @@ -22,6 +22,10 @@ consistent state. By default, the greatest extent of lost writes, i.e., those not made to the journal, is no more than the last 100 milliseconds. +With journaling, if you want a data set to reside entirely in RAM, you +need twice as much RAM available as the dataset size. See +:ref:`journaling-storage-views`. + .. versionchanged:: 2.0 Journaling is enabled by default for 64-bit platforms. For other platforms, see :setting:`journal`. @@ -32,8 +36,8 @@ Configuration and Setup Enable Journaling ~~~~~~~~~~~~~~~~~ -Beginning with version 2.0, journaling is enabled by default for 64-bit -platforms. +.. versionchanged:: 2.0 + Journaling is enabled by default for 64-bit platforms. To enable journaling, start :program:`mongod` with the :option:`--journal` command line option. @@ -55,7 +59,7 @@ with :option:`--nojournal `. Get Commit Acknowledgement ~~~~~~~~~~~~~~~~~~~~~~~~~~ -You can wait for group commit acknowledgement with the +You can get commit acknowledgement with the :dbcommand:`getLastError` command and the ``j`` option. For details, see :ref:`write-concern-operation`. @@ -64,87 +68,109 @@ You can wait for group commit acknowledgement with the Avoid Preallocation Lag ~~~~~~~~~~~~~~~~~~~~~~~ -To avoid preallocation lag, you can preallocate files in the journal -directory by copying them from another instance of :program:`mongod`. -(For details on preallocation lag, see :ref:`journaling-journal-files`.) - -.. example:: The following sequence of commands preallocates journal - files for an instance of :program:`mongod` running on port ``27017`` - with a database path of ``/data/db``. - - .. code-block:: sh - - $ mkdir ~/tmpDbpath - $ mongod --port 10000 --dbpath ~/tmpDbpath --journal - # startup messages - # . - # . - # . - # wait for prealloc to finish - Thu Mar 17 10:02:52 [initandlisten] preallocating a journal file - ~/tmpDbpath/journal/prealloc.0 - Thu Mar 17 10:03:03 [initandlisten] preallocating a journal file - ~/tmpDbpath/journal/prealloc.1 - Thu Mar 17 10:03:14 [initandlisten] preallocating a journal file - ~/tmpDbpath/journal/prealloc.2 - Thu Mar 17 10:03:25 [initandlisten] flushing directory - ~/tmpDbpath/journal - Thu Mar 17 10:03:25 [initandlisten] flushing directory - ~/tmpDbpath/journal - Thu Mar 17 10:03:25 [initandlisten] waiting for connections on port - 10000 - Thu Mar 17 10:03:25 [websvr] web admin interface listening on port 11000 - # then Ctrl-C to kill this instance - ^C - $ mv ~/tmpDbpath/journal /data/db/ - $ # restart mongod on port 27017 with --journal - -Preallocated files do not contain data. It is safe to remove them. But -if you restart :program:`mongod` with journaling, :program:`mongod` will -create them again. +To avoid :ref:`preallocation lag `, you can +preallocate files in the journal directory by copying them from another +instance of :program:`mongod`. -Change the Group Commit Interval -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Preallocated files do not contain data. It is safe to later remove them. +But if you restart :program:`mongod` with journaling, :program:`mongod` +will create them again. + +.. example:: The following sequence preallocates journal files for an + instance of :program:`mongod` running on port ``27017`` with a database + path of ``/data/db``. + + For demonstration purposes, the sequence starts by creating a set of + journal files in the usual way. + + 1. Create a temporary directory into which to create a set of journal + files: + + .. code-block:: sh + + mkdir ~/tmpDbpath + + #. Create a set of journal files by staring a :program:`mongod` + instance that uses the temporary directory: + + .. code-block:: sh + + mongod --port 10000 --dbpath ~/tmpDbpath --journal + + #. When you see the following log output, indicating :program:`mongod` has the files, + press CONTROL+C to stop the :program:`mongod` instance: + + .. code-block:: sh + + web admin interface listening on port 11000 + + #. Preallocate journal files for the new instance of + :program:`mongod` by moving the journal files from the data directory + of the existing instance to the data directory of the new instance: -Beginning with version 1.9.2, you can set the group commit interval -using the :option:`--journalCommitInterval ` command line option. The allowed range is ``2`` to -``300`` milliseconds. + .. code-block:: sh -Monitor journal Status + mv ~/tmpDbpath/journal /data/db/ + + #. Start the new :program:`mongod` instance: + + .. code-block:: sh + + mongod --port 27017 --dbpath /data/db --journal + +Monitor Journal Status ~~~~~~~~~~~~~~~~~~~~~~ -serverStatus command +Use the following commands and methods to monitor journal status: + +- :dbcommand:`serverStatus` + + The :dbcommand:`serverStatus` command returns database status + information that is useful for assessing performance. + +- :dbcommand:`journalLatencyTest` + + Use :dbcommand:`journalLatencyTest` to measure how long it takes on + your volume to write to the disk in an append-only fashion. You can + run this command on an idle system to get a baseline sync time for + journaling. You can also run this command on a busy system to see the + sync time on a busy system, which may be higher if the journal + directory is on the same volume as the data files. -The serverStatus command now includes some statistics regarding -journaling. + The :dbcommand:`journalLatencyTest` command also provides a way to + check if your disk drive is buffering writes in its local cache. If + the number is very low (i.e., less than 2 milliseconds) and the drive + is non-SSD, the drive is probably buffering writes. In that case, + enable cache write-through for the device in your operating system, + unless you have a disk controller card with battery backed RAM. -journalLatencyTest Command +.. _journaling-journal-commit-interval: -You can use the journalLatencyTest command to measure how long it takes -on your volume to write to the disk (including fsyncing the data) in an -append-only fashion. +Change the Group Commit Interval +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. versionchanged:: 1.9.2 + +You can set the group commit interval using the +:option:`--journalCommitInterval ` +command line option. The allowed range is ``2`` to ``300`` milliseconds. -> use admin +Lower values increase the durability of the journal at the expense of +disk performance. -> db.runCommand("journalLatencyTest") +Recover Data After Unexpected Shutdown +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -You can run this command on an idle system to get a baseline sync time -for journaling. In addition, it is safe to run this command on a busy -system to see the sync time on a busy system (which may be higher if the -journal directory is on the same volume as the data files). +On a restart after a crash, journal files in journal are replayed +before the server goes online. This is indicated in the log output. +You do not need to run a repair. -In version 1.9.2+ you can set the group commit interval, using ---journalCommitInterval command-line option, to between 2 and 300 -milliseconds (default is 100ms). The actual interval are the maximum -of this setting and your disk latency as measured above. +Journaling Internals +-------------------- -journalLatencyTest is also a good way to check if your disk drive is -buffering writes in its local cache. If the number is very low (e.g., -less than 2ms) and the drive is non-ssd, the drive is probably buffering -writes. In that case, you will want to enable cache write-through for -the device in your operating system. (Unless you have a disk controller -card with battery backed ram, then this is a good thing.) +When running with journaling, MongoDB stores and applies :doc:`write +operations ` in memory and in the journal before +the changes are in the data files. .. _journaling-journal-files: @@ -180,66 +206,7 @@ invocations. To avoid preallocation lag, see :ref:`journaling-avoid-preallocation-lag`. -Command-line Options --------------------- - -- `--master`: The :term:`master` mode. - -- :option:`--oplogSize`: This takes an argument and specifies the size - limit in MB for the oplog. - -- `--slave`: The :term:`slave` mode. - -- `--source`: This takes an argument and specifies the master as - . - -- `--only`: This takes an argument and specifies a single database to - replicate. - -Recovery --------- - -On a restart after a crash, journal files in journal are replayed -before the server goes online. This is indicated in the log output. -You do not need to run a repair. - -With journaling if you want a dataset to reside entirely in RAM, you -need twice as much RAM available as the dataset size to be able to store -the ``shared view`` and ``private view``. - -Recommendations -~~~~~~~~~~~~~~~ - -Recommend to set (or at least check) for a low read ahead value -for the data disks, say 40 blocks. - -– And 0 for non-spinning disks - -Recommend to use a separate disk for the journal entries, with a -slightly higher read ahead, say 100 blocks - -– Writes are always at the end of the journal - -– Deletes are always at the beginning of the journal - -Include checking the read ahead values in onboarding interviews - -Set the read ahead values in the templates we distribute - -Be aware of the issue for sudden performance breakdown tickets - -– Beware of resident memory estimates when diagnosing RAM usage - -How Journaling Works --------------------- - -When running with journaling, MongoDB stores and applies :doc:`write -operations ` in memory and in the journal before -the changes are in the data files. - -This section explains this process in detail. - -.. _journaling-configuring-storage: +.. _journaling-storage-views: Storage Views used in Journaling ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/source/faq/journaling.txt b/source/faq/journaling.txt deleted file mode 100644 index c4e8a46e512..00000000000 --- a/source/faq/journaling.txt +++ /dev/null @@ -1,61 +0,0 @@ -=============== -FAQ: Journaling -=============== - -.. default-domain:: mongodb - -This document addresses common questions regarding MongoDB journaling. - -If you don't find the answer you're looking for, check -the :doc:`complete list of FAQs ` or post your question to the -`MongoDB User Mailing List `_. - -.. contents:: Frequently Asked Questions: - :backlinks: none - :local: - -If I am using replication, can some members use journaling and others not? --------------------------------------------------------------------------- -Yes. It is OK to use journaling on some replica set members and not others. - -Can I use the journaling feature to perform safe hot backups? -------------------------------------------------------------- - -Yes, see Backups with Journaling Enabled. - -32 bit nuances? ---------------- - -There is extra memory mapped file activity with journaling. This will -further constrain the limited db size of 32 bit builds. Thus, for now -journaling by default is disabled on 32 bit systems. - -When did the --journal option change from --dur? ------------------------------------------------- - -In 1.8 the option was renamed to --journal, but the old name is still -accepted for backwards compatibility; please change to --journal if you -are using the old option. - -Will the journal replay have problems if entries are incomplete (like the failure happened in the middle of one)? ------------------------------------------------------------------------------------------------------------------ - -Each journal (group) write is consistent and won't be replayed during -recovery unless it is complete. - -How many times is data written to disk when replication and journaling are both on? ------------------------------------------------------------------------------------ - -In v1.8, for an insert, four times. The object is written to the main -collection, and also the oplog collection (so that is twice). Both of -those writes are journaled as a single mini-transaction in the journal -file (the files in /data/db/journal). Thus 4 times total. - -There is an open item in to reduce this by having the journal be -compressed. This will reduce from 4x to probably ~2.5x. - -The above applies to collection data and inserts which is the worst case -scenario. Index updates are written to the index and the journal, but -not the oplog, so they should be 2X today not 4X. Likewise updates with -things like $set, $addToSet, $inc, etc. are compactly logged all around -so those are generally small. From 573200972968bd43337d762f7a2c9f9588151931 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Tue, 18 Dec 2012 11:22:47 -0500 Subject: [PATCH 7/8] DOCS-814 review edits --- source/administration/journaling.txt | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/source/administration/journaling.txt b/source/administration/journaling.txt index 03848659bc5..e2e207a6e03 100644 --- a/source/administration/journaling.txt +++ b/source/administration/journaling.txt @@ -8,6 +8,7 @@ Journaling :doc:`write operations ` in an on-disk journal prior to applying them to the data files. The journal ensures write operations can be re-applied in the event of a crash. +Journaling is also referred to as "write ahead logging." Journaling ensures that :program:`mongodb` is crash resilient. *Without* a journal, if :program:`mongodb` exits unexpectedly, you must assume @@ -23,8 +24,10 @@ those not made to the journal, is no more than the last 100 milliseconds. With journaling, if you want a data set to reside entirely in RAM, you -need twice as much RAM available as the dataset size. See -:ref:`journaling-storage-views`. +need enough RAM to hold the dataset plus the "write working set." The +"write working set" is the amount of unique data you expect to see +written between re-mappings of the private view. For information on +views, see :ref:`journaling-storage-views`. .. versionchanged:: 2.0 Journaling is enabled by default for 64-bit platforms. @@ -193,7 +196,9 @@ To limit the size of journal files to 128 megabytes per file, use the ``--smallfiles`` command line option when starting :program:`mongod`. To speed the frequent sequential writes that occur to the current -journal file, ensure that the journal directory is on a different system. +journal file, you can ensure that the journal directory is on a +different system. However, doing so prevents use of a snapshotting +filesystem to take backups. Depending on your file system, you might experience a preallocation lag the first time you start a :program:`mongod` instance with journaling From 84c28dc5f2a162a4b170ce91808e074c16fba154 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Tue, 18 Dec 2012 13:46:34 -0500 Subject: [PATCH 8/8] DOCS-814 more review edits --- source/administration/journaling.txt | 49 +++++++++++++++------------- 1 file changed, 27 insertions(+), 22 deletions(-) diff --git a/source/administration/journaling.txt b/source/administration/journaling.txt index e2e207a6e03..5ff628ee8ef 100644 --- a/source/administration/journaling.txt +++ b/source/administration/journaling.txt @@ -45,14 +45,23 @@ Enable Journaling To enable journaling, start :program:`mongod` with the :option:`--journal` command line option. -If :program:`mongod` decides to preallocate the files, it will not start -listening on port 27017 until this process completes, which can take a -few minutes. Your applications and the shell will not be able to connect -to the database until the process completes. +If the :program:`mongod` process preallocates the files, the process +delays listening on port 27017 until preallocation completes, which can +take a few minutes. Your applications and the shell will not be able to +connect to the database until the process completes. Disable Journaling ~~~~~~~~~~~~~~~~~~ +.. warning:: + + Do not disable journaling on production systems. If your MongoDB + system stops unexpectedly from a power failure or other condition, + and if you are not running with journaling, then you must recover + from an unaffected :term:`replica set` member or backup, as described + in :doc:`repair + `. + To disable journaling, start :program:`mongod` with the :option:`--nojournal ` command line option. @@ -152,7 +161,7 @@ Use the following commands and methods to monitor journal status: Change the Group Commit Interval ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. versionchanged:: 1.9.2 +.. versionchanged:: 2.0 You can set the group commit interval using the :option:`--journalCommitInterval ` @@ -229,10 +238,7 @@ needed. The ``private view`` stores data for use in :doc:`read operations `. The ``private view`` is mapped to the ``shared view`` and is the first place MongoDB applies new :doc:`write operations -`, mean read operations get the most up-to-date -data. Keep in mind that because the ``private view`` is a second mapping -of data files, journaling often doubles the amount of virtual memory -:program:`mongod` uses. +`. The journal is an on-disk view that stores new write operations after they have been applied to the ``private cache`` but before they @@ -246,11 +252,10 @@ the ``shared view`` for eventual upload to the data files. How Journaling Records Write Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -MongoDB copies the write operations to the journal in batches -called group commits. By default, MongoDB performs a group commit every -100 milliseconds, which means a series of operations over 100 -milliseconds are committed as a single batch. This is done to achieve -high performance. +MongoDB copies the write operations to the journal in batches called +group commits. By default, MongoDB performs a group commit every 100 +milliseconds, which means a series of operations over 100 milliseconds +are committed as a single batch. This is done to improve performance. Journaling stores raw operations that allow MongoDB to reconstruct the following: @@ -259,16 +264,16 @@ following: - index modifications - changes to the namespace files -As users perform :doc:`write operations `, -MongoDB writes the data first to the ``private view`` in RAM. MongoDB -then copies the write operations in batches from the ``private view`` to -the journal, which stores the operations on disk to ensure durability. -MongoDB adds write operations as entries on the journal's forward -pointer. Each entry on the pointer describes which bytes the write -operation changed in the data files. +As :doc:`write operations ` occur, MongoDB +writes the data to the ``private view`` in RAM and then copies the write +operations in batches to the journal. The journal stores the operations +on disk to ensure durability. MongoDB adds the operations as entries on +the journal's forward pointer. Each entry describes which bytes the +write operation changed in the data files. MongoDB next applies the journal's write operations to the ``shared -view``. At this point, the ``shared view`` becomes inconsistent with the data files. +view``. At this point, the ``shared view`` becomes inconsistent with the +data files. At default intervals of 60 seconds, MongoDB asks the operating system to flush the ``shared view`` to disk. This brings the data files up-to-date