DOCSP-18759 pre post images (#941)

jason-price-mongodb · jason-price-mongodb · web-flow · commit cc2f1389263d · 2022-04-27T11:27:58.000-07:00
* DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images * DOCSP-18759-PIT-pre-post-images Co-authored-by: jason-price-mongodb <jshfjghsdfgjsdjh@aolsdjfhkjsdhfkjsdf.com>
diff --git a/source/changeStreams.txt b/source/changeStreams.txt
@@ -1050,6 +1050,16 @@ Change Streams and Orphan Documents
 
 .. include:: /includes/change-streams-and-orphans.rst
 
+Change Streams with Document Pre- and Post-Images
+-------------------------------------------------
+
+.. include:: /includes/change-stream-pre-and-post-images-introduction.rst
+
+.. include:: /includes/change-stream-pre-and-post-images-additional-information.rst
+
+For complete examples with the change stream output, see
+:ref:`db.collection.watch-change-streams-pre-and-post-images-example`.
+
 .. toctree::
    :titlesonly:
    :hidden:
diff --git a/source/includes/change-stream-pre-and-post-images-additional-information.rst b/source/includes/change-stream-pre-and-post-images-additional-information.rst
@@ -0,0 +1,88 @@
+Pre- and post-images are not available for a :ref:`change stream event
+<change-stream-output>` if the images were:
+
+- Not enabled on the collection at the time of a document update or
+  delete operation.
+
+- Removed after the pre- and post-image retention time set in
+  ``expireAfterSeconds``.
+  
+  - The following example sets ``expireAfterSeconds`` to ``100``
+    seconds:
+
+    .. code-block:: javascript
+
+       use admin
+       db.runCommand( {
+          setClusterParameter:
+             { changeStreamOptions: { preAndPostImages: { expireAfterSeconds: 100 } } }
+       } )
+
+  - The following example returns the current ``changeStreamOptions``
+    settings, including ``expireAfterSeconds``:
+
+    .. code-block:: javascript
+
+       db.adminCommand( { getClusterParameter: "changeStreamOptions" } )
+
+  - Setting ``expireAfterSeconds`` to ``off`` uses the default retention
+    policy: pre- and post-images are retained until the corresponding
+    change stream events are removed from the :term:`oplog`.
+
+  - If a change stream event is removed from the oplog, then the
+    corresponding pre- and post-images are also deleted regardless of
+    the ``expireAfterSeconds`` pre- and post-image retention time.
+
+Additional considerations:
+
+- Enabling pre- and post-images consumes storage space and adds
+  processing time. Only enable pre- and post-images if you need them.
+
+- Limit the change stream event size to less than 16 megabytes. To limit
+  the event size, you can:
+
+  - Limit the document size to 8 megabytes. You can request pre- and
+    post-images simultaneously in the :ref:`change stream output
+    <change-stream-output>` if other change stream event fields like
+    ``updateDescription`` are not large.
+
+  - Request only post-images in the change stream output for documents
+    up to 16 megabytes if other change stream event fields like
+    ``updateDescription`` are not large.
+
+  - Request only pre-images in the change stream output for documents up
+    to 16 megabytes if:
+    
+    - document updates affect only a small fraction of the document
+      structure or content, *and*
+    
+    - do not cause a ``replace`` change event. A ``replace`` event
+      always includes the post-image.
+
+- To request a pre-image, you set ``fullDocumentBeforeChange`` to
+  ``required`` or ``whenAvailable`` in :method:`db.collection.watch()`.
+  To request a post-image, you set ``fullDocument`` using the same
+  method.
+
+- Pre-images are written to the ``config.system.preimages`` collection.
+    
+  - The ``config.system.preimages`` collection may become large. To
+    limit the collection size, you can set ``expireAfterSeconds``
+    time for the pre-images as shown earlier.
+  
+  - Pre-images are removed asynchronously by a background process.
+
+- If you need to downgrade MongoDB to a version that does not support
+  pre- and post-images, you must first disable pre- and post-images for
+  the collections.
+
+.. seealso::
+
+   - For change stream events and output, see
+     :ref:`change-stream-output`.
+
+   - To watch a collection for changes, see
+     :method:`db.collection.watch`.
+
+   - For complete examples with the change stream output, see
+     :ref:`db.collection.watch-change-streams-pre-and-post-images-example`.
diff --git a/source/includes/change-stream-pre-and-post-images-change-events.rst b/source/includes/change-stream-pre-and-post-images-change-events.rst
@@ -0,0 +1,25 @@
+Starting in MongoDB 6.0, you see a ``fullDocumentBeforeChange``
+document with the fields before the document was changed (or deleted)
+if you perform these steps:
+
+#. Enable the new ``changeStreamPreAndPostImages`` field for a
+   collection using :method:`db.createCollection()`,
+   :dbcommand:`create`, or :dbcommand:`collMod`.
+
+#. Set ``fullDocumentBeforeChange`` to ``"required"`` or
+   ``"whenAvailable"`` in :method:`db.collection.watch()`.
+
+Example ``fullDocumentBeforeChange`` document in the change stream
+output:
+
+.. code-block:: javascript
+   :copyable: false
+
+   "fullDocumentBeforeChange" : {
+      "_id" : ObjectId("599af247bb69cd89961c986d"), 
+      "userName" : "alice123",
+      "name" : "Alice Smith"
+   }
+
+For complete examples with the change stream output, see
+:ref:`db.collection.watch-change-streams-pre-and-post-images-example`.
diff --git a/source/includes/change-stream-pre-and-post-images-example-cursor-methods.rst b/source/includes/change-stream-pre-and-post-images-example-cursor-methods.rst
@@ -0,0 +1,6 @@
+In the example:
+
+- The ``while`` loop runs until the cursor is closed.
+
+- :method:`~cursor.hasNext()` returns ``true`` if the cursor has
+  documents.
diff --git a/source/includes/change-stream-pre-and-post-images-field.rst b/source/includes/change-stream-pre-and-post-images-field.rst
@@ -0,0 +1,27 @@
+Optional.
+
+.. include:: /includes/change-stream-pre-and-post-images-introduction.rst
+
+``changeStreamPreAndPostImages`` has the following syntax:
+
+.. code-block:: javascript
+   :copyable: false
+
+   changeStreamPreAndPostImages: {
+      enabled: <boolean>
+   }
+
+.. list-table::
+   :header-rows: 1
+
+   * - ``enabled``
+     - Description
+
+   * - ``true``
+     - Enables change stream pre- and post-images for a collection.
+
+   * - ``false``
+     - Disables change stream pre- and post-images for a collection.
+
+For complete examples with the change stream output, see
+:ref:`db.collection.watch-change-streams-pre-and-post-images-example`.
diff --git a/source/includes/change-stream-pre-and-post-images-full-document-before-change.rst b/source/includes/change-stream-pre-and-post-images-full-document-before-change.rst
@@ -0,0 +1,12 @@
+Starting in MongoDB 6.0, you can use the new
+``fullDocumentBeforeChange`` field and set it to:
+
+- ``"whenAvailable"`` to output the document pre-image, if available,
+  before the document was replaced, updated, or deleted.
+
+- ``"required"`` to output the document pre-image before the document
+  was replaced, updated, or deleted. Raises an error if the pre-image
+  is not available.
+
+- ``"off"`` to suppress the document pre-image. ``"off"`` is the
+  default.
diff --git a/source/includes/change-stream-pre-and-post-images-full-document.rst b/source/includes/change-stream-pre-and-post-images-full-document.rst
@@ -0,0 +1,8 @@
+Starting in MongoDB 6.0, you can set ``fullDocument`` to:
+          
+- ``"whenAvailable"`` to output the document post-image, if available,
+  after the document was inserted, replaced, or updated.
+
+- ``"required"`` to output the document post-image after the document
+  was inserted, replaced, or updated. Raises an error if the post-image
+  is not available.
diff --git a/source/includes/change-stream-pre-and-post-images-introduction.rst b/source/includes/change-stream-pre-and-post-images-introduction.rst
@@ -0,0 +1,13 @@
+Starting in MongoDB 6.0, you can use :ref:`change stream events
+<change-stream-output>` to output the version of a document before and
+after changes (the document pre- and post-images):
+
+- The pre-image is the document before it was replaced, updated, or
+  deleted. There is no pre-image for an inserted document.
+
+- The post-image is the document after it was inserted, replaced, or
+  updated. There is no post-image for a deleted document.
+
+- Enable ``changeStreamPreAndPostImages`` for a collection using
+  :method:`db.createCollection()`, :dbcommand:`create`, or
+  :dbcommand:`collMod`.
diff --git a/source/includes/change-stream-pre-and-post-images-output.rst b/source/includes/change-stream-pre-and-post-images-output.rst
@@ -0,0 +1,6 @@
+.. seealso::
+
+   - For document update output details, see :ref:`change
+     stream update events <change-streams-update-event>`.
+
+   - For change stream output details, see :ref:`change-stream-output`.
diff --git a/source/reference/built-in-roles.txt b/source/reference/built-in-roles.txt
@@ -258,7 +258,7 @@ Cluster Administration Roles
              - :authaction:`setDefaultRWConcern` (New in version 4.4)
              - :authaction:`setFeatureCompatibilityVersion`
              - :authaction:`setFreeMonitoring`
-                  
+
       * - *All* :ref:`databases <resource-specific-db>`
 
         - .. hlist::
@@ -832,6 +832,12 @@ The following role provides full privileges on all resources:
       The :authrole:`root` role includes privileges from the
       :authrole:`backup` and :authrole:`restore` roles.
 
+   .. versionchanged:: 6.0
+
+      The :authrole:`root` role includes :authaction:`find` and
+      :authaction:`remove` privileges on the ``system.preimages``
+      collection in the ``config`` database.
+
 Internal Role
 -------------
 
diff --git a/source/reference/change-events.txt b/source/reference/change-events.txt
@@ -26,6 +26,7 @@ stream response document can have.
       _id : { <BSON Object> },
       "operationType" : "<operation>",
       "fullDocument" : { <document> },
+      "fullDocumentBeforeChange" : { <document> },  // Added in MongoDB 6.0
       "ns" : {
          "db" : "<database>",
          "coll" : "<collection>"
@@ -119,6 +120,25 @@ following table describes each field in the change stream response document:
        if other majority-committed operations modified the document between
        the original update operation and the full document lookup.
 
+       Starting in MongoDB 6.0, if you enable the
+       ``changeStreamPreAndPostImages`` field using
+       :method:`db.createCollection()`, :dbcommand:`create`, or
+       :dbcommand:`collMod`, then ``fullDocument`` is the document after
+       it was inserted, replaced, or updated (the document post-image).
+       ``fullDocument`` is always included for ``insert`` events.
+
+   * - ``fullDocumentBeforeChange``
+     - document
+     - The document before it was replaced, updated, or deleted (the
+       document pre-image).
+
+       To obtain ``fullDocumentBeforeChange``, you must enable the
+       ``changeStreamPreAndPostImages`` field for a collection using
+       :method:`db.createCollection()`, :dbcommand:`create`, or
+       :dbcommand:`collMod`.
+
+       .. versionadded:: 6.0
+
    * - ``ns``
      - document
      - The namespace (database and or collection) affected by the event.
@@ -352,6 +372,10 @@ the document at the time of the update operation depending on the number of
 interleaving majority-committed operations that occur between the update
 operation and the document lookup.
 
+.. include:: /includes/change-stream-pre-and-post-images-change-events.rst
+
+.. include:: /includes/change-stream-pre-and-post-images-additional-information.rst
+
 ``replace`` Event
 -----------------
 
@@ -385,6 +409,10 @@ A ``replace`` operation uses the update command, and consists of two stages:
 The ``fullDocument`` of a ``replace`` event represents the document after the
 insert of the replacement document.
 
+.. include:: /includes/change-stream-pre-and-post-images-change-events.rst
+
+.. include:: /includes/change-stream-pre-and-post-images-additional-information.rst
+
 ``delete`` Event
 ----------------
 
@@ -408,6 +436,10 @@ The following example illustrates a ``delete`` event:
 The ``fullDocument`` document is omitted as the document no longer exists at the
 time the change stream cursor sends the ``delete`` event to the client.
 
+.. include:: /includes/change-stream-pre-and-post-images-change-events.rst
+
+.. include:: /includes/change-stream-pre-and-post-images-additional-information.rst
+
 .. _change-streams-drop-event:
 
 ``drop`` Event
diff --git a/source/reference/command/collMod.txt b/source/reference/command/collMod.txt
@@ -287,6 +287,36 @@ to 100000 bytes and sets the maximum number of documents in the collection to 50
       cappedMax: 500
    } )
 
+.. _collMod-change-stream-pre-and-post-images:
+
+Change Streams with Document Pre- and Post-Images
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. include:: /includes/change-stream-pre-and-post-images-introduction.rst
+
+To use :dbcommand:`collMod` to enable change stream pre- and post-images
+for a collection, use the ``changeStreamPreAndPostImages`` field:
+
+.. code-block:: javascript
+   :copyable: false
+
+   db.runCommand( {
+      collMod: <collection>,
+      changeStreamPreAndPostImages: { enabled: <boolean> }
+   } )
+
+To enable change stream pre- and post-images for a collection, set
+``changeStreamPreAndPostImages`` to ``true``. For example:
+
+.. code-block:: javascript
+
+   db.runCommand( {
+      "collMod": "orders",
+      changeStreamPreAndPostImages: { enabled: true }
+   } )
+
+.. include:: /includes/change-stream-pre-and-post-images-additional-information.rst
+
 Write Concern
 ~~~~~~~~~~~~~
 
diff --git a/source/reference/command/create.txt b/source/reference/command/create.txt
diff --git a/source/reference/glossary.txt b/source/reference/glossary.txt
diff --git a/source/reference/method/db.collection.watch.txt b/source/reference/method/db.collection.watch.txt
diff --git a/source/reference/method/db.createCollection.txt b/source/reference/method/db.createCollection.txt
diff --git a/source/reference/method/db.watch.txt b/source/reference/method/db.watch.txt
diff --git a/source/release-notes/6.0.txt b/source/release-notes/6.0.txt
