From e58212af8034e867bcfcec697232773d35010d4f Mon Sep 17 00:00:00 2001 From: Andrew Feierabend Date: Tue, 23 Mar 2021 16:20:04 -0400 Subject: [PATCH] DOCSP-15179-DOCS-9187 upsert with unique index refactor --- .../includes/extracts-parameter-upsert.yaml | 55 +++++++- .../extracts-upsert-unique-index.yaml | 129 ++++++++++++++++++ source/reference/command/findAndModify.txt | 47 +------ source/reference/command/update.txt | 11 +- .../method/db.collection.findAndModify.txt | 43 +----- .../reference/method/db.collection.update.txt | 60 ++------ 6 files changed, 209 insertions(+), 136 deletions(-) create mode 100644 source/includes/extracts-upsert-unique-index.yaml diff --git a/source/includes/extracts-parameter-upsert.yaml b/source/includes/extracts-parameter-upsert.yaml index f431ccbf86f..6a5b519a0dc 100644 --- a/source/includes/extracts-parameter-upsert.yaml +++ b/source/includes/extracts-parameter-upsert.yaml @@ -8,13 +8,34 @@ content: | - Updates a single document that matches the ``{{queryOrFilter}}``. - To avoid multiple upserts, ensure that the ``{{queryOrFilter}}`` fields - are :ref:`uniquely indexed `. + {{usageWithMulti}} + + To avoid multiple :term:`upserts `, ensure that the + ``{{queryOrFilter}}`` field(s) are :ref:`uniquely indexed + `. {{uniqueIndexExample}} + + Defaults to ``false``, which does *not* insert a new document when no + match is found. - Defaults to ``false``. replacement: conjunction: '' returnNewDocument: '' + usageWithMulti: '' + uniqueIndexExample: '' +--- +ref: findAndModify-behavior-command +source: + file: extracts-parameter-upsert.yaml + ref: _update-single-upsert-behavior +replacement: + upsertMethod: ":dbcommand:`findAndModify`" + queryOrFilter: query + uniqueIndexExample: "See :ref:`upsert-and-unique-index-dbcommand` for + an example." + conjunction: "Used in conjunction with the ``update`` field. + + +" --- ref: findAndModify-behavior-method source: @@ -23,11 +44,39 @@ source: replacement: upsertMethod: ":method:`~db.collection.findAndModify()`" queryOrFilter: query + uniqueIndexExample: "See :ref:`upsert-and-unique-index` for an + example." conjunction: "Used in conjunction with the ``update`` field. " --- +ref: update-upsert-behavior-command +source: + file: extracts-parameter-upsert.yaml + ref: _update-single-upsert-behavior +replacement: + upsertMethod: ":dbcommand:`update`" + queryOrFilter: query + usageWithMulti: "If both ``upsert`` and ``multi`` are true and no + documents match the query, the update operation inserts only a + single document." + uniqueIndexExample: "See :ref:`update-command-behaviors-unique-index` + for an example." +--- +ref: update-upsert-behavior-method +source: + file: extracts-parameter-upsert.yaml + ref: _update-single-upsert-behavior +replacement: + upsertMethod: ":method:`~db.collection.update()`" + queryOrFilter: query + usageWithMulti: "If both ``upsert`` and ``multi`` are true and no + documents match the query, the update operation inserts only a + single document." + uniqueIndexExample: "See :ref:`update-with-unique-indexes` for an + example." +--- ref: findOneAndUpdate-behavior-method source: file: extracts-parameter-upsert.yaml diff --git a/source/includes/extracts-upsert-unique-index.yaml b/source/includes/extracts-upsert-unique-index.yaml new file mode 100644 index 00000000000..70304426a80 --- /dev/null +++ b/source/includes/extracts-upsert-unique-index.yaml @@ -0,0 +1,129 @@ +ref: _upsert-unique-index-base +content: | + + When using the {{upsert}} option with the {{command}} + {{commandOrMethod}}, **and not** using a :ref:`unique index + ` on the query field(s), multiple + instances of {{aOrAn}} {{command}} operation with similar query + field(s) could result in duplicate documents being inserted in + certain circumstances. + + Consider an example where no document with the name ``Andy`` exists + and multiple clients issue the following command at roughly the same + time: + + {{codeExample}} + + If all {{command}} operations finish the query phase + before any client successfully inserts data, **and** there is no + :ref:`unique index ` on the ``name`` field, each + {{command}} operation may result in an insert, creating multiple + documents with ``name: Andy``. + + To ensure that only one such document is created, and the other + {{command}} operations update this new document instead, create a + :ref:`unique index ` on the ``name`` field. This + guarantees that only one document with ``name: Andy`` is permitted + in the collection. + + With this unique index in place, the multiple {{command}} operations + now exhibit the following behavior: + + - Exactly one {{command}} operation will successfully insert a new + document. + + - All other {{command}} operations will update the newly-inserted + document, incrementing the ``score`` value. + +--- +ref: upsert-unique-index-findAndModify-command +source: + file: extracts-upsert-unique-index.yaml + ref: _upsert-unique-index-base +replacement: + command: ":dbcommand:`findAndModify`" + commandOrMethod: "command" + aOrAn: "a" + upsert: "``upsert: true``" + codeExample: | + + .. code-block:: javascript + + db.runCommand( + { + findAndModify: "people", + query: { name: "Andy" }, + update: { $inc: { score: 1 } }, + upsert: true + } + ) + +--- +ref: upsert-unique-index-findAndModify-method +source: + file: extracts-upsert-unique-index.yaml + ref: _upsert-unique-index-base +replacement: + command: ":method:`~db.collection.findOneAndUpdate()`" + commandOrMethod: "method" + aOrAn: "a" + upsert: "``upsert: true``" + codeExample: | + + .. code-block:: javascript + + db.people.findAndModify( + { + query: { name: "Andy" }, + update: { $inc: { score: 1 } }, + upsert: true + } + ) + +--- +ref: upsert-unique-index-update-command +source: + file: extracts-upsert-unique-index.yaml + ref: _upsert-unique-index-base +replacement: + command: ":dbcommand:`update`" + commandOrMethod: "command" + aOrAn: "an" + upsert: ":ref:`upsert: true `" + codeExample: | + + .. code-block:: javascript + + db.runCommand( + { + update: "people", + updates: [ + { q: { name: "Andy" }, u: { $inc: { score: 1 } }, multi: true, upsert: true } + ] + } + ) + +--- +ref: upsert-unique-index-update-method +source: + file: extracts-upsert-unique-index.yaml + ref: _upsert-unique-index-base +replacement: + command: ":method:`~db.collection.update()`" + commandOrMethod: "method" + aOrAn: "a" + upsert: ":ref:`upsert: true `" + codeExample: | + + .. code-block:: javascript + + db.people.update( + { name: "Andy" }, + { $inc: { score: 1 } }, + { + upsert: true, + multi: true + } + ) + +... diff --git a/source/reference/command/findAndModify.txt b/source/reference/command/findAndModify.txt index 96a6df75a93..fe66efe2b5e 100644 --- a/source/reference/command/findAndModify.txt +++ b/source/reference/command/findAndModify.txt @@ -157,7 +157,7 @@ Definition - boolean - - .. include:: /includes/extracts/findAndModify-behavior-method.rst + - .. include:: /includes/extracts/findAndModify-behavior-command.rst @@ -357,49 +357,12 @@ following: Behavior -------- -Upsert and Unique Index -~~~~~~~~~~~~~~~~~~~~~~~ +.. _upsert-and-unique-index-dbcommand: -When the :dbcommand:`findAndModify` command includes the ``upsert: -true`` option **and** the query field(s) is not uniquely indexed, the -command could insert a document multiple times in certain circumstances. +Upsert with Unique Index +~~~~~~~~~~~~~~~~~~~~~~~~ -Consider an example where no document with the name ``Andy`` exists and -multiple clients issue the following command: - -.. code-block:: javascript - - db.runCommand( - { - findAndModify: "people", - query: { name: "Andy" }, - sort: { rating: 1 }, - update: { $inc: { score: 1 } }, - upsert: true - } - ) - -If all the commands finish the ``query`` phase before any command -starts the ``modify`` phase, **and** there is no unique index on the -``name`` field, the commands may each perform an upsert, creating -multiple duplicate documents. - -To prevent the creation of multiple duplicate documents, -create a :ref:`unique index ` on -the ``name`` field. With the unique index in place, then the multiple -:dbcommand:`findAndModify` commands will exhibit one of the -following behaviors: - -- Exactly one :dbcommand:`findAndModify` successfully inserts a - new document. - -- Zero or more :dbcommand:`findAndModify` commands update the - newly inserted document. - -- Zero or more :dbcommand:`findAndModify` commands fail when - they attempt to insert a duplicate. If the command fails due to - a unique index constraint violation, you can retry the command. - Absent a delete of the document, the retry should not fail. +.. include:: /includes/extracts/upsert-unique-index-findAndModify-command.rst .. _cmd-findAndModify-sharded-collection: diff --git a/source/reference/command/update.txt b/source/reference/command/update.txt index e7efb72e78b..c7bb848ad32 100644 --- a/source/reference/command/update.txt +++ b/source/reference/command/update.txt @@ -193,9 +193,7 @@ Each document contains the following fields: - .. _update-command-upsert: - Optional. If ``true``, perform an insert if no documents match the query. If - both ``upsert`` and ``multi`` are true and no documents match the - query, the update operation inserts only a single document. + .. include:: /includes/extracts/update-upsert-behavior-command.rst * - ``multi`` @@ -386,6 +384,13 @@ For example: For examples, see :ref:`update-command-example-agg`. +.. _update-command-behaviors-unique-index: + +Upsert with Unique Index +~~~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/extracts/upsert-unique-index-update-command.rst + Limits ~~~~~~ diff --git a/source/reference/method/db.collection.findAndModify.txt b/source/reference/method/db.collection.findAndModify.txt index 99dc3a0389a..3e41752d9cd 100644 --- a/source/reference/method/db.collection.findAndModify.txt +++ b/source/reference/method/db.collection.findAndModify.txt @@ -288,47 +288,10 @@ For more information on projection, see also: .. _upsert-and-unique-index: -Upsert and Unique Index -~~~~~~~~~~~~~~~~~~~~~~~ +Upsert with Unique Index +~~~~~~~~~~~~~~~~~~~~~~~~ -When :method:`~db.collection.findAndModify()` includes the ``upsert: -true`` option **and** the query field(s) is not uniquely indexed, the -method could insert a document multiple times in certain circumstances. - -In the following example, no document with the name ``Andy`` exists, -and multiple clients issue the following command: - -.. code-block:: javascript - - db.people.findAndModify({ - query: { name: "Andy" }, - sort: { rating: 1 }, - update: { $inc: { score: 1 } }, - upsert: true - }) - -Then, if these clients' :method:`~db.collection.findAndModify()` -methods finish the ``query`` phase before any command starts the -``modify`` phase, **and** there is no unique index on the ``name`` -field, the commands may all perform an upsert, creating -multiple duplicate documents. - -To prevent the creation of multiple duplicate documents with the same -name, create a :ref:`unique index ` on the ``name`` -field. With this unique index in place, the multiple methods will -exhibit one of the following behaviors: - -- Exactly one :method:`~db.collection.findAndModify()` - successfully inserts a new document. - -- Zero or more :method:`~db.collection.findAndModify()` methods - update the newly inserted document. - -- Zero or more :method:`~db.collection.findAndModify()` methods fail - when they attempt to insert documents with the same name. If the - method fails due to the unique index constraint violation on the - ``name`` field, you can retry the method. Absent a delete of the - document, the retry should not fail. +.. include:: /includes/extracts/upsert-unique-index-findAndModify-method.rst .. _method-findAndModify-sharded-collection: diff --git a/source/reference/method/db.collection.update.txt b/source/reference/method/db.collection.update.txt index 6e8de24c5e1..49d91654b9a 100644 --- a/source/reference/method/db.collection.update.txt +++ b/source/reference/method/db.collection.update.txt @@ -118,10 +118,7 @@ parameters: - .. _update-upsert: - Optional. If set to ``true``, creates a new document when no - document matches the query criteria. The default value is - ``false``, which does *not* insert a new document when no match - is found. + .. include:: /includes/extracts/update-upsert-behavior-method.rst * - :ref:`multi ` @@ -759,6 +756,14 @@ When you specify the option :ref:`upsert: true `: - If no document matches the query criteria, :method:`db.collection.update()` inserts a *single* document. + .. note:: + + If multiple, identical :term:`upserts ` are issued at + roughly the same time, it is possible for + :method:`~db.collection.update()` used with :ref:`upsert: true + ` to create duplicate documents. See + :ref:`update-with-unique-indexes` for more information. + If you specify ``upsert: true`` on a sharded collection, you must include the full shard key in the ``filter``. For additional :method:`db.collection.update()` behavior on a sharded collection, see @@ -1060,51 +1065,10 @@ include the full shard key in the ``filter``. For additional .. _update-with-unique-indexes: -Use Unique Indexes -`````````````````` - -.. warning:: To avoid inserting the same document more than once, - only use ``upsert: true`` if the ``query`` field is uniquely - indexed. - -Given a collection named ``people`` where no documents have -a ``name`` field that holds the value ``Andy``, consider when multiple -clients issue the following :method:`db.collection.update()` with -``upsert: true`` at the same time: - -.. code-block:: javascript - - db.people.update( - { name: "Andy" }, // Query parameter - { // Update document - name: "Andy", - rating: 1, - score: 1 - }, - { upsert: true } // Options - ) - -If all :method:`db.collection.update()` operations complete the -``query`` portion before any client successfully inserts data, **and** -there is no unique index on the ``name`` field, then each update -operation may result in an insert. - -To prevent MongoDB from inserting the same document more than once, -create a :ref:`unique index ` on the ``name`` field. -With a unique index, if multiple applications issue the same update -with ``upsert: true``, *exactly one* -:method:`db.collection.update()` would successfully insert a new -document. - -The remaining operations would either: - -- update the newly inserted document, or - -- fail when they attempted to insert a duplicate. +Upsert with Unique Index +```````````````````````` - If the operation fails because of a duplicate index key error, - applications may retry the operation which will succeed as an update - operation. +.. include:: /includes/extracts/upsert-unique-index-update-method.rst .. seealso::