From 6210f5bafd95655eea62cda0dfa331cb9f2a343b Mon Sep 17 00:00:00 2001 From: schmalliso Date: Tue, 9 Jul 2013 15:28:55 -0400 Subject: [PATCH 01/14] crud-reorg: breaking apart the read operations page --- ...operations-architecture-considerations.txt | 82 ++ source/core/read-operations-cursors.txt | 256 ++++ source/core/read-operations-indexes.txt | 305 +++++ source/core/read-operations-queries.txt | 401 +++++++ source/core/read-operations.txt | 1054 +---------------- 5 files changed, 1053 insertions(+), 1045 deletions(-) create mode 100644 source/core/read-operations-architecture-considerations.txt create mode 100644 source/core/read-operations-cursors.txt create mode 100644 source/core/read-operations-indexes.txt create mode 100644 source/core/read-operations-queries.txt diff --git a/source/core/read-operations-architecture-considerations.txt b/source/core/read-operations-architecture-considerations.txt new file mode 100644 index 00000000000..fd0638adb84 --- /dev/null +++ b/source/core/read-operations-architecture-considerations.txt @@ -0,0 +1,82 @@ +.. index:: read operation; architecture +.. _read-operations-architecture: + +Architecture +------------ + +.. index:: read operation; connection pooling +.. index:: connection pooling; read operations +.. _read-operations-connection-pooling: + +Read Operations from Sharded Clusters +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:term:`Sharded clusters ` allow you to partition a +data set among a cluster of :program:`mongod` in a way that is nearly +transparent to the application. See the :doc:`/sharding` section of +this manual for additional information about these deployments. + +For a sharded cluster, you issue all operations to one of the +:program:`mongos` instances associated with the +cluster. :program:`mongos` instances route operations to the +:program:`mongod` in the cluster and behave like :program:`mongod` +instances to the application. Read operations to a sharded collection +in a sharded cluster are largely the same as operations to a :term:`replica +set` or :term:`standalone` instances. See the section on :ref:`Read +Operations in Sharded Clusters ` for more +information. + +In sharded deployments, the :program:`mongos` instance routes +the queries from the clients to the :program:`mongod` instances that +hold the data, using the cluster metadata stored in the :ref:`config +database `. + +For sharded collections, if queries do not include the :ref:`shard key +`, the :program:`mongos` must direct the query to +all shards in a collection. These *scatter gather* queries can be +inefficient, particularly on larger clusters, and are unfeasible for +routine operations. + +For more information on read operations in sharded clusters, consider +the following resources: + +- :ref:`An Introduction to Shard Keys ` +- :ref:`Shard Key Internals and Operations ` +- :ref:`Querying Sharded Clusters ` +- :doc:`/core/sharded-cluster-query-router` + +Read Operations from Replica Sets +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:term:`Replica sets ` use :term:`read preferences ` to determine where and how to route read operations to +members of the replica set. By default, MongoDB always reads data from +a replica set's :term:`primary`. You can modify that behavior by +changing the :ref:`read preference mode +`. + +You can configure the :ref:`read preference mode +` on a per-connection or +per-operation basis to allow reads from :term:`secondaries +` to: + +- reduce latency in multi-data-center deployments, + +- improve read throughput by distributing high read-volumes (relative + to write volume), + +- for backup operations, and/or + +- to allow reads during :ref:`failover ` + situations. + +Read operations from secondary members of replica sets are not +guaranteed to reflect the current state of the primary, and the state +of secondaries will trail the primary by some amount of time. Often, +applications don't rely on this kind of strict consistency, but +application developers should always consider the needs of their +application before setting read preference. + +For more information on read preference or on the read preference +modes, see :doc:`/core/read-preference` and +:ref:`replica-set-read-preference-modes`. diff --git a/source/core/read-operations-cursors.txt b/source/core/read-operations-cursors.txt new file mode 100644 index 00000000000..134df32f67a --- /dev/null +++ b/source/core/read-operations-cursors.txt @@ -0,0 +1,256 @@ +.. _read-operations-cursors: + +Cursors +------- + +The :method:`find() ` method returns a +:term:`cursor` to the results; however, in the :program:`mongo` shell, +if the returned cursor is not assigned to a variable, then the cursor +is automatically iterated up to 20 times [#set-shell-batch-size]_ to print +up to the first 20 documents that match the query, as in the following +example: + +.. code-block:: javascript + + db.inventory.find( { type: 'food' } ); + +When you assign the :method:`find() ` to a +variable: + +- you can call the cursor variable in the shell to iterate up to 20 + times [#set-shell-batch-size]_ and print the matching documents, as in + the following example: + + .. code-block:: javascript + + var myCursor = db.inventory.find( { type: 'food' } ); + + myCursor + +- you can use the cursor method :method:`next() ` to + access the documents, as in the following example: + + .. code-block:: javascript + + var myCursor = db.inventory.find( { type: 'food' } ); + var myDocument = myCursor.hasNext() ? myCursor.next() : null; + + if (myDocument) { + var myItem = myDocument.item; + print(tojson(myItem)); + } + + As an alternative print operation, consider the ``printjson()`` + helper method to replace ``print(tojson())``: + + .. code-block:: javascript + + if (myDocument) { + var myItem = myDocument.item; + printjson(myItem); + } + +- you can use the cursor method :method:`forEach() ` + to iterate the cursor and access the documents, as in the following + example: + + .. code-block:: javascript + + var myCursor = db.inventory.find( { type: 'food' } ); + + myCursor.forEach(printjson); + +See :ref:`JavaScript cursor methods ` and your +:doc:`driver ` documentation for more +information on cursor methods. + +.. [#set-shell-batch-size] You can use the ``DBQuery.shellBatchSize`` to + change the number of iteration from the default value ``20``. See + :ref:`mongo-shell-executing-queries` for more information. + +Iterator Index +~~~~~~~~~~~~~~ + +In the :program:`mongo` shell, you can use the +:method:`~cursor.toArray()` method to iterate the cursor and return +the documents in an array, as in the following: + +.. code-block:: javascript + + var myCursor = db.inventory.find( { type: 'food' } ); + var documentArray = myCursor.toArray(); + var myDocument = documentArray[3]; + +The :method:`~cursor.toArray()` method loads into RAM all +documents returned by the cursor; the :method:`~cursor.toArray()` +method exhausts the cursor. + +Additionally, some :doc:`drivers ` provide +access to the documents by using an index on the cursor (i.e. +``cursor[index]``). This is a shortcut for first calling the +:method:`~cursor.toArray()` method and then using an index +on the resulting array. + +Consider the following example: + +.. code-block:: javascript + + var myCursor = db.inventory.find( { type: 'food' } ); + var myDocument = myCursor[3]; + +The ``myCursor[3]`` is equivalent to the following example: + +.. code-block:: javascript + + myCursor.toArray() [3]; + +.. _cursor-behaviors: + +Cursor Behaviors +~~~~~~~~~~~~~~~~ + +Consider the following behaviors related to cursors: + +- By default, the server will automatically close the cursor after 10 + minutes of inactivity or if client has exhausted the cursor. To + override this behavior, you can specify the ``noTimeout`` + :meta-driver:`wire protocol flag ` in + your query; however, you should either close the cursor manually or + exhaust the cursor. In the :program:`mongo` shell, you can set the + ``noTimeout`` flag: + + .. code-block:: javascript + + var myCursor = db.inventory.find().addOption(DBQuery.Option.noTimeout); + + See your :doc:`driver ` documentation for + information on setting the ``noTimeout`` flag. See + :ref:`cursor-flags` for a complete list of available cursor flags. + +- Because the cursor is not isolated during its lifetime, intervening + write operations may result in a cursor that returns a single + document [#single-document-def]_ more than once. To handle this + situation, see the information on :ref:`snapshot mode + `. + +- The MongoDB server returns the query results in batches: + + - For most queries, the *first* batch returns 101 documents or just + enough documents to exceed 1 megabyte. Subsequent batch size is 4 + megabytes. To override the default size of the batch, see + :method:`~cursor.batchSize()` and :method:`~cursor.limit()`. + + - For queries that include a sort operation *without* an index, the + server must load all the documents in memory to perform the sort + and will return all documents in the first batch. + + - Batch size will not exceed the :ref:`maximum BSON document size + `. + + - As you iterate through the cursor and reach the end of the returned + batch, if there are more results, :method:`cursor.next()` will + perform a :data:`getmore operation ` to retrieve the + next batch. + + To see how many documents remain in the batch as you iterate the + cursor, you can use the :method:`~cursor.objsLeftInBatch()` method, + as in the following example: + + .. code-block:: javascript + + var myCursor = db.inventory.find(); + + var myFirstDocument = myCursor.hasNext() ? myCursor.next() : null; + + myCursor.objsLeftInBatch(); + +- You can use the command :dbcommand:`cursorInfo` to retrieve the + following information on cursors: + + - total number of open cursors + + - size of the client cursors in current use + + - number of timed out cursors since the last server restart + + Consider the following example: + + .. code-block:: javascript + + db.runCommand( { cursorInfo: 1 } ) + + The result from the command returns the following document: + + .. code-block:: javascript + + { + "totalOpen" : , + "clientCursors_size" : , + "timedOut" : , + "ok" : 1 + } + +.. [#single-document-def] A single document relative to value of the + ``_id`` field. A cursor cannot return the same document more than + once *if* the document has not changed. + +.. _cursor-flags: + +Cursor Flags +~~~~~~~~~~~~ + +The :program:`mongo` shell provides the following cursor flags: + +- ``DBQuery.Option.tailable`` +- ``DBQuery.Option.slaveOk`` +- ``DBQuery.Option.oplogReplay`` +- ``DBQuery.Option.noTimeout`` +- ``DBQuery.Option.awaitData`` +- ``DBQuery.Option.exhaust`` +- ``DBQuery.Option.partial`` + +.. _read-operations-aggregation: + +Aggregation +~~~~~~~~~~~ + +.. versionchanged:: 2.2 + +MongoDB can perform some basic data aggregation operations on results +before returning data to the application. These operations are not +queries; they use :term:`database commands ` rather +than queries, and they do not return a cursor. However, they still +require MongoDB to read data. + +Running aggregation operations on the database side can be more +efficient than running them in the application layer and can reduce +the amount of data MongoDB needs to send to the application. These +aggregation operations include basic grouping, counting, and even +processing data using a map reduce framework. Additionally, in 2.2 +MongoDB provides a complete aggregation framework for more rich +aggregation operations. + +The aggregation framework provides users with a "pipeline" like +framework: documents enter from a collection and then pass through a +series of steps by a sequence of :ref:`pipeline operators +` that manipulate and +transform the documents until they're output at the end. The +aggregation framework is accessible via the :dbcommand:`aggregate` +command or the :method:`db.collection.aggregate()` helper in the +:program:`mongo` shell. + +For more information on the aggregation framework see +:doc:`/aggregation`. + +Additionally, MongoDB provides a number of simple data aggregation +operations for more basic data aggregation operations: + +- :dbcommand:`count` (:method:`~cursor.count()`) + +- :dbcommand:`distinct` (:method:`db.collection.distinct()`) + +- :dbcommand:`group` (:method:`db.collection.group()`) + +- :dbcommand:`mapReduce`. (Also consider + :method:`~db.collection.mapReduce()` and + :doc:`/core/map-reduce`.) \ No newline at end of file diff --git a/source/core/read-operations-indexes.txt b/source/core/read-operations-indexes.txt new file mode 100644 index 00000000000..e5acebbe000 --- /dev/null +++ b/source/core/read-operations-indexes.txt @@ -0,0 +1,305 @@ +.. _read-operations-indexing: + +Indexes +------- + +Indexes improve the efficiency of read operations by reducing the +amount of data that query operations need to process and thereby +simplifying the work associated with fulfilling queries within +MongoDB. The indexes themselves are a special data structure that +MongoDB maintains when inserting or modifying documents, and any given +index can: support and optimize specific queries, sort operations, and +allow for more efficient storage utilization. For more information +about indexes in MongoDB see: :doc:`/indexes` and :doc:`/core/indexes`. + +You can create indexes using the :method:`db.collection.ensureIndex()` method +in the :program:`mongo` shell, as in the following prototype +operation: + +.. code-block:: javascript + + db.collection.ensureIndex( { : , : , ... } ) + +- The ``field`` specifies the field to index. The field may be a field + from a subdocument, using :term:`dot notation` to specify subdocument + fields. + + You can create an index on a single field or a :ref:`compound index + ` that includes multiple fields in the index. + +- The ``order`` option is specifies either ascending ( ``1`` ) or + descending ( ``-1`` ). + + MongoDB can read the index in either direction. In most cases, you + only need to specify :ref:`indexing order + ` to support sort operations in + compound queries. + +.. _read-operations-covered-query: + +Covering a Query +~~~~~~~~~~~~~~~~ + +An index :ref:`covers ` a query, a *covered +query*, when: + +- all the fields in the :ref:`query ` + are part of that index, **and** + +- all the fields returned in the documents that match the query are in + the same index. + +For these queries, MongoDB does not need to inspect at documents +outside of the index, which is often more efficient than inspecting +entire documents. + +.. example:: + + Given a collection ``inventory`` with the following index on the + ``type`` and ``item`` fields: + + .. code-block:: sh + + { type: 1, item: 1 } + + This index will cover the following query on the ``type`` and ``item`` + fields, which returns only the ``item`` field: + + .. code-block:: javascript + + db.inventory.find( { type: "food", item:/^c/ }, + { item: 1, _id: 0 } ) + + However, this index will **not** cover the following query, which + returns the ``item`` field **and** the ``_id`` field: + + .. code-block:: javascript + + db.inventory.find( { type: "food", item:/^c/ }, + { item: 1 } ) + +See :ref:`indexes-covered-queries` for more information on the +behavior and use of covered queries. + +Measuring Index Use +~~~~~~~~~~~~~~~~~~~ + +The :method:`~cursor.explain()` cursor method allows you to +inspect the operation of the query system, and is useful for analyzing +the efficiency of queries, and for determining how the query uses the +index. Call the :method:`~cursor.explain()` method on a +cursor returned by :method:`~db.collection.find()`, as in the +following example: + +.. code-block:: javascript + + db.inventory.find( { type: 'food' } ).explain() + +.. note:: + + Only use :method:`~cursor.explain()` to test the query + operation, and *not* the timing of query performance. Because + :method:`~cursor.explain()` attempts multiple query + plans, it does not reflect accurate query performance. + +If the above operation could not use an index, the output of +:method:`~cursor.explain()` would resemble the following: + +.. code-block:: javascript + + { + "cursor" : "BasicCursor", + "isMultiKey" : false, + "n" : 5, + "nscannedObjects" : 4000006, + "nscanned" : 4000006, + "nscannedObjectsAllPlans" : 4000006, + "nscannedAllPlans" : 4000006, + "scanAndOrder" : false, + "indexOnly" : false, + "nYields" : 2, + "nChunkSkips" : 0, + "millis" : 1591, + "indexBounds" : { }, + "server" : "mongodb0.example.net:27017" + } + +The ``BasicCursor`` value in the :data:`~explain.cursor` field +confirms that this query does not use an index. The +:data:`explain.nscannedObjects` value shows that MongoDB must scan +4,000,006 documents to return only 5 documents. To increase the +efficiency of the query, create an index on the ``type`` field, as in +the following example: + +.. code-block:: javascript + + db.inventory.ensureIndex( { type: 1 } ) + +Run the :method:`~cursor.explain()` operation, as follows, +to test the use of the index: + +.. code-block:: javascript + + db.inventory.find( { type: 'food' } ).explain() + +Consider the results: + +.. code-block:: javascript + + { + "cursor" : "BtreeCursor type_1", + "isMultiKey" : false, + "n" : 5, + "nscannedObjects" : 5, + "nscanned" : 5, + "nscannedObjectsAllPlans" : 5, + "nscannedAllPlans" : 5, + "scanAndOrder" : false, + "indexOnly" : false, + "nYields" : 0, + "nChunkSkips" : 0, + "millis" : 0, + "indexBounds" : { "type" : [ + [ "food", + "food" ] + ] }, + "server" : "mongodbo0.example.net:27017" } + +The ``BtreeCursor`` value of the :data:`~explain.cursor` field indicates that +the query used an index. This query: + +- returned 5 documents, as indicated by the :data:`~explain.n` field; + +- scanned 5 documents from the index, as indicated by the + :data:`~explain.nscanned` field; + +- then read 5 full documents from the collection, as indicated by + the :data:`~explain.nscannedObjects` field. + + Although the query uses an index to find the matching documents, if + :data:`~explain.indexOnly` is false then an index could + not :ref:`cover ` the query: + MongoDB could not both match the :ref:`query conditions + ` **and** return the results using + only this index. See :ref:`indexes-covered-queries` for more + information. + +.. index:: query optimizer +.. _read-operations-query-optimization: + +Query Optimization +~~~~~~~~~~~~~~~~~~ + +The MongoDB query optimizer processes queries and chooses the most +efficient query plan for a query given the available indexes. The +query system then uses this query plan each time the query runs. The +query optimizer occasionally reevaluates query plans as the content of +the collection changes to ensure optimal query plans. + +To create a new query plan, the query optimizer: + +1. runs the query against several candidate indexes in parallel. + +#. records the matches in a common results buffer + or buffers. + + - If the candidate plans include only :term:`ordered query plans + `, there is a single common results buffer. + + - If the candidate plans include only :term:`unordered query plans + `, there is a single common results buffer. + + - If the candidate plans include *both* :term:`ordered query plans + ` and :term:`unordered query plans + `, there are two common results buffers, one + for the ordered plans and the other for the unordered plans. + + If an index returns a result already returned by another index, the + optimizer skips the duplicate match. In the case of the two buffers, + both buffers are de-duped. + +#. stops the testing of candidate plans and selects an index when one of + the following events occur: + + - An :term:`unordered query plan` has returned all the matching results; *or* + + - An :term:`ordered query plan` has returned all the matching results; *or* + + - An :term:`ordered query plan` has returned a threshold number of + matching results: + + - Version 2.0: Threshold is the query batch size. The default + batch size is 101. + + - Version 2.2: Threshold is 101. + +The selected index becomes the index specified in the query plan; +future iterations of this query or queries with the same query +pattern will use this index. Query pattern refers to query select +conditions that differ only in the values, as in the following two +queries with the same query pattern: + +.. code-block:: javascript + + db.inventory.find( { type: 'food' } ) + db.inventory.find( { type: 'utensil' } ) + +To manually compare the performance of a query using more than one +index, you can use the :method:`hint() ` and +:method:`explain() ` methods in conjunction, as in +the following prototype: + +.. code-block:: javascript + + db.collection.find().hint().explain() + +The following operations each run the same query but will reflect the +use of the different indexes: + +.. code-block:: javascript + + db.inventory.find( { type: 'food' } ).hint( { type: 1 } ).explain() + db.inventory.find( { type: 'food' } ).hint( { type: 1, name: 1 }).explain() + +This returns the statistics regarding the execution of the query. For +more information on the output of :method:`explain() +`, see :doc:`/reference/method/cursor.explain`. + +.. note:: + + If you run :method:`explain() ` without including + :method:`hint() `, the query optimizer reevaluates + the query and runs against multiple indexes before returning the + query statistics. + +As collections change over time, the query optimizer deletes a query +plan and reevaluates the after any of the following events: + +- the collection receives 1,000 write operations. + +- the :dbcommand:`reIndex` rebuilds the index. + +- you add or drop an index. + +- the :program:`mongod` process restarts. + +For more information, see :doc:`/applications/indexes`. + +Query Operations that Cannot Use Indexes Effectively +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Some query operations cannot use indexes effectively or cannot use +indexes at all. Consider the following situations: + +- The inequality operators :operator:`$nin` and :operator:`$ne` are + not very selective, as they often match a large portion of the + index. + + As a result, in most cases, a :operator:`$nin` or :operator:`$ne` + query with an index may perform no better than a :operator:`$nin` or + :operator:`$ne` query that must scan all documents in a collection. + +- Queries that specify regular expressions, with inline JavaScript + regular expressions or :operator:`$regex` operator expressions, + cannot use an index. *However*, the regular expression with anchors + to the beginning of a string *can* use an index. diff --git a/source/core/read-operations-queries.txt b/source/core/read-operations-queries.txt new file mode 100644 index 00000000000..871d7359b30 --- /dev/null +++ b/source/core/read-operations-queries.txt @@ -0,0 +1,401 @@ +.. index:: read operations; query +.. _read-operations-query-operations: +.. _read-operations-queries: + +Read Operations in MongoDB +-------------------------- + +In the :program:`mongo` shell, the :method:`find() +` and :method:`findOne() +` methods perform read operations. The +:method:`find() ` method has the following +syntax: [#formal-query-structure]_ + +.. code-block:: javascript + + db.collection.find( , ) + +- The ``db.collection`` object specifies the database and collection + to query. All queries in MongoDB address a *single* collection. + +.. TODO: why would you need to find the name of the current database? + + You can enter ``db`` in the :program:`mongo` shell to return the + name of the current database. Use the ``show collections`` operation + in the :program:`mongo` shell to list the current collections in the + database. + +- Find queries in MongoDB are :term:`BSON` objects that use a set of + :doc:`query operators ` to describe query + parameters. + + - The ```` argument of the :method:`find() + ` method holds this query document. A read + operation without a query document will return all documents in the + collection. + + - The ```` argument describes the result set in the form + of a document. Projections specify or limit the fields to return. + + Without a projection, the operation will return all fields of the + documents. Specify a projection if your documents are larger, or + when your application only needs a subset of available fields. + +- The order of documents returned by a query is not defined and is not + necessarily consistent unless you specify a sort (:method:`sort() + `). + +For example, the following operation on the ``inventory`` collection +selects all documents where the ``type`` field equals ``'food'`` and +the ``price`` field has a value less than ``9.95`` (using the +:operator:`$lt` operator). The projection limits the response to the +``item`` and ``qty``, and ``_id`` field: + +.. code-block:: javascript + + db.inventory.find( { type: 'food', price: { $lt: 9.95 } }, + { item: 1, qty: 1 } ) + +The :method:`findOne() ` method is similar to +the :method:`find() ` method except the +:method:`findOne() ` method returns a single +document from a collection rather than a cursor. The method has the +syntax: + +.. code-block:: javascript + + db.collection.findOne( , ) + +For additional documentation and examples of the main MongoDB read +operators, refer to the :doc:`/core/read` page of the +:doc:`/crud` section. + +.. [#formal-query-structure] :method:`db.collection.find()` is a + wrapper for the more formal query structure with the + :operator:`$query` operator. + +.. _read-operations-query-document: +.. _read-operations-query-argument: + +The Query Document +~~~~~~~~~~~~~~~~~~ + +This section provides an overview of the query document for MongoDB +queries. See the preceding section for more information on +:ref:`queries in MongoDB `. + +The following examples demonstrate the key properties of the query +document in MongoDB queries, using the :method:`find() +` method from the :program:`mongo` shell, and a +collection of documents named ``inventory``: + +- An empty query document (``{}``) selects all documents in the + collection: + + .. code-block:: javascript + + db.inventory.find( {} ) + + Not specifying a query document to the :method:`find() + ` is equivalent to specifying an empty query + document. Therefore the following operation is equivalent to the + previous operation: + + .. code-block:: javascript + + db.inventory.find() + +- A single-clause query selects all documents in a collection where a + field has a certain value. These are simple "equality" queries. + + In the following example, the query selects all documents in the + collection where the ``type`` field has the value ``snacks``: + + .. code-block:: javascript + + db.inventory.find( { type: "snacks" } ) + +- A single-clause query document can also select all documents in a + collection given a condition or set of conditions for one field in + the collection's documents. Use the :ref:`query operators + ` to specify conditions in a MongoDB query. + + In the following example, the query selects all documents in the + collection where the value of the ``type`` field is either + ``'food'`` or ``'snacks'``: + + .. code-block:: javascript + + db.inventory.find( { type: { $in: [ 'food', 'snacks' ] } } ) + + .. note:: + + Although you can express this query using the :operator:`$or` + operator, choose the :operator:`$in` operator rather than the + :operator:`$or` operator when performing equality checks on the + same field. + +- A compound query can specify conditions for more than one field in + the collection's documents. Implicitly, a logical ``AND`` conjunction + connects the clauses of a compound query so that the query selects + the documents in the collection that match all the conditions. + + In the following example, the query document specifies an equality + match on a single field, followed by a range of values for a second + field using a :ref:`comparison operator `: + + .. code-block:: javascript + + db.inventory.find( { type: 'food', price: { $lt: 9.95 } } ) + + This query selects all documents where the ``type`` field has the + value ``'food'`` **and** the value of the ``price`` field is less + than (:operator:`$lt`) ``9.95``. + +- Using the :operator:`$or` operator, you can specify a compound query + that joins each clause with a logical ``OR`` conjunction so that the + query selects the documents in the collection that match at least one + condition. + + In the following example, the query document selects all documents + in the collection where the field ``qty`` has a value greater than + (:operator:`$gt`) ``100`` **or** the value of the ``price`` + field is less than (:operator:`$lt`) ``9.95``: + + .. code-block:: javascript + + db.inventory.find( { $or: [ { qty: { $gt: 100 } }, + { price: { $lt: 9.95 } } ] + } ) + +- With additional clauses, you can specify precise conditions for + matching documents. In the following example, the compound query + document selects all documents in the collection where the value of + the ``type`` field is ``'food'`` **and** *either* the ``qty`` has a + value greater than (:operator:`$gt`) ``100`` *or* the value of + the ``price`` field is less than (:operator:`$lt`) ``9.95``: + + .. code-block:: javascript + + db.inventory.find( { type: 'food', $or: [ { qty: { $gt: 100 } }, + { price: { $lt: 9.95 } } ] + } ) + +.. _read-operations-subdocuments: + +Subdocuments +```````````` + +When the field holds an embedded document (i.e. subdocument), you can +either specify the entire subdocument as the value of a field, or +"reach into" the subdocument using :term:`dot notation`, to specify +values for individual fields in the subdocument: + +- Equality matches within subdocuments select documents if the + subdocument matches *exactly* the specified subdocument, including the + field order. + + In the following example, the query matches all documents where + the value of the field ``producer`` is a subdocument that contains + *only* the field ``company`` with the value ``'ABC123'`` and the field + ``address`` with the value ``'123 Street'``, in the exact order: + + .. code-block:: javascript + + db.inventory.find( { + producer: { + company: 'ABC123', + address: '123 Street' + } + } + ) + +- Equality matches for specific fields within subdocuments select + documents when the field in the subdocument contains a field that + matches the specified value. + + In the following example, the query uses the :term:`dot notation` to + match all documents where the value of the field ``producer`` is a + subdocument that contains a field ``company`` with the value + ``'ABC123'`` and may contain other fields: + + .. code-block:: javascript + + db.inventory.find( { 'producer.company': 'ABC123' } ) + +.. _read-operations-arrays: + +Arrays +```````` + +When the field holds an array, you can query for values in the array, +and if the array holds sub-documents, you query for specific fields +within the sub-documents using :term:`dot notation`: + +- Equality matches can specify an entire array, to select an array + that matches exactly. In the following example, the query matches all + documents where the value of the field ``tags`` is an array and + holds three elements, ``'fruit'``, ``'food'``, and ``'citrus'``, in + this order: + + .. code-block:: javascript + + db.inventory.find( { tags: [ 'fruit', 'food', 'citrus' ] } ) + +- Equality matches can specify a single element in the array. If the + array contains at least *one* element with the specified value, as + in the following example: the query matches all documents where + the value of the field ``tags`` is an array that contains, as one of + its elements, the element ``'fruit'``: + + .. code-block:: javascript + + db.inventory.find( { tags: 'fruit' } ) + + Equality matches can also select documents by values in an array + using the array index (i.e. position) of the element in the array, + as in the following example: the query uses the :term:`dot notation` + to match all documents where the value of the ``tags`` field is an + array whose first element equals ``'fruit'``: + + .. code-block:: javascript + + db.inventory.find( { 'tags.0' : 'fruit' } ) + +In the following examples, consider an array that contains +subdocuments: + +- If you know the array index of the subdocument, you can specify the + document using the subdocument's position. + + The following example selects all documents where the ``memos`` + contains an array whose first element (i.e. index is ``0``) is a + subdocument with the field ``by`` with the value ``'shipping'``: + + .. code-block:: javascript + + db.inventory.find( { 'memos.0.by': 'shipping' } ) + +- If you do not know the index position of the subdocument, + concatenate the name of the field that contains the array, with a + dot (``.``) and the name of the field in the subdocument. + + The following example selects all documents where the ``memos`` field + contains an array that contains at least one subdocument with the + field ``by`` with the value ``'shipping'``: + + .. code-block:: javascript + + db.inventory.find( { 'memos.by': 'shipping' } ) + +- To match by multiple fields in the subdocument, you can use either + dot notation or the :operator:`$elemMatch` operator: + + The following example uses dot notation to query for documents + where the value of the ``memos`` field is an array that has at least + one subdocument that contains the field ``memo`` equal to ``'on time'`` + and the field ``by`` equal to ``'shipping'``: + + .. code-block:: javascript + + db.inventory.find( + { + 'memos.memo': 'on time', + 'memos.by': 'shipping' + } + ) + + The following example uses :operator:`$elemMatch` to query for + documents where the value of the ``memos`` field is an array that + has at least one subdocument that contains the field ``memo`` equal + to ``'on time'`` and the field ``by`` equal to ``'shipping'``: + + .. code-block:: javascript + + db.inventory.find( { memos: { + $elemMatch: { + memo : 'on time', + by: 'shipping' + } + } + } + ) + +Refer to the :doc:`/reference/operator` document for the complete list +of query operators. + +.. _read-operations-projection: +.. _projection: + +The Projection Argument +~~~~~~~~~~~~~~~~~~~~~~~ + +The :term:`projection` specification limits the fields to return for +all matching documents. Restricting the fields to return can minimize +network transit costs and the costs of deserializing documents in the +application layer. + +The second argument to the :method:`find() ` +method is a projection, and it takes the form of a :term:`document` with +a list of fields for inclusion or exclusion from the result set. You +can either specify the fields to include (e.g. ``{ field: 1 }``) or specify the +fields to exclude (e.g. ``{ field: 0 }``). The ``_id`` field is, by +default, included in the result set. To exclude the ``_id`` field from +the result set, you need to specify in the projection document the +exclusion of the ``_id`` field (i.e. ``{ _id: 0 }``). + +.. note:: + + You cannot combine inclusion and exclusion semantics in a single + projection with the *exception* of the ``_id`` field. + +Consider the following projection specifications in :method:`find() +` operations: + +- If you specify no projection, the :method:`find() + ` method returns all fields of all documents + that match the query. + + .. code-block:: javascript + + db.inventory.find( { type: 'food' } ) + + This operation will return all documents in the ``inventory`` + collection where the value of the ``type`` field is ``'food'``. + +- A projection can explicitly include several fields. In the following + operation, :method:`find() ` method returns + all documents that match the query as well as ``item`` and ``qty`` + fields. The results also include the ``_id`` field: + + .. code-block:: javascript + + db.inventory.find( { type: 'food' }, { item: 1, qty: 1 } ) + +- You can remove the ``_id`` field from the results by specifying its + exclusion in the projection, as in the following example: + + .. code-block:: javascript + + db.inventory.find( { type: 'food' }, { item: 1, qty: 1, _id:0 } ) + + This operation returns all documents that match the query, and + *only* includes the ``item`` and ``qty`` fields in the result set. + +- To exclude a single field or group of fields you can use a + projection in the following form: + + .. code-block:: javascript + + db.inventory.find( { type: 'food' }, { type:0 } ) + + This operation returns all documents where the value of the ``type`` + field is ``food``, but does not include the ``type`` field in the + output. + + With the exception of the ``_id`` field you cannot combine inclusion + and exclusion statements in projection documents. + +The :projection:`$elemMatch` and :projection:`$slice` projection +operators provide more control when projecting only a portion of an +array. \ No newline at end of file diff --git a/source/core/read-operations.txt b/source/core/read-operations.txt index 41cd67eeb1a..d73d8a061a4 100644 --- a/source/core/read-operations.txt +++ b/source/core/read-operations.txt @@ -11,10 +11,6 @@ do not return a cursor but have similar properties as queries. These commands include :dbcommand:`aggregate`, :dbcommand:`count`, and :dbcommand:`distinct`. -This document describes the syntax and structure of the queries -applications use to request data from MongoDB and how different -factors affect the efficiency of reads. - .. note:: All of the examples in this document use the :program:`mongo` @@ -23,1048 +19,16 @@ factors affect the efficiency of reads. Driver `. See your :api:`driver documentation <>` for full API documentation. -.. index:: read operations; query -.. _read-operations-query-operations: -.. _read-operations-queries: - -Queries in MongoDB ------------------- - -In the :program:`mongo` shell, the :method:`find() -` and :method:`findOne() -` methods perform read operations. The -:method:`find() ` method has the following -syntax: [#formal-query-structure]_ - -.. code-block:: javascript - - db.collection.find( , ) - -- The ``db.collection`` object specifies the database and collection - to query. All queries in MongoDB address a *single* collection. - - You can enter ``db`` in the :program:`mongo` shell to return the - name of the current database. Use the ``show collections`` operation - in the :program:`mongo` shell to list the current collections in the - database. - -- Queries in MongoDB are :term:`BSON` objects that use a set of - :doc:`query operators ` to describe query - parameters. - - The ```` argument of the :method:`find() - ` method holds this query document. A read - operation without a query document will return all documents in the - collection. - -- The ```` argument describes the result set in the form of - a document. Projections specify or limit the fields to return. - - Without a projection, the operation will return all - fields of the documents. Specify a projection if your documents are - larger, or when your application only needs a subset of available - fields. - -- The order of documents returned by a query is not defined and is not - necessarily consistent unless you specify a sort (:method:`sort() - `). - -For example, the following operation on the ``inventory`` collection -selects all documents where the ``type`` field equals ``'food'`` and -the ``price`` field has a value less than ``9.95``. The projection -limits the response to the ``item`` and ``qty``, and ``_id`` field: - -.. code-block:: javascript - - db.inventory.find( { type: 'food', price: { $lt: 9.95 } }, - { item: 1, qty: 1 } ) - -The :method:`findOne() ` method is similar to -the :method:`find() ` method except the -:method:`findOne() ` method returns a single -document from a collection rather than a cursor. The method has the -syntax: - -.. code-block:: javascript - - db.collection.findOne( , ) - -For additional documentation and examples of the main MongoDB read -operators, refer to the :doc:`/core/read` page of the -:doc:`/crud` section. - -.. [#formal-query-structure] :method:`db.collection.find()` is a - wrapper for the more formal query structure with the - :operator:`$query` operator. - -.. _read-operations-query-document: -.. _read-operations-query-argument: - -Query Document -~~~~~~~~~~~~~~ - -This section provides an overview of the query document for MongoDB -queries. See the preceding section for more information on -:ref:`queries in MongoDB `. - -The following examples demonstrate the key properties of the query -document in MongoDB queries, using the :method:`find() -` method from the :program:`mongo` shell, and a -collection of documents named ``inventory``: - -- An empty query document (``{}``) selects all documents in the - collection: - - .. code-block:: javascript - - db.inventory.find( {} ) - - Not specifying a query document to the :method:`find() - ` is equivalent to specifying an empty query - document. Therefore the following operation is equivalent to the - previous operation: - - .. code-block:: javascript - - db.inventory.find() - -- A single-clause query selects all documents in a collection where a - field has a certain value. These are simple "equality" queries. - - In the following example, the query selects all documents in the - collection where the ``type`` field has the value ``snacks``: - - .. code-block:: javascript - - db.inventory.find( { type: "snacks" } ) - -- A single-clause query document can also select all documents in a - collection given a condition or set of conditions for one field in - the collection's documents. Use the :ref:`query operators - ` to specify conditions in a MongoDB query. - - In the following example, the query selects all documents in the - collection where the value of the ``type`` field is either - ``'food'`` or ``'snacks'``: - - .. code-block:: javascript - - db.inventory.find( { type: { $in: [ 'food', 'snacks' ] } } ) - - .. note:: - - Although you can express this query using the :operator:`$or` - operator, choose the :operator:`$in` operator rather than the - :operator:`$or` operator when performing equality checks on the - same field. - -- A compound query can specify conditions for more than one field in - the collection's documents. Implicitly, a logical ``AND`` conjunction - connects the clauses of a compound query so that the query selects - the documents in the collection that match all the conditions. - - In the following example, the query document specifies an equality - match on a single field, followed by a range of values for a second - field using a :ref:`comparison operator `: - - .. code-block:: javascript - - db.inventory.find( { type: 'food', price: { $lt: 9.95 } } ) - - This query selects all documents where the ``type`` field has the - value ``'food'`` **and** the value of the ``price`` field is less - than (:operator:`$lt`) ``9.95``. - -- Using the :operator:`$or` operator, you can specify a compound query - that joins each clause with a logical ``OR`` conjunction so that the - query selects the documents in the collection that match at least one - condition. - - In the following example, the query document selects all documents - in the collection where the field ``qty`` has a value greater than - (:operator:`$gt`) ``100`` **or** the value of the ``price`` - field is less than (:operator:`$lt`) ``9.95``: - - .. code-block:: javascript - - db.inventory.find( { $or: [ { qty: { $gt: 100 } }, - { price: { $lt: 9.95 } } ] - } ) - -- With additional clauses, you can specify precise conditions for - matching documents. In the following example, the compound query - document selects all documents in the collection where the value of - the ``type`` field is ``'food'`` **and** *either* the ``qty`` has a - value greater than (:operator:`$gt`) ``100`` *or* the value of - the ``price`` field is less than (:operator:`$lt`) ``9.95``: - - .. code-block:: javascript - - db.inventory.find( { type: 'food', $or: [ { qty: { $gt: 100 } }, - { price: { $lt: 9.95 } } ] - } ) - -.. _read-operations-subdocuments: - -Subdocuments -```````````` - -When the field holds an embedded document (i.e. subdocument), you can -either specify the entire subdocument as the value of a field, or -"reach into" the subdocument using :term:`dot notation`, to specify -values for individual fields in the subdocument: - -- Equality matches within subdocuments select documents if the - subdocument matches *exactly* the specified subdocument, including the - field order. - - In the following example, the query matches all documents where - the value of the field ``producer`` is a subdocument that contains - *only* the field ``company`` with the value ``'ABC123'`` and the field - ``address`` with the value ``'123 Street'``, in the exact order: - - .. code-block:: javascript - - db.inventory.find( { - producer: { - company: 'ABC123', - address: '123 Street' - } - } - ) - -- Equality matches for specific fields within subdocuments select - documents when the field in the subdocument contains a field that - matches the specified value. - - In the following example, the query uses the :term:`dot notation` to - match all documents where the value of the field ``producer`` is a - subdocument that contains a field ``company`` with the value - ``'ABC123'`` and may contain other fields: - - .. code-block:: javascript - - db.inventory.find( { 'producer.company': 'ABC123' } ) - -.. _read-operations-arrays: - -Arrays -```````` - -When the field holds an array, you can query for values in the array, -and if the array holds sub-documents, you query for specific fields -within the sub-documents using :term:`dot notation`: - -- Equality matches can specify an entire array, to select an array - that matches exactly. In the following example, the query matches all - documents where the value of the field ``tags`` is an array and - holds three elements, ``'fruit'``, ``'food'``, and ``'citrus'``, in - this order: - - .. code-block:: javascript - - db.inventory.find( { tags: [ 'fruit', 'food', 'citrus' ] } ) - -- Equality matches can specify a single element in the array. If the - array contains at least *one* element with the specified value, as - in the following example: the query matches all documents where - the value of the field ``tags`` is an array that contains, as one of - its elements, the element ``'fruit'``: - - .. code-block:: javascript - - db.inventory.find( { tags: 'fruit' } ) - - Equality matches can also select documents by values in an array - using the array index (i.e. position) of the element in the array, - as in the following example: the query uses the :term:`dot notation` - to match all documents where the value of the ``tags`` field is an - array whose first element equals ``'fruit'``: - - .. code-block:: javascript - - db.inventory.find( { 'tags.0' : 'fruit' } ) - -In the following examples, consider an array that contains -subdocuments: - -- If you know the array index of the subdocument, you can specify the - document using the subdocument's position. - - The following example selects all documents where the ``memos`` - contains an array whose first element (i.e. index is ``0``) is a - subdocument with the field ``by`` with the value ``'shipping'``: - - .. code-block:: javascript - - db.inventory.find( { 'memos.0.by': 'shipping' } ) - -- If you do not know the index position of the subdocument, - concatenate the name of the field that contains the array, with a - dot (``.``) and the name of the field in the subdocument. - - The following example selects all documents where the ``memos`` field - contains an array that contains at least one subdocument with the - field ``by`` with the value ``'shipping'``: - - .. code-block:: javascript - - db.inventory.find( { 'memos.by': 'shipping' } ) - -- To match by multiple fields in the subdocument, you can use either - dot notation or the :operator:`$elemMatch` operator: - - The following example uses dot notation to query for documents - where the value of the ``memos`` field is an array that has at least - one subdocument that contains the field ``memo`` equal to ``'on time'`` - and the field ``by`` equal to ``'shipping'``: - - .. code-block:: javascript - - db.inventory.find( - { - 'memos.memo': 'on time', - 'memos.by': 'shipping' - } - ) - - The following example uses :operator:`$elemMatch` to query for - documents where the value of the ``memos`` field is an array that - has at least one subdocument that contains the field ``memo`` equal - to ``'on time'`` and the field ``by`` equal to ``'shipping'``: - - .. code-block:: javascript - - db.inventory.find( { memos: { - $elemMatch: { - memo : 'on time', - by: 'shipping' - } - } - } - ) - -Refer to the :doc:`/reference/operator` document for the complete list -of query operators. - -.. _read-operations-projection: -.. _projection: - -Result Projections -~~~~~~~~~~~~~~~~~~ - -The :term:`projection` specification limits the fields to return for -all matching documents. Restricting the fields to return can minimize -network transit costs and the costs of deserializing documents in the -application layer. - -The second argument to the :method:`find() ` -method is a projection, and it takes the form of a :term:`document` with -a list of fields for inclusion or exclusion from the result set. You -can either specify the fields to include (e.g. ``{ field: 1 }``) or specify the -fields to exclude (e.g. ``{ field: 0 }``). The ``_id`` field is, by -default, included in the result set. To exclude the ``_id`` field from -the result set, you need to specify in the projection document the -exclusion of the ``_id`` field (i.e. ``{ _id: 0 }``). - -.. note:: - - You cannot combine inclusion and exclusion semantics in a single - projection with the *exception* of the ``_id`` field. - -Consider the following projection specifications in :method:`find() -` operations: - -- If you specify no projection, the :method:`find() - ` method returns all fields of all documents - that match the query. - - .. code-block:: javascript - - db.inventory.find( { type: 'food' } ) - - This operation will return all documents in the ``inventory`` - collection where the value of the ``type`` field is ``'food'``. - -- A projection can explicitly include several fields. In the following - operation, :method:`find() ` method returns - all documents that match the query as well as ``item`` and ``qty`` - fields. The results also include the ``_id`` field: - - .. code-block:: javascript - - db.inventory.find( { type: 'food' }, { item: 1, qty: 1 } ) - -- You can remove the ``_id`` field from the results by specifying its - exclusion in the projection, as in the following example: - - .. code-block:: javascript - - db.inventory.find( { type: 'food' }, { item: 1, qty: 1, _id:0 } ) - - This operation returns all documents that match the query, and - *only* includes the ``item`` and ``qty`` fields in the result set. - -- To exclude a single field or group of fields you can use a - projection in the following form: - - .. code-block:: javascript - - db.inventory.find( { type: 'food' }, { type:0 } ) - - This operation returns all documents where the value of the ``type`` - field is ``food``, but does not include the ``type`` field in the - output. - - With the exception of the ``_id`` field you cannot combine inclusion - and exclusion statements in projection documents. - -The :projection:`$elemMatch` and :projection:`$slice` projection -operators provide more control when projecting only a portion of an -array. - -.. _read-operations-indexing: - -Indexes -------- - -Indexes improve the efficiency of read operations by reducing the -amount of data that query operations need to process and thereby -simplifying the work associated with fulfilling queries within -MongoDB. The indexes themselves are a special data structure that -MongoDB maintains when inserting or modifying documents, and any given -index can: support and optimize specific queries, sort operations, and -allow for more efficient storage utilization. For more information -about indexes in MongoDB see: :doc:`/indexes` and :doc:`/core/indexes`. - -You can create indexes using the :method:`db.collection.ensureIndex()` method -in the :program:`mongo` shell, as in the following prototype -operation: - -.. code-block:: javascript - - db.collection.ensureIndex( { : , : , ... } ) - -- The ``field`` specifies the field to index. The field may be a field - from a subdocument, using :term:`dot notation` to specify subdocument - fields. - - You can create an index on a single field or a :ref:`compound index - ` that includes multiple fields in the index. - -- The ``order`` option is specifies either ascending ( ``1`` ) or - descending ( ``-1`` ). - - MongoDB can read the index in either direction. In most cases, you - only need to specify :ref:`indexing order - ` to support sort operations in - compound queries. - -.. _read-operations-covered-query: - -Covering a Query -~~~~~~~~~~~~~~~~ - -An index :ref:`covers ` a query, a *covered -query*, when: - -- all the fields in the :ref:`query ` - are part of that index, **and** - -- all the fields returned in the documents that match the query are in - the same index. - -For these queries, MongoDB does not need to inspect at documents -outside of the index, which is often more efficient than inspecting -entire documents. - -.. example:: - - Given a collection ``inventory`` with the following index on the - ``type`` and ``item`` fields: - - .. code-block:: sh - - { type: 1, item: 1 } - - This index will cover the following query on the ``type`` and ``item`` - fields, which returns only the ``item`` field: - - .. code-block:: javascript - - db.inventory.find( { type: "food", item:/^c/ }, - { item: 1, _id: 0 } ) - - However, this index will **not** cover the following query, which - returns the ``item`` field **and** the ``_id`` field: - - .. code-block:: javascript - - db.inventory.find( { type: "food", item:/^c/ }, - { item: 1 } ) - -See :ref:`indexes-covered-queries` for more information on the -behavior and use of covered queries. - -Measuring Index Use -~~~~~~~~~~~~~~~~~~~ - -The :method:`~cursor.explain()` cursor method allows you to -inspect the operation of the query system, and is useful for analyzing -the efficiency of queries, and for determining how the query uses the -index. Call the :method:`~cursor.explain()` method on a -cursor returned by :method:`~db.collection.find()`, as in the -following example: - -.. code-block:: javascript - - db.inventory.find( { type: 'food' } ).explain() - -.. note:: - - Only use :method:`~cursor.explain()` to test the query - operation, and *not* the timing of query performance. Because - :method:`~cursor.explain()` attempts multiple query - plans, it does not reflect accurate query performance. - -If the above operation could not use an index, the output of -:method:`~cursor.explain()` would resemble the following: - -.. code-block:: javascript - - { - "cursor" : "BasicCursor", - "isMultiKey" : false, - "n" : 5, - "nscannedObjects" : 4000006, - "nscanned" : 4000006, - "nscannedObjectsAllPlans" : 4000006, - "nscannedAllPlans" : 4000006, - "scanAndOrder" : false, - "indexOnly" : false, - "nYields" : 2, - "nChunkSkips" : 0, - "millis" : 1591, - "indexBounds" : { }, - "server" : "mongodb0.example.net:27017" - } - -The ``BasicCursor`` value in the :data:`~explain.cursor` field -confirms that this query does not use an index. The -:data:`explain.nscannedObjects` value shows that MongoDB must scan -4,000,006 documents to return only 5 documents. To increase the -efficiency of the query, create an index on the ``type`` field, as in -the following example: - -.. code-block:: javascript - - db.inventory.ensureIndex( { type: 1 } ) - -Run the :method:`~cursor.explain()` operation, as follows, -to test the use of the index: - -.. code-block:: javascript - - db.inventory.find( { type: 'food' } ).explain() - -Consider the results: - -.. code-block:: javascript - - { - "cursor" : "BtreeCursor type_1", - "isMultiKey" : false, - "n" : 5, - "nscannedObjects" : 5, - "nscanned" : 5, - "nscannedObjectsAllPlans" : 5, - "nscannedAllPlans" : 5, - "scanAndOrder" : false, - "indexOnly" : false, - "nYields" : 0, - "nChunkSkips" : 0, - "millis" : 0, - "indexBounds" : { "type" : [ - [ "food", - "food" ] - ] }, - "server" : "mongodbo0.example.net:27017" } - -The ``BtreeCursor`` value of the :data:`~explain.cursor` field indicates that -the query used an index. This query: - -- returned 5 documents, as indicated by the :data:`~explain.n` field; - -- scanned 5 documents from the index, as indicated by the - :data:`~explain.nscanned` field; - -- then read 5 full documents from the collection, as indicated by - the :data:`~explain.nscannedObjects` field. - - Although the query uses an index to find the matching documents, if - :data:`~explain.indexOnly` is false then an index could - not :ref:`cover ` the query: - MongoDB could not both match the :ref:`query conditions - ` **and** return the results using - only this index. See :ref:`indexes-covered-queries` for more - information. - -.. index:: query optimizer -.. _read-operations-query-optimization: - -Query Optimization -~~~~~~~~~~~~~~~~~~ - -The MongoDB query optimizer processes queries and chooses the most -efficient query plan for a query given the available indexes. The -query system then uses this query plan each time the query runs. The -query optimizer occasionally reevaluates query plans as the content of -the collection changes to ensure optimal query plans. - -To create a new query plan, the query optimizer: - -1. runs the query against several candidate indexes in parallel. - -#. records the matches in a common results buffer - or buffers. - - - If the candidate plans include only :term:`ordered query plans - `, there is a single common results buffer. - - - If the candidate plans include only :term:`unordered query plans - `, there is a single common results buffer. - - - If the candidate plans include *both* :term:`ordered query plans - ` and :term:`unordered query plans - `, there are two common results buffers, one - for the ordered plans and the other for the unordered plans. - - If an index returns a result already returned by another index, the - optimizer skips the duplicate match. In the case of the two buffers, - both buffers are de-duped. - -#. stops the testing of candidate plans and selects an index when one of - the following events occur: - - - An :term:`unordered query plan` has returned all the matching results; *or* - - - An :term:`ordered query plan` has returned all the matching results; *or* - - - An :term:`ordered query plan` has returned a threshold number of - matching results: - - - Version 2.0: Threshold is the query batch size. The default - batch size is 101. - - - Version 2.2: Threshold is 101. - -The selected index becomes the index specified in the query plan; -future iterations of this query or queries with the same query -pattern will use this index. Query pattern refers to query select -conditions that differ only in the values, as in the following two -queries with the same query pattern: - -.. code-block:: javascript - - db.inventory.find( { type: 'food' } ) - db.inventory.find( { type: 'utensil' } ) - -To manually compare the performance of a query using more than one -index, you can use the :method:`hint() ` and -:method:`explain() ` methods in conjunction, as in -the following prototype: - -.. code-block:: javascript - - db.collection.find().hint().explain() - -The following operations each run the same query but will reflect the -use of the different indexes: - -.. code-block:: javascript - - db.inventory.find( { type: 'food' } ).hint( { type: 1 } ).explain() - db.inventory.find( { type: 'food' } ).hint( { type: 1, name: 1 }).explain() - -This returns the statistics regarding the execution of the query. For -more information on the output of :method:`explain() -`, see :doc:`/reference/method/cursor.explain`. - -.. note:: - - If you run :method:`explain() ` without including - :method:`hint() `, the query optimizer reevaluates - the query and runs against multiple indexes before returning the - query statistics. - -As collections change over time, the query optimizer deletes a query -plan and reevaluates the after any of the following events: - -- the collection receives 1,000 write operations. - -- the :dbcommand:`reIndex` rebuilds the index. - -- you add or drop an index. - -- the :program:`mongod` process restarts. - -For more information, see :doc:`/applications/indexes`. - -Query Operations that Cannot Use Indexes Effectively -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Some query operations cannot use indexes effectively or cannot use -indexes at all. Consider the following situations: - -- The inequality operators :operator:`$nin` and :operator:`$ne` are - not very selective, as they often match a large portion of the - index. - - As a result, in most cases, a :operator:`$nin` or :operator:`$ne` - query with an index may perform no better than a :operator:`$nin` or - :operator:`$ne` query that must scan all documents in a collection. - -- Queries that specify regular expressions, with inline JavaScript - regular expressions or :operator:`$regex` operator expressions, - cannot use an index. *However*, the regular expression with anchors - to the beginning of a string *can* use an index. - -.. _read-operations-cursors: - -Cursors -------- - -The :method:`find() ` method returns a -:term:`cursor` to the results; however, in the :program:`mongo` shell, -if the returned cursor is not assigned to a variable, then the cursor -is automatically iterated up to 20 times [#set-shell-batch-size]_ to print -up to the first 20 documents that match the query, as in the following -example: - -.. code-block:: javascript - - db.inventory.find( { type: 'food' } ); - -When you assign the :method:`find() ` to a -variable: - -- you can call the cursor variable in the shell to iterate up to 20 - times [#set-shell-batch-size]_ and print the matching documents, as in - the following example: - - .. code-block:: javascript - - var myCursor = db.inventory.find( { type: 'food' } ); - - myCursor - -- you can use the cursor method :method:`next() ` to - access the documents, as in the following example: - - .. code-block:: javascript - - var myCursor = db.inventory.find( { type: 'food' } ); - var myDocument = myCursor.hasNext() ? myCursor.next() : null; - - if (myDocument) { - var myItem = myDocument.item; - print(tojson(myItem)); - } - - As an alternative print operation, consider the ``printjson()`` - helper method to replace ``print(tojson())``: - - .. code-block:: javascript - - if (myDocument) { - var myItem = myDocument.item; - printjson(myItem); - } - -- you can use the cursor method :method:`forEach() ` - to iterate the cursor and access the documents, as in the following - example: - - .. code-block:: javascript - - var myCursor = db.inventory.find( { type: 'food' } ); - - myCursor.forEach(printjson); - -See :ref:`JavaScript cursor methods ` and your -:doc:`driver ` documentation for more -information on cursor methods. - -.. [#set-shell-batch-size] You can use the ``DBQuery.shellBatchSize`` to - change the number of iteration from the default value ``20``. See - :ref:`mongo-shell-executing-queries` for more information. - -Iterator Index -~~~~~~~~~~~~~~ - -In the :program:`mongo` shell, you can use the -:method:`~cursor.toArray()` method to iterate the cursor and return -the documents in an array, as in the following: - -.. code-block:: javascript - - var myCursor = db.inventory.find( { type: 'food' } ); - var documentArray = myCursor.toArray(); - var myDocument = documentArray[3]; - -The :method:`~cursor.toArray()` method loads into RAM all -documents returned by the cursor; the :method:`~cursor.toArray()` -method exhausts the cursor. - -Additionally, some :doc:`drivers ` provide -access to the documents by using an index on the cursor (i.e. -``cursor[index]``). This is a shortcut for first calling the -:method:`~cursor.toArray()` method and then using an index -on the resulting array. - -Consider the following example: - -.. code-block:: javascript - - var myCursor = db.inventory.find( { type: 'food' } ); - var myDocument = myCursor[3]; - -The ``myCursor[3]`` is equivalent to the following example: - -.. code-block:: javascript - - myCursor.toArray() [3]; - -.. _cursor-behaviors: - -Cursor Behaviors -~~~~~~~~~~~~~~~~ - -Consider the following behaviors related to cursors: - -- By default, the server will automatically close the cursor after 10 - minutes of inactivity or if client has exhausted the cursor. To - override this behavior, you can specify the ``noTimeout`` - :meta-driver:`wire protocol flag ` in - your query; however, you should either close the cursor manually or - exhaust the cursor. In the :program:`mongo` shell, you can set the - ``noTimeout`` flag: - - .. code-block:: javascript - - var myCursor = db.inventory.find().addOption(DBQuery.Option.noTimeout); - - See your :doc:`driver ` documentation for - information on setting the ``noTimeout`` flag. See - :ref:`cursor-flags` for a complete list of available cursor flags. - -- Because the cursor is not isolated during its lifetime, intervening - write operations may result in a cursor that returns a single - document [#single-document-def]_ more than once. To handle this - situation, see the information on :ref:`snapshot mode - `. - -- The MongoDB server returns the query results in batches: - - - For most queries, the *first* batch returns 101 documents or just - enough documents to exceed 1 megabyte. Subsequent batch size is 4 - megabytes. To override the default size of the batch, see - :method:`~cursor.batchSize()` and :method:`~cursor.limit()`. - - - For queries that include a sort operation *without* an index, the - server must load all the documents in memory to perform the sort - and will return all documents in the first batch. - - - Batch size will not exceed the :ref:`maximum BSON document size - `. - - - As you iterate through the cursor and reach the end of the returned - batch, if there are more results, :method:`cursor.next()` will - perform a :data:`getmore operation ` to retrieve the - next batch. - - To see how many documents remain in the batch as you iterate the - cursor, you can use the :method:`~cursor.objsLeftInBatch()` method, - as in the following example: - - .. code-block:: javascript - - var myCursor = db.inventory.find(); - - var myFirstDocument = myCursor.hasNext() ? myCursor.next() : null; - - myCursor.objsLeftInBatch(); - -- You can use the command :dbcommand:`cursorInfo` to retrieve the - following information on cursors: - - - total number of open cursors - - - size of the client cursors in current use - - - number of timed out cursors since the last server restart - - Consider the following example: - - .. code-block:: javascript - - db.runCommand( { cursorInfo: 1 } ) - - The result from the command returns the following document: - - .. code-block:: javascript - - { - "totalOpen" : , - "clientCursors_size" : , - "timedOut" : , - "ok" : 1 - } - -.. [#single-document-def] A single document relative to value of the - ``_id`` field. A cursor cannot return the same document more than - once *if* the document has not changed. - -.. _cursor-flags: - -Cursor Flags -~~~~~~~~~~~~ - -The :program:`mongo` shell provides the following cursor flags: - -- ``DBQuery.Option.tailable`` -- ``DBQuery.Option.slaveOk`` -- ``DBQuery.Option.oplogReplay`` -- ``DBQuery.Option.noTimeout`` -- ``DBQuery.Option.awaitData`` -- ``DBQuery.Option.exhaust`` -- ``DBQuery.Option.partial`` - -.. _read-operations-aggregation: - -Aggregation -~~~~~~~~~~~ - -.. versionchanged:: 2.2 - -MongoDB can perform some basic data aggregation operations on results -before returning data to the application. These operations are not -queries; they use :term:`database commands ` rather -than queries, and they do not return a cursor. However, they still -require MongoDB to read data. - -Running aggregation operations on the database side can be more -efficient than running them in the application layer and can reduce -the amount of data MongoDB needs to send to the application. These -aggregation operations include basic grouping, counting, and even -processing data using a map reduce framework. Additionally, in 2.2 -MongoDB provides a complete aggregation framework for more rich -aggregation operations. - -The aggregation framework provides users with a "pipeline" like -framework: documents enter from a collection and then pass through a -series of steps by a sequence of :ref:`pipeline operators -` that manipulate and -transform the documents until they're output at the end. The -aggregation framework is accessible via the :dbcommand:`aggregate` -command or the :method:`db.collection.aggregate()` helper in the -:program:`mongo` shell. - -For more information on the aggregation framework see -:doc:`/aggregation`. - -Additionally, MongoDB provides a number of simple data aggregation -operations for more basic data aggregation operations: - -- :dbcommand:`count` (:method:`~cursor.count()`) - -- :dbcommand:`distinct` (:method:`db.collection.distinct()`) - -- :dbcommand:`group` (:method:`db.collection.group()`) - -- :dbcommand:`mapReduce`. (Also consider - :method:`~db.collection.mapReduce()` and - :doc:`/core/map-reduce`.) - -.. index:: read operation; architecture -.. _read-operations-architecture: - -Architecture ------------- - -.. index:: read operation; connection pooling -.. index:: connection pooling; read operations -.. _read-operations-connection-pooling: - -Read Operations from Sharded Clusters -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -:term:`Sharded clusters ` allow you to partition a -data set among a cluster of :program:`mongod` in a way that is nearly -transparent to the application. See the :doc:`/sharding` section of -this manual for additional information about these deployments. - -For a sharded cluster, you issue all operations to one of the -:program:`mongos` instances associated with the -cluster. :program:`mongos` instances route operations to the -:program:`mongod` in the cluster and behave like :program:`mongod` -instances to the application. Read operations to a sharded collection -in a sharded cluster are largely the same as operations to a :term:`replica -set` or :term:`standalone` instances. See the section on :ref:`Read -Operations in Sharded Clusters ` for more -information. - -In sharded deployments, the :program:`mongos` instance routes -the queries from the clients to the :program:`mongod` instances that -hold the data, using the cluster metadata stored in the :ref:`config -database `. - -For sharded collections, if queries do not include the :ref:`shard key -`, the :program:`mongos` must direct the query to -all shards in a collection. These *scatter gather* queries can be -inefficient, particularly on larger clusters, and are unfeasible for -routine operations. - -For more information on read operations in sharded clusters, consider -the following resources: - -- :ref:`An Introduction to Shard Keys ` -- :ref:`Shard Key Internals and Operations ` -- :ref:`Querying Sharded Clusters ` -- :doc:`/core/sharded-cluster-query-router` - -Read Operations from Replica Sets -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -:term:`Replica sets ` use :term:`read preferences ` to determine where and how to route read operations to -members of the replica set. By default, MongoDB always reads data from -a replica set's :term:`primary`. You can modify that behavior by -changing the :ref:`read preference mode -`. - -You can configure the :ref:`read preference mode -` on a per-connection or -per-operation basis to allow reads from :term:`secondaries -` to: - -- reduce latency in multi-data-center deployments, - -- improve read throughput by distributing high read-volumes (relative - to write volume), +The following documents describe the syntax and structure of the +queries applications use to request data from MongoDB and how different +factors affect the efficiency of reads. -- for backup operations, and/or +.. toctree:: + :maxdepth: 1 -- to allow reads during :ref:`failover ` - situations. + /core/read-operations-queries + /core/read-operations-indexes + /core/read-operations-cursors + /core/read-operations-architecture-considerations -Read operations from secondary members of replica sets are not -guaranteed to reflect the current state of the primary, and the state -of secondaries will trail the primary by some amount of time. Often, -applications don't rely on this kind of strict consistency, but -application developers should always consider the needs of their -application before setting read preference. -For more information on read preference or on the read preference -modes, see :doc:`/core/read-preference` and -:ref:`replica-set-read-preference-modes`. From b8d8dbb25988d34d0afa264e111fad6db72a3b8d Mon Sep 17 00:00:00 2001 From: Sam Kleinman Date: Tue, 9 Jul 2013 15:49:22 -0400 Subject: [PATCH 02/14] crud-reorg: heading levels and default domains --- ...operations-architecture-considerations.txt | 12 +++++++++--- source/core/read-operations-cursors.txt | 17 +++++++++++------ source/core/read-operations-indexes.txt | 12 +++++++++--- source/core/read-operations-queries.txt | 19 +++++++++++-------- 4 files changed, 40 insertions(+), 20 deletions(-) diff --git a/source/core/read-operations-architecture-considerations.txt b/source/core/read-operations-architecture-considerations.txt index fd0638adb84..0fd7ccd0c12 100644 --- a/source/core/read-operations-architecture-considerations.txt +++ b/source/core/read-operations-architecture-considerations.txt @@ -1,15 +1,21 @@ .. index:: read operation; architecture .. _read-operations-architecture: +============ Architecture ------------- +============ + +.. default-domain:: mongodb .. index:: read operation; connection pooling .. index:: connection pooling; read operations .. _read-operations-connection-pooling: +.. might make sense to break this in half and move it to + replication/sharding and cross reference here? + Read Operations from Sharded Clusters -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------------------- :term:`Sharded clusters ` allow you to partition a data set among a cluster of :program:`mongod` in a way that is nearly @@ -46,7 +52,7 @@ the following resources: - :doc:`/core/sharded-cluster-query-router` Read Operations from Replica Sets -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +--------------------------------- :term:`Replica sets ` use :term:`read preferences ` to determine where and how to route read operations to diff --git a/source/core/read-operations-cursors.txt b/source/core/read-operations-cursors.txt index 134df32f67a..cd007c43bd7 100644 --- a/source/core/read-operations-cursors.txt +++ b/source/core/read-operations-cursors.txt @@ -1,7 +1,12 @@ .. _read-operations-cursors: +======= Cursors -------- +======= + +.. default-domain:: mongodb + +.. TODO restructure introduction and remove most of the list. The :method:`find() ` method returns a :term:`cursor` to the results; however, in the :program:`mongo` shell, @@ -69,7 +74,7 @@ information on cursor methods. :ref:`mongo-shell-executing-queries` for more information. Iterator Index -~~~~~~~~~~~~~~ +-------------- In the :program:`mongo` shell, you can use the :method:`~cursor.toArray()` method to iterate the cursor and return @@ -107,7 +112,7 @@ The ``myCursor[3]`` is equivalent to the following example: .. _cursor-behaviors: Cursor Behaviors -~~~~~~~~~~~~~~~~ +---------------- Consider the following behaviors related to cursors: @@ -197,7 +202,7 @@ Consider the following behaviors related to cursors: .. _cursor-flags: Cursor Flags -~~~~~~~~~~~~ +------------ The :program:`mongo` shell provides the following cursor flags: @@ -212,7 +217,7 @@ The :program:`mongo` shell provides the following cursor flags: .. _read-operations-aggregation: Aggregation -~~~~~~~~~~~ +----------- .. versionchanged:: 2.2 @@ -253,4 +258,4 @@ operations for more basic data aggregation operations: - :dbcommand:`mapReduce`. (Also consider :method:`~db.collection.mapReduce()` and - :doc:`/core/map-reduce`.) \ No newline at end of file + :doc:`/core/map-reduce`.) diff --git a/source/core/read-operations-indexes.txt b/source/core/read-operations-indexes.txt index e5acebbe000..1ecc4cfa295 100644 --- a/source/core/read-operations-indexes.txt +++ b/source/core/read-operations-indexes.txt @@ -1,7 +1,13 @@ .. _read-operations-indexing: +======= Indexes -------- +======= + +.. TODO this page needs to be retitled. also restructure introduction + and remove lists when needed. + +.. default-domain:: mongodb Indexes improve the efficiency of read operations by reducing the amount of data that query operations need to process and thereby @@ -38,7 +44,7 @@ operation: .. _read-operations-covered-query: Covering a Query -~~~~~~~~~~~~~~~~ +---------------- An index :ref:`covers ` a query, a *covered query*, when: @@ -286,7 +292,7 @@ plan and reevaluates the after any of the following events: For more information, see :doc:`/applications/indexes`. Query Operations that Cannot Use Indexes Effectively -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +---------------------------------------------------- Some query operations cannot use indexes effectively or cannot use indexes at all. Consider the following situations: diff --git a/source/core/read-operations-queries.txt b/source/core/read-operations-queries.txt index 871d7359b30..7393f92ccda 100644 --- a/source/core/read-operations-queries.txt +++ b/source/core/read-operations-queries.txt @@ -2,8 +2,11 @@ .. _read-operations-query-operations: .. _read-operations-queries: +========================== Read Operations in MongoDB --------------------------- +========================== + +.. default-domain:: mongodb In the :program:`mongo` shell, the :method:`find() ` and :method:`findOne() @@ -77,8 +80,8 @@ operators, refer to the :doc:`/core/read` page of the .. _read-operations-query-document: .. _read-operations-query-argument: -The Query Document -~~~~~~~~~~~~~~~~~~ +Query Documents +--------------- This section provides an overview of the query document for MongoDB queries. See the preceding section for more information on @@ -184,7 +187,7 @@ collection of documents named ``inventory``: .. _read-operations-subdocuments: Subdocuments -```````````` +~~~~~~~~~~~~ When the field holds an embedded document (i.e. subdocument), you can either specify the entire subdocument as the value of a field, or @@ -226,7 +229,7 @@ values for individual fields in the subdocument: .. _read-operations-arrays: Arrays -```````` +~~~~~~ When the field holds an array, you can query for values in the array, and if the array holds sub-documents, you query for specific fields @@ -327,8 +330,8 @@ of query operators. .. _read-operations-projection: .. _projection: -The Projection Argument -~~~~~~~~~~~~~~~~~~~~~~~ +Projections +----------- The :term:`projection` specification limits the fields to return for all matching documents. Restricting the fields to return can minimize @@ -398,4 +401,4 @@ Consider the following projection specifications in :method:`find() The :projection:`$elemMatch` and :projection:`$slice` projection operators provide more control when projecting only a portion of an -array. \ No newline at end of file +array. From 27d21dd0de44e1f507440a9a536de5e281040abb Mon Sep 17 00:00:00 2001 From: schmalliso Date: Wed, 10 Jul 2013 10:51:40 -0400 Subject: [PATCH 03/14] crud-reorg: write operations first pass --- source/core/write-operations-architecture.txt | 62 ++++ source/core/write-operations-bulk-inserts.txt | 52 +++ .../core/write-operations-updates-padding.txt | 122 +++++++ source/core/write-operations.txt | 331 ++++-------------- 4 files changed, 296 insertions(+), 271 deletions(-) create mode 100644 source/core/write-operations-architecture.txt create mode 100644 source/core/write-operations-bulk-inserts.txt create mode 100644 source/core/write-operations-updates-padding.txt diff --git a/source/core/write-operations-architecture.txt b/source/core/write-operations-architecture.txt new file mode 100644 index 00000000000..c6f332904be --- /dev/null +++ b/source/core/write-operations-architecture.txt @@ -0,0 +1,62 @@ +================================================= +Architectural Considerations for Write Operations +================================================= + +.. default-domain:: mongodb + +.. _write-operations-replica-sets: + +Replica Sets +------------ + +In :term:`replica sets `, all write operations go to the +set's :term:`primary`, which applies the write operation then records +the operations on the primary's operation log or :term:`oplog`. The +oplog is a reproducible sequence of operations to the data +set. :term:`Secondary` members of the set are continuously replicating the +oplog and applying the operations to themselves in an asynchronous +process. + +Large volumes of write operations, particularly bulk operations, may +create situations where the secondary members have difficulty applying +the replicating operations from the primary at a sufficient rate: this +can cause the secondary's state to fall behind that of the primary. Secondaries +that are significantly behind the primary present problems for normal +operation of the replica set, particularly :ref:`failover +` in the form of :ref:`rollbacks +` as well as general :doc:`read consistency +`. + +To help avoid this issue, you can customize the :ref:`write concern +` to return confirmation of the write +operation to another member [#write-concern-throttling]_ of the replica +set every 100 or 1,000 operations. This provides an opportunity for +secondaries to catch up with the primary. Write concern can slow the +overall progress of write operations but ensure that the secondaries +can maintain a largely current state with respect to the primary. + +For more information on replica sets and write operations, see +:ref:`replica-set-write-concern`, :ref:`replica-set-oplog-sizing`, and +:doc:`/tutorial/change-oplog-size`. + +.. [#write-concern-throttling] Calling :dbcommand:`getLastError` + intermittently with a ``w`` value of ``2`` or ``majority`` will + slow the throughput of write traffic; however, this practice will + allow the secondaries to remain current with the state of the + primary. + +.. _write-operations-sharded-clusters: + +Sharded Clusters +---------------- + +In a :term:`sharded cluster`, MongoDB directs a given write operation to +a :term:`shard` and then performs the write on a particular +:term:`chunk` on that shard. Shards and chunks are range-based. +:term:`Shard keys ` affect how MongoDB distributes documents +among shards. Choosing the correct shard key can have a great impact on +the performance, capability, and functioning of your database and +cluster. + +For more information, see :doc:`/administration/sharded-clusters` and +:ref:`write-operations-bulk-insert`. diff --git a/source/core/write-operations-bulk-inserts.txt b/source/core/write-operations-bulk-inserts.txt new file mode 100644 index 00000000000..cb45c41529d --- /dev/null +++ b/source/core/write-operations-bulk-inserts.txt @@ -0,0 +1,52 @@ +.. _write-operations-bulk-insert: + +======================= +Bulk Inserts in MongoDB +======================= + +.. default-domain:: mongodb + +In some situations you may need to insert or ingest a large amount of +data into a MongoDB database. These *bulk inserts* have some +special considerations that are different from other write +operations. + +.. TODO: section on general write operation considerations? + +The :method:`insert() ` method, when passed an +array of documents, will perform a bulk insert, and inserts each +document atomically. :doc:`Drivers ` +provide their own interface for this kind of operation. + +.. versionadded:: 2.2 + :method:`insert() ` in the :program:`mongo` + shell gained support for bulk inserts in version 2.2. + +Bulk insert can significantly increase performance by amortizing +:ref:`write concern ` costs. In the +drivers, you can configure write concern for batches rather than on a +per-document level. + +Drivers also have a ``ContinueOnError`` option in their insert +operation, so that the bulk operation will continue to insert +remaining documents in a batch even if an insert fails. + +.. note:: + + .. versionadded:: 2.0 + Support for ``ContinueOnError`` depends on version 2.0 of the + core :program:`mongod` and :program:`mongos` components. + +If the bulk insert process generates more than one error in a batch +job, the client will only receive the most recent error. All bulk +operations to a :term:`sharded collection ` run with +``ContinueOnError``, which applications cannot disable. See +:ref:`sharding-bulk-inserts` section for more information on +consideration for bulk inserts in sharded clusters. + +For more information see your :doc:`driver documentation +` for details on performing bulk inserts in +your application. Also consider the following resources: +:ref:`write-operations-sharded-clusters`, +:ref:`sharding-bulk-inserts`, and +:doc:`/core/import-export`. diff --git a/source/core/write-operations-updates-padding.txt b/source/core/write-operations-updates-padding.txt new file mode 100644 index 00000000000..ca3efffa42d --- /dev/null +++ b/source/core/write-operations-updates-padding.txt @@ -0,0 +1,122 @@ +================================ +Update Considerations in MongoDB +================================ + +.. default-domain:: mongodb + +Each document in a MongoDB collection has allocated *record* space +which includes the entire document *and* a small amount of +padding. This padding makes it possible for update operations to +increase the size of a document slightly without causing the document +to outgrow the allocated record size. + +Documents in MongoDB can grow up to the full maximum :limit:`BSON +document size `. However, when documents outgrow +their allocated record size MongoDB must allocate a new record and +move the document to the new record. Update operations that do not +cause a document to grow, (i.e. *in-place* updates,) are significantly +more efficient than those updates that cause document growth. Use +:doc:`data models ` that minimize the need for +document growth when possible. + +For complete examples of update operations, see +:doc:`/core/update`. + +.. _write-operations-padding-factor: + +Padding Factor +-------------- + +If an update operation does not cause the document to increase in +size, MongoDB can apply the update in-place. Some updates +change the size of the document, for example using the +:operator:`$push` operator to append a sub-document to an array can +cause the top level document to grow beyond its allocated space. + +When documents grow, MongoDB relocates the document on disk with +enough contiguous space to hold the document. These relocations +take longer than in-place updates, particularly if the collection has +indexes that MongoDB must update all index entries. If +collection has many indexes, the move will impact write throughput. + +To minimize document movements, MongoDB employs padding. MongoDB +adaptively learns if documents in a collection tend to grow, and if +they do, adds a :data:`~collStats.paddingFactor` so that the documents +have room to grow on subsequent writes. The +:data:`~collStats.paddingFactor` indicates the padding for new inserts +and moves. + +.. versionadded:: 2.2 + You can use the :dbcommand:`collMod` command + with the :collflag:`usePowerOf2Sizes` flag so that MongoDB + allocates document space in sizes that are powers of 2. This helps + ensure that MongoDB can efficiently reuse the space freed as a + result of deletions or document relocations. As with all padding, + using document space allocations with power of 2 sizes minimizes, + but does not eliminate, document movements. + +To check the current :data:`~collStats.paddingFactor` on a collection, you can +run the :method:`db.collection.stats()` operation in the +:program:`mongo` shell, as in the following example: + +.. code-block:: javascript + + db.myCollection.stats() + +Since MongoDB writes each document at a different point in time, the +padding for each document will not be the same. You can calculate the +padding size by subtracting 1 from the :data:`~collStats.paddingFactor`, for +example: + +.. code-block:: none + + padding size = (paddingFactor - 1) * . + +For example, a :data:`~collStats.paddingFactor` of ``1.0`` specifies no padding +whereas a paddingFactor of ``1.5`` specifies a padding size of ``0.5`` or 50 +percent (50%) of the document size. + +Because the :data:`~collStats.paddingFactor` is relative to the size of each +document, you cannot calculate the exact amount of padding for a +collection based on the average document size and padding factor. + +If an update operation causes the document to *decrease* in size, for +instance if you perform an :operator:`$unset` or a :operator:`$pop` +update, the document remains in place and effectively has more +padding. If the document remains this size, the space is not reclaimed +until you perform a :dbcommand:`compact` or a +:dbcommand:`repairDatabase` operation. + +.. note:: + + The following operations remove padding: + + - :dbcommand:`compact`, + - :dbcommand:`repairDatabase`, and + - initial replica sync operations. + + However, with the :dbcommand:`compact` command, you can run the + command with a ``paddingFactor`` or a ``paddingBytes`` parameter. + + Padding is also removed if you use :program:`mongoexport` from a + collection. If you use :program:`mongoimport` into a new + collection, :program:`mongoimport` will not add padding. If you + use :program:`mongoimport` with an existing collection with + padding, :program:`mongoimport` will not affect the existing + padding. + + When a database operation removes padding, subsequent update that + require changes in record sizes will have reduced throughput until + the collection's padding factor grows. Padding does not affect + in-place, and after :dbcommand:`compact`, + :dbcommand:`repairDatabase`, and replica set initial sync + the collection will require less storage. + +.. COMMENT -- not sure if we really need this manual padding info or if + it belongs here, but ... + +.. seealso:: + + - :ref:`faq-developers-manual-padding` + + - The :operator:`$inc` for in-place updates. diff --git a/source/core/write-operations.txt b/source/core/write-operations.txt index c45957da938..7d9aed1e251 100644 --- a/source/core/write-operations.txt +++ b/source/core/write-operations.txt @@ -18,12 +18,72 @@ This document introduces the write operators available in MongoDB as well as presents strategies to increase the efficiency of writes in applications. +Write Operations in MongoDB +--------------------------- + +[words about general considerations for write operations here] + +.. _write-operations-isolation: + +Isolation +~~~~~~~~~ + +When a single write operation modifies multiple documents, the +operation as a whole is not atomic, and other operations may +interleave. The modification of a single document, or record, is always +atomic, even if the write operation modifies multiple sub-document +*within* the single record. + +No other operations are atomic; however, you can attempt to isolate a +write operation that affects multiple documents using the +:doc:`isolation operator `. + +To isolate a sequence of write operations from other read and write +operations, see :doc:`/tutorial/perform-two-phase-commits`. + +.. _write-operations-indexing: + +Impact on Indexes +~~~~~~~~~~~~~~~~~ + +After every insert, update, or delete operation, MongoDB must update +*every* index associated with the collection in addition to the data +itself. Therefore, every index on a collection adds some amount of +overhead for the performance of write operations. [#exceptions]_ + +In general, the performance gains that indexes provide for *read +operations* are worth the insertion penalty; however, when optimizing +write performance, be careful when creating new indexes and always +evaluate the indexes on the collection and ensure that your queries are +actually using these indexes. + +For more information on indexes in MongoDB consider :doc:`/indexes` +and :doc:`/applications/indexes`. + +.. [#exceptions] The overhead for :ref:`sparse indexes ` + inserts and updates to un-indexed fields is less than for non-sparse + indexes. Also for non-sparse indexes, updates that don't change the + record size have less indexing overhead. + +Certain operations and architectures have their own set of +considerations that are important to be aware of. The following +documents detail these considerations: + +.. toctree:: + :maxdepth: 1 + + /core/write-operations-bulk-inserts + /core/write-operations-updates-padding + /core/write-operations-architecture + .. index:: write operators .. _write-operations-operators: Write Operators --------------- +.. AM: should this be here? Is this conceptual or just linking out to the relevant example pages? + For information on write operators and how to write data to a MongoDB database, see the following pages: @@ -60,274 +120,3 @@ Write Concern .. seealso:: :doc:`/core/write-concern` and :doc:`/reference/write-concern`. - -.. _write-operations-bulk-insert: - -Bulk Inserts ------------- - -In some situations you may need to insert or ingest a large amount of -data into a MongoDB database. These *bulk inserts* have some -special considerations that are different from other write -operations. - -The :method:`insert() ` method, when passed an -array of documents, will perform a bulk insert, and inserts each -document atomically. :doc:`Drivers ` -provide their own interface for this kind of operation. - -.. versionadded:: 2.2 - :method:`insert() ` in the :program:`mongo` - shell gained support for bulk inserts in version 2.2. - -Bulk insert can significantly increase performance by amortizing -:ref:`write concern ` costs. In the -drivers, you can configure write concern for batches rather than on a -per-document level. - -Drivers also have a ``ContinueOnError`` option in their insert -operation, so that the bulk operation will continue to insert -remaining documents in a batch even if an insert fails. - -.. note:: - - .. versionadded:: 2.0 - Support for ``ContinueOnError`` depends on version 2.0 of the - core :program:`mongod` and :program:`mongos` components. - -If the bulk insert process generates more than one error in a batch -job, the client will only receive the most recent error. All bulk -operations to a :term:`sharded collection ` run with -``ContinueOnError``, which applications cannot disable. See -:ref:`sharding-bulk-inserts` section for more information on -consideration for bulk inserts in sharded clusters. - -For more information see your :doc:`driver documentation -` for details on performing bulk inserts in -your application. Also consider the following resources: -:ref:`write-operations-sharded-clusters`, -:ref:`sharding-bulk-inserts`, and -:doc:`/core/import-export`. - -.. _write-operations-indexing: - -Indexing --------- - -After every insert, update, or delete operation, MongoDB must update -*every* index associated with the collection in addition to the data -itself. Therefore, every index on a collection adds some amount of -overhead for the performance of write operations. [#exceptions]_ - -In general, the performance gains that indexes provide for *read -operations* are worth the insertion penalty; however, when optimizing -write performance, be careful when creating new indexes and always -evaluate the indexes on the collection and ensure that your queries are -actually using these indexes. - -For more information on indexes in MongoDB consider :doc:`/indexes` -and :doc:`/applications/indexes`. - -.. [#exceptions] The overhead for :ref:`sparse indexes - ` inserts and updates to un-indexed fields - is less than for non-sparse indexes. Also for non-sparse indexes, - updates that don't change the record size have less indexing - overhead. - -.. _write-operations-isolation: - -Isolation ---------- - -When a single write operation modifies multiple documents, the -operation as a whole is not atomic, and other operations may -interleave. The modification of a single document, or record, is always -atomic, even if the write operation modifies multiple sub-document -*within* the single record. - -No other operations are atomic; however, you can attempt to isolate a -write operation that affects multiple documents using the -:doc:`isolation operator `. - -To isolate a sequence of write operations from other read and write -operations, see :doc:`/tutorial/perform-two-phase-commits`. - -Updates -------- - -Each document in a MongoDB collection has allocated *record* space -which includes the entire document *and* a small amount of -padding. This padding makes it possible for update operations to -increase the size of a document slightly without causing the document -to outgrow the allocated record size. - -Documents in MongoDB can grow up to the full maximum :limit:`BSON -document size `. However, when documents outgrow -their allocated record size MongoDB must allocate a new record and -move the document to the new record. Update operations that do not -cause a document to grow, (i.e. *in-place* updates,) are significantly -more efficient than those updates that cause document growth. Use -:doc:`data models ` that minimize the need for -document growth when possible. - -For complete examples of update operations, see -:doc:`/core/update`. - -.. _write-operations-padding-factor: - -Padding Factor --------------- - -If an update operation does not cause the document to increase in -size, MongoDB can apply the update in-place. Some updates -change the size of the document, for example using the -:operator:`$push` operator to append a sub-document to an array can -cause the top level document to grow beyond its allocated space. - -When documents grow, MongoDB relocates the document on disk with -enough contiguous space to hold the document. These relocations -take longer than in-place updates, particularly if the collection has -indexes that MongoDB must update all index entries. If -collection has many indexes, the move will impact write throughput. - -To minimize document movements, MongoDB employs padding. MongoDB -adaptively learns if documents in a collection tend to grow, and if -they do, adds a :data:`~collStats.paddingFactor` so that the documents -have room to grow on subsequent writes. The -:data:`~collStats.paddingFactor` indicates the padding for new inserts -and moves. - -.. versionadded:: 2.2 - You can use the :dbcommand:`collMod` command - with the :collflag:`usePowerOf2Sizes` flag so that MongoDB - allocates document space in sizes that are powers of 2. This helps - ensure that MongoDB can efficiently reuse the space freed as a - result of deletions or document relocations. As with all padding, - using document space allocations with power of 2 sizes minimizes, - but does not eliminate, document movements. - -To check the current :data:`~collStats.paddingFactor` on a collection, you can -run the :method:`db.collection.stats()` operation in the -:program:`mongo` shell, as in the following example: - -.. code-block:: javascript - - db.myCollection.stats() - -Since MongoDB writes each document at a different point in time, the -padding for each document will not be the same. You can calculate the -padding size by subtracting 1 from the :data:`~collStats.paddingFactor`, for -example: - -.. code-block:: none - - padding size = (paddingFactor - 1) * . - -For example, a :data:`~collStats.paddingFactor` of ``1.0`` specifies no padding -whereas a paddingFactor of ``1.5`` specifies a padding size of ``0.5`` or 50 -percent (50%) of the document size. - -Because the :data:`~collStats.paddingFactor` is relative to the size of each -document, you cannot calculate the exact amount of padding for a -collection based on the average document size and padding factor. - -If an update operation causes the document to *decrease* in size, for -instance if you perform an :operator:`$unset` or a :operator:`$pop` -update, the document remains in place and effectively has more -padding. If the document remains this size, the space is not reclaimed -until you perform a :dbcommand:`compact` or a -:dbcommand:`repairDatabase` operation. - -.. note:: - - The following operations remove padding: - - - :dbcommand:`compact`, - - :dbcommand:`repairDatabase`, and - - initial replica sync operations. - - However, with the :dbcommand:`compact` command, you can run the - command with a ``paddingFactor`` or a ``paddingBytes`` parameter. - - Padding is also removed if you use :program:`mongoexport` from a - collection. If you use :program:`mongoimport` into a new - collection, :program:`mongoimport` will not add padding. If you - use :program:`mongoimport` with an existing collection with - padding, :program:`mongoimport` will not affect the existing - padding. - - When a database operation removes padding, subsequent update that - require changes in record sizes will have reduced throughput until - the collection's padding factor grows. Padding does not affect - in-place, and after :dbcommand:`compact`, - :dbcommand:`repairDatabase`, and replica set initial sync - the collection will require less storage. - -.. COMMENT -- not sure if we really need this manual padding info or if - it belongs here, but ... - -.. seealso:: - - - :ref:`faq-developers-manual-padding` - - - The :operator:`$inc` for in-place updates. - -Architecture ------------- - -.. _write-operations-replica-sets: - -Replica Sets -~~~~~~~~~~~~ - -In :term:`replica sets `, all write operations go to the -set's :term:`primary`, which applies the write operation then records -the operations on the primary's operation log or :term:`oplog`. The -oplog is a reproducible sequence of operations to the data -set. :term:`Secondary` members of the set are continuously replicating the -oplog and applying the operations to themselves in an asynchronous -process. - -Large volumes of write operations, particularly bulk operations, may -create situations where the secondary members have difficulty applying -the replicating operations from the primary at a sufficient rate: this -can cause the secondary's state to fall behind that of the primary. Secondaries -that are significantly behind the primary present problems for normal -operation of the replica set, particularly :ref:`failover -` in the form of :ref:`rollbacks -` as well as general :doc:`read consistency -`. - -To help avoid this issue, you can customize the :ref:`write concern -` to return confirmation of the write -operation to another member [#write-concern-throttling]_ of the replica -set every 100 or 1,000 operations. This provides an opportunity for -secondaries to catch up with the primary. Write concern can slow the -overall progress of write operations but ensure that the secondaries -can maintain a largely current state with respect to the primary. - -For more information on replica sets and write operations, see -:ref:`replica-set-write-concern`, :ref:`replica-set-oplog-sizing`, and -:doc:`/tutorial/change-oplog-size`. - -.. [#write-concern-throttling] Calling :dbcommand:`getLastError` - intermittently with a ``w`` value of ``2`` or ``majority`` will - slow the throughput of write traffic; however, this practice will - allow the secondaries to remain current with the state of the - primary. - -.. _write-operations-sharded-clusters: - -Sharded Clusters -~~~~~~~~~~~~~~~~ - -In a :term:`sharded cluster`, MongoDB directs a given write operation to -a :term:`shard` and then performs the write on a particular -:term:`chunk` on that shard. Shards and chunks are range-based. -:term:`Shard keys ` affect how MongoDB distributes documents -among shards. Choosing the correct shard key can have a great impact on -the performance, capability, and functioning of your database and -cluster. - -For more information, see :doc:`/administration/sharded-clusters` and -:ref:`write-operations-bulk-insert`. From 21e14b9a14ef85bb1d2d0ae333db63cb08ee90bb Mon Sep 17 00:00:00 2001 From: schmalliso Date: Wed, 10 Jul 2013 14:03:46 -0400 Subject: [PATCH 04/14] crud-reorg: breaking up write concern --- source/core/replica-set-write-concern.txt | 198 ++++++++++++++++++ source/core/write-concern.txt | 198 +----------------- .../toc-replica-set-read-write-semantics.yaml | 2 +- 3 files changed, 204 insertions(+), 194 deletions(-) create mode 100644 source/core/replica-set-write-concern.txt diff --git a/source/core/replica-set-write-concern.txt b/source/core/replica-set-write-concern.txt new file mode 100644 index 00000000000..5c73e05e636 --- /dev/null +++ b/source/core/replica-set-write-concern.txt @@ -0,0 +1,198 @@ +============================== +Write Concern for Replica Sets +============================== + +.. default-domain:: mongodb + +.. TODO: connect this to the actual write concern documentation and add an intro + +MongoDB's built-in :term:`write concern` confirms the success of write +operations to a :term:`replica set's ` :term:`primary`. +Write concern uses the :dbcommand:`getLastError` command after write +operations to return an object with error information or confirmation +that there are no errors. + +From the perspective of a client application, whether a MongoDB +instance is running as a single server (i.e. "standalone") or a +:term:`replica set` is transparent. However, replica sets offer some +configuration options for write and read operations. [#sharded-clusters]_ + +.. [#sharded-clusters] :term:`Sharded clusters ` where the + shards are also replica sets provide the same configuration options + with regards to write and read operations. + +.. TODO: remove footnote + +Verify Write Operations +~~~~~~~~~~~~~~~~~~~~~~~ + +The default write concern confirms write operations only on the +primary. You can configure write concern to confirm write operations +to additional replica set members as well by issuing the +:dbcommand:`getLastError` command with the ``w`` option. + +.. TODO: provide an example of how to do this (i.e. write the code block) + +The ``w`` option confirms that write operations have replicated to the +specified number of replica set members, including the primary. You can +either specify a number or specify ``majority``, which ensures the write +propagates to a majority of set members. + +If you specify a ``w`` value greater than the number of members that +hold a copy of the data (i.e., greater than the number of +non-:term:`arbiter` members), the operation blocks until those members +become available. This can cause the operation to block forever. To +specify a timeout threshold for the :dbcommand:`getLastError` operation, +use the ``wtimeout`` argument. + +See :ref:`getLastError Examples ` for example +invocations. + +Modify Default Write Concern +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can configure your own "default" :dbcommand:`getLastError` +behavior for a replica set. Use the +:data:`~local.system.replset.settings.getLastErrorDefaults` setting in +the :doc:`replica set configuration +`. The following sequence of +commands creates a configuration that waits for the write operation to +complete on a majority of the set members before returning: + +.. code-block:: javascript + + cfg = rs.conf() + cfg.settings = {} + cfg.settings.getLastErrorDefaults = {w: "majority"} + rs.reconfig(cfg) + +The :data:`~local.system.replset.settings.getLastErrorDefaults` +setting affects only those :dbcommand:`getLastError` commands that +have *no* other arguments. + +.. note:: + + Use of insufficient write concern can lead to :ref:`rollbacks + ` in the case of :ref:`replica set failover + `. Always ensure that your operations have + specified the required write concern for your application. + +.. seealso:: :ref:`write-operations-write-concern` and + :ref:`connections-write-concern` + +Custom Write Concerns +~~~~~~~~~~~~~~~~~~~~~ + +You can use replica set tags to create custom write concerns using the +:data:`~local.system.replset.settings.getLastErrorDefaults` and +:data:`~local.system.replset.settings.getLastErrorModes` replica set +settings. + +.. note:: + + Custom write concern modes specify the field name and a number of + *distinct* values for that field. By contrast, read preferences use + the value of fields in the tag document to direct read operations. + + In some cases, you may be able to use the same tags for read + preferences and write concerns; however, you may need to create + additional tags for write concerns depending on the requirements of + your application. + +Single Tag Write Concerns +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Consider a five member replica set, where each member has one of the +following tag sets: + +.. code-block:: javascript + + { "use": "reporting" } + { "use": "backup" } + { "use": "application" } + { "use": "application" } + { "use": "application" } + +You could create a custom write concern mode that will ensure that +applicable write operations will not return until members with two +different values of the ``use`` tag have acknowledged the write +operation. Create the mode with the following sequence of operations +in the :program:`mongo` shell: + +.. code-block:: javascript + + cfg = rs.conf() + cfg.settings = { getLastErrorModes: { use2: { "use": 2 } } } + rs.reconfig(cfg) + +.. these examples need to be better so that they avoid overwriting + getLastErrorModes upon repetition (i.e. they don't $push documents + to getLastErrorModes.) + +To use this mode pass the string ``multiUse`` to the ``w`` option of +:dbcommand:`getLastError` as follows: + +.. code-block:: javascript + + db.runCommand( { getLastError: 1, w: "use2" } ) + +Specific Custom Write Concerns +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you have a three member replica with the following tag sets: + +.. code-block:: javascript + + { "disk": "ssd" } + { "disk": "san" } + { "disk": "spinning" } + +You cannot specify a custom +:data:`~local.system.replset.settings.getLastErrorModes` value to +ensure that the write propagates to the ``san`` before +returning. However, you may implement this write concern policy by +creating the following additional tags, so that the set resembles the +following: + +.. code-block:: javascript + + { "disk": "ssd" } + { "disk": "san", "disk.san": "san" } + { "disk": "spinning" } + +Then, create a custom +:data:`~local.system.replset.settings.getLastErrorModes` value, as +follows: + +.. code-block:: javascript + + cfg = rs.conf() + cfg.settings = { getLastErrorModes: { san: { "disk.san": 1 } } } + rs.reconfig(cfg) + +.. these examples need to be better so that they avoid overwriting + getLastErrorModes upon repetition (i.e. they don't $push documents + to getLastErrorModes.) + +To use this mode pass the string ``san`` to the ``w`` option of +:dbcommand:`getLastError` as follows: + +.. code-block:: javascript + + db.runCommand( { getLastError: 1, w: "san" } ) + +This operation will not return until a replica set member with the tag +``disk.san`` returns. + +You may set a custom write concern mode as the default write concern +mode using :data:`~local.system.replset.settings.getLastErrorDefaults` +replica set as in the following setting: + +.. code-block:: javascript + + cfg = rs.conf() + cfg.settings.getLastErrorDefaults = { ssd: 1 } + rs.reconfig(cfg) + +.. seealso:: :ref:`replica-set-configuration-tag-sets` for further + information about replica set reconfiguration and tag sets. \ No newline at end of file diff --git a/source/core/write-concern.txt b/source/core/write-concern.txt index cbb3e0c1b69..07fc67218b2 100644 --- a/source/core/write-concern.txt +++ b/source/core/write-concern.txt @@ -12,10 +12,11 @@ Write Concern .. seealso:: :doc:`/reference/write-concern` for a reference of specific write concern configuration. Also consider :doc:`/core/write-operations` for a general overview of write - operations with MongoDB. + operations with MongoDB and :doc:`/core/replica-set-write-concern` + for considerations specific to replica sets. -.. note:: After the :doc:`driver write concern change - ` all officially supported +.. note:: After the :doc:`driver write concern change ` + all officially supported MongoDB drivers enable write concern by default. Types of Write Concern @@ -109,7 +110,7 @@ write operation on only one :program:`mongod` instance. The ``w`` argument to :dbcommand:`getLastError` provides a *replica acknowledged* level of write concern. With *replica acknowledged* you can guarantee that the write operation has propagated to the members -of a replica set. See the :ref:`Write Concern for Replica Sets +of a replica set. See the :doc:`Write Concern for Replica Sets ` document for more information. .. note:: @@ -119,192 +120,3 @@ of a replica set. See the :ref:`Write Concern for Replica Sets set regardless of the level of *replica acknowledged* write concern. -Write Concern for Replica Sets ------------------------------- - -MongoDB's built-in :term:`write concern` confirms the success of write -operations to a :term:`replica set's ` :term:`primary`. -Write concern uses the :dbcommand:`getLastError` command after write -operations to return an object with error information or confirmation -that there are no errors. - -From the perspective of a client application, whether a MongoDB -instance is running as a single server (i.e. "standalone") or a -:term:`replica set` is transparent. However, replica sets offer some -configuration options for write and read operations. [#sharded-clusters]_ - -.. [#sharded-clusters] :term:`Sharded clusters ` where the - shards are also replica sets provide the same configuration options - with regards to write and read operations. - -Verify Write Operations -~~~~~~~~~~~~~~~~~~~~~~~ - -The default write concern confirms write operations only on the -primary. You can configure write concern to confirm write operations -to additional replica set members as well by issuing the -:dbcommand:`getLastError` command with the ``w`` option. - -The ``w`` option confirms that write operations have replicated to the -specified number of replica set members, including the primary. You can -either specify a number or specify ``majority``, which ensures the write -propagates to a majority of set members. - -If you specify a ``w`` value greater than the number of members that -hold a copy of the data (i.e., greater than the number of -non-:term:`arbiter` members), the operation blocks until those members -become available. This can cause the operation to block forever. To -specify a timeout threshold for the :dbcommand:`getLastError` operation, -use the ``wtimeout`` argument. - -See :ref:`getLastError Examples ` for example -invocations. - -Modify Default Write Concern -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -You can configure your own "default" :dbcommand:`getLastError` -behavior for a replica set. Use the -:data:`~local.system.replset.settings.getLastErrorDefaults` setting in -the :doc:`replica set configuration -`. The following sequence of -commands creates a configuration that waits for the write operation to -complete on a majority of the set members before returning: - -.. code-block:: javascript - - cfg = rs.conf() - cfg.settings = {} - cfg.settings.getLastErrorDefaults = {w: "majority"} - rs.reconfig(cfg) - -The :data:`~local.system.replset.settings.getLastErrorDefaults` -setting affects only those :dbcommand:`getLastError` commands that -have *no* other arguments. - -.. note:: - - Use of insufficient write concern can lead to :ref:`rollbacks - ` in the case of :ref:`replica set failover - `. Always ensure that your operations have - specified the required write concern for your application. - -.. seealso:: :ref:`write-operations-write-concern` and - :ref:`connections-write-concern` - -Custom Write Concerns -~~~~~~~~~~~~~~~~~~~~~ - -You can use replica set tags to create custom write concerns using the -:data:`~local.system.replset.settings.getLastErrorDefaults` and -:data:`~local.system.replset.settings.getLastErrorModes` replica set -settings. - -.. note:: - - Custom write concern modes specify the field name and a number of - *distinct* values for that field. By contrast, read preferences use - the value of fields in the tag document to direct read operations. - - In some cases, you may be able to use the same tags for read - preferences and write concerns; however, you may need to create - additional tags for write concerns depending on the requirements of - your application. - -Single Tag Write Concerns -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Consider a five member replica set, where each member has one of the -following tag sets: - -.. code-block:: javascript - - { "use": "reporting" } - { "use": "backup" } - { "use": "application" } - { "use": "application" } - { "use": "application" } - -You could create a custom write concern mode that will ensure that -applicable write operations will not return until members with two -different values of the ``use`` tag have acknowledged the write -operation. Create the mode with the following sequence of operations -in the :program:`mongo` shell: - -.. code-block:: javascript - - cfg = rs.conf() - cfg.settings = { getLastErrorModes: { use2: { "use": 2 } } } - rs.reconfig(cfg) - -.. these examples need to be better so that they avoid overwriting - getLastErrorModes upon repetition (i.e. they don't $push documents - to getLastErrorModes.) - -To use this mode pass the string ``multiUse`` to the ``w`` option of -:dbcommand:`getLastError` as follows: - -.. code-block:: javascript - - db.runCommand( { getLastError: 1, w: "use2" } ) - -Specific Custom Write Concerns -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you have a three member replica with the following tag sets: - -.. code-block:: javascript - - { "disk": "ssd" } - { "disk": "san" } - { "disk": "spinning" } - -You cannot specify a custom -:data:`~local.system.replset.settings.getLastErrorModes` value to -ensure that the write propagates to the ``san`` before -returning. However, you may implement this write concern policy by -creating the following additional tags, so that the set resembles the -following: - -.. code-block:: javascript - - { "disk": "ssd" } - { "disk": "san", "disk.san": "san" } - { "disk": "spinning" } - -Then, create a custom -:data:`~local.system.replset.settings.getLastErrorModes` value, as -follows: - -.. code-block:: javascript - - cfg = rs.conf() - cfg.settings = { getLastErrorModes: { san: { "disk.san": 1 } } } - rs.reconfig(cfg) - -.. these examples need to be better so that they avoid overwriting - getLastErrorModes upon repetition (i.e. they don't $push documents - to getLastErrorModes.) - -To use this mode pass the string ``san`` to the ``w`` option of -:dbcommand:`getLastError` as follows: - -.. code-block:: javascript - - db.runCommand( { getLastError: 1, w: "san" } ) - -This operation will not return until a replica set member with the tag -``disk.san`` returns. - -You may set a custom write concern mode as the default write concern -mode using :data:`~local.system.replset.settings.getLastErrorDefaults` -replica set as in the following setting: - -.. code-block:: javascript - - cfg = rs.conf() - cfg.settings.getLastErrorDefaults = { ssd: 1 } - rs.reconfig(cfg) - -.. seealso:: :ref:`replica-set-configuration-tag-sets` for further - information about replica set reconfiguration and tag sets. diff --git a/source/includes/toc-replica-set-read-write-semantics.yaml b/source/includes/toc-replica-set-read-write-semantics.yaml index 17f554e5d2c..227dadb1146 100644 --- a/source/includes/toc-replica-set-read-write-semantics.yaml +++ b/source/includes/toc-replica-set-read-write-semantics.yaml @@ -1,4 +1,4 @@ -file: /core/write-concern +file: /core/replica-set-write-concern description: | Write concern is the guarantee an application requires from MongoDB to consider a write operation successful. From c662548319a2e2c67cd13384cfda9c836dcc6a7f Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Wed, 10 Jul 2013 16:04:56 -0400 Subject: [PATCH 05/14] crud reorg: bson --- source/core/bson-types.txt | 187 ++++++ .../document-type-index-specification.txt | 45 ++ source/core/document-type-operators.txt | 36 ++ .../document-type-query-specification.txt | 40 ++ source/core/document-type-record.txt | 95 +++ source/core/document-type-sort-order.txt | 40 ++ .../document-type-update-specification.txt | 64 ++ source/core/document.txt | 591 +++--------------- source/includes/fact-dot-notation.rst | 4 +- source/reference/glossary.txt | 29 +- 10 files changed, 585 insertions(+), 546 deletions(-) create mode 100644 source/core/bson-types.txt create mode 100644 source/core/document-type-index-specification.txt create mode 100644 source/core/document-type-operators.txt create mode 100644 source/core/document-type-query-specification.txt create mode 100644 source/core/document-type-record.txt create mode 100644 source/core/document-type-sort-order.txt create mode 100644 source/core/document-type-update-specification.txt diff --git a/source/core/bson-types.txt b/source/core/bson-types.txt new file mode 100644 index 00000000000..10a3484060f --- /dev/null +++ b/source/core/bson-types.txt @@ -0,0 +1,187 @@ +========== +BSON Types +========== + +.. default-domain:: mongodb + +.. contents:: + :backlinks: none + :local: + +:term:`BSON` is a binary serialization format used to store documents +and make remote procedure calls in MongoDB. The BSON specification is +located at `bsonspec.org `_. + +BSON supports the following data types as values in documents. Each data +type has a corresponding number that can be used with the +:operator:`$type` operator to query documents by BSON type. + +======================= ========== +**Type** **Number** +----------------------- ---------- +Double 1 +String 2 +Object 3 +Array 4 +Binary data 5 +Object id 7 +Boolean 8 +Date 9 +Null 10 +Regular Expression 11 +JavaScript 13 +Symbol 14 +JavaScript (with scope) 15 +32-bit integer 16 +Timestamp 17 +64-bit integer 18 +Min key 255 +Max key 127 +======================= ========== + +To determine a field's type, see :doc:`/core/document-type-operators`. + +If you convert BSON to JSON, see +:ref:`bson-json-type-conversion-fidelity` for more information. + +The next sections describe special considerations for particular BSON +types. + +.. _document-bson-type-object-id: + +ObjectId +-------- + +ObjectIds are: small, likely unique, fast to generate, and ordered. +These values consists of 12-bytes, where the first four bytes are a +timestamp that reflect the ObjectId's creation. Refer to the +:doc:`ObjectId ` documentation for more +information. + +.. _document-bson-type-string: + +String +------ + +BSON strings are UTF-8. In general, drivers for each programming +language convert from the language's string format to UTF-8 when +serializing and deserializing BSON. This makes it possible to store +most international characters in BSON strings with ease. +[#sort-string-internationalization]_ In addition, MongoDB +:operator:`$regex` queries support UTF-8 in the regex string. + +.. [#sort-string-internationalization] Given strings using UTF-8 + character sets, using :method:`sort() ` on strings + will be reasonably correct. However, because internally + :method:`sort() ` uses the C++ ``strcmp`` api, the + sort order may handle some characters incorrectly. + +.. _document-bson-type-timestamp: + +Timestamps +---------- + +BSON has a special timestamp type for *internal* MongoDB use and is +**not** associated with the regular :ref:`document-bson-type-date` +type. Timestamp values are a 64 bit value where: + +- the first 32 bits are a ``time_t`` value (seconds since the Unix epoch) + +- the second 32 bits are an incrementing ``ordinal`` for operations + within a given second. + +Within a single :program:`mongod` instance, timestamp values are +always unique. + +In replication, the :term:`oplog` has a ``ts`` field. The values in +this field reflect the operation time, which uses a BSON timestamp +value. + +.. note:: + + The BSON Timestamp type is for *internal* MongoDB use. For most + cases, in application development, you will want to use the BSON + date type. See :ref:`document-bson-type-date` for more + information. + +If you create a BSON Timestamp using the empty constructor (e.g. ``new +Timestamp()``), MongoDB will only generate a timestamp *if* you use +the constructor in the first field of the document. [#id_exception]_ +Otherwise, MongoDB will generate an empty timestamp value +(i.e. ``Timestamp(0, 0)``.) + +.. versionchanged:: 2.1 + :program:`mongo` shell displays the Timestamp value with the wrapper: + + .. code-block:: javascript + + Timestamp(, ) + + Prior to version 2.1, the :program:`mongo` shell display the + Timestamp value as a document: + + .. code-block:: javascript + + { t : , i : } + +.. [#id_exception] If the first field in the document is ``_id``, then + you can generate a timestamp in the *second* field + of a document. + + .. only:: html or dirhtml or singlehtml or epub or gettext + + In the following example, MongoDB will generate a Timestamp + value, even though the ``Timestamp()`` constructor is *not* in + the first field in the document: + + .. code-block:: javascript + + db.bios.insert( { _id: 9, last_updated: new Timestamp() } ) + +.. _document-bson-type-date: + +Date +---- + +BSON Date is a 64-bit integer that represents the number of +milliseconds since the Unix epoch (Jan 1, 1970). The `official BSON +specification `_ refers to the +BSON Date type as the *UTC datetime*. + +.. versionchanged:: 2.0 + BSON Date type is signed. [#unsigned-date]_ Negative values + represent dates before 1970. + +.. example:: Construct a Date using the ``new Date()`` constructor in the + :program:`mongo` shell: + + .. code-block:: javascript + + var mydate1 = new Date() + +.. example:: Construct a Date using the ``ISODate()`` constructor in the + :program:`mongo` shell: + + .. code-block:: javascript + + var mydate2 = ISODate() + +.. example:: Return the ``Date`` value as string: + + .. code-block:: javascript + + mydate1.toString() + +.. example:: Return the month portion of the Date value; months are zero-indexed, + so that January is month ``0``: + + .. code-block:: javascript + + mydate1.getMonth() + +.. [#unsigned-date] Prior to version 2.0, ``Date`` values were + incorrectly interpreted as *unsigned* integers, which affected + sorts, range queries, and indexes on ``Date`` fields. Because + indexes are not recreated when upgrading, please re-index if you + created an index on ``Date`` values with an earlier version, and + dates before 1970 are relevant to your application. diff --git a/source/core/document-type-index-specification.txt b/source/core/document-type-index-specification.txt new file mode 100644 index 00000000000..838d4ccea20 --- /dev/null +++ b/source/core/document-type-index-specification.txt @@ -0,0 +1,45 @@ +.. _documents-index: +.. _document-index-specification: + +============================= +Index Specification Documents +============================= + +.. default-domain:: mongodb + +Index specification documents describe the fields to index on during +the :doc:`index creation +`. See :doc:`indexes +` for an overview of indexes. [#index-def]_ + +Index documents contain field and value pairs, in the following form: + +.. code-block:: javascript + + { field: value } + +- ``field`` is the field in the documents to index. + +- ``value`` is either 1 for ascending or -1 for descending. + +The following document specifies the :ref:`multi-key index +` on the ``_id`` field and the ``last`` field +contained in the subdocument ``name`` field. The document uses +:ref:`dot notation ` to access a field in a +subdocument: + +.. code-block:: javascript + + { _id: 1, 'name.last': 1 } + +When passed as an argument to the :method:`ensureIndex() +` method, the index documents specifies +the index to create: + +.. code-block:: javascript + + db.bios.ensureIndex( { _id: 1, 'name.last': 1 } ) + +.. [#index-def] Indexes optimize a number of key :doc:`read + ` and :doc:`write ` + operations. diff --git a/source/core/document-type-operators.txt b/source/core/document-type-operators.txt new file mode 100644 index 00000000000..c4d95df6008 --- /dev/null +++ b/source/core/document-type-operators.txt @@ -0,0 +1,36 @@ +==================== +Determine Field Type +==================== + +.. default-domain:: mongodb + +To determine the type of fields, the :program:`mongo` shell provides +the following operators: + +- ``instanceof`` returns a boolean to test if a value has + a specific type. + +- ``typeof`` returns the type of a field. + +.. example:: + + Consider the following operations using ``instanceof`` and + ``typeof``: + + - The following operation tests whether the ``_id`` field is of type + ``ObjectId``: + + .. code-block:: javascript + + mydoc._id instanceof ObjectId + + The operation returns ``true``. + + - The following operation returns the type of the ``_id`` field: + + .. code-block:: javascript + + typeof mydoc._id + + In this case ``typeof`` will return the more generic ``object`` + type rather than ``ObjectId`` type. diff --git a/source/core/document-type-query-specification.txt b/source/core/document-type-query-specification.txt new file mode 100644 index 00000000000..524a699586f --- /dev/null +++ b/source/core/document-type-query-specification.txt @@ -0,0 +1,40 @@ +.. _documents-query-selectors: +.. _mongodb-query-document: + +============================= +Query Specification Documents +============================= + +.. default-domain:: mongodb + +Query documents specify the conditions that determine which records to +select for read, update, and delete operations. You can use +``:`` expressions to specify the equality condition and +:doc:`query operator ` expressions to specify +additional conditions. + +When passed as an argument to methods such as the :method:`find() +` method, the :method:`remove() +` method, or the :method:`update() +` method, the query document selects documents +for MongoDB to return, remove, or update, as in the following: + +.. code-block:: javascript + + db.bios.find( { _id: 1 } ) + db.bios.remove( { _id: { $gt: 3 } } ) + db.bios.update( { _id: 1, name: { first: 'John', last: 'Backus' } }, + , + ) + +.. seealso:: + + - :ref:`read-operations-query-argument` and + :doc:`/core/read` for more examples on selecting documents + for reads. + + - :doc:`/core/update` for more examples on + selecting documents for updates. + + - :doc:`/core/delete` for more examples on selecting + documents for deletes. diff --git a/source/core/document-type-record.txt b/source/core/document-type-record.txt new file mode 100644 index 00000000000..100234fffc5 --- /dev/null +++ b/source/core/document-type-record.txt @@ -0,0 +1,95 @@ +.. _documents-records: + +================ +Record Documents +================ + +.. default-domain:: mongodb + +Record documents store the database's data. Each record document is +equivalent to a row in an :term:`RDBMS`. Record documents are grouped +into :term:`collections `, which are the equivalent of RDBMS +tables. + +The following is an example record document: + + .. code-block:: javascript + + { + _id: 1, + name: { first: 'John', last: 'Backus' }, + birth: new Date('Dec 03, 1924'), + death: new Date('Mar 17, 2007'), + contribs: [ 'Fortran', 'ALGOL', 'Backus-Naur Form', 'FP' ], + awards: [ + { award: 'National Medal of Science', + year: 1975, + by: 'National Science Foundation' }, + { award: 'Turing Award', + year: 1977, + by: 'ACM' } + ] + } + +The document contains the following fields: + +- ``_id``, which must hold a unique value and is *immutable*. + +- ``name`` that holds another *document*. This sub-document contains + the fields ``first`` and ``last``, which both hold *strings*. + +- ``birth`` and ``death`` that both have *date* types. + +- ``contribs`` that holds an *array of strings*. + +- ``awards`` that holds an *array of documents*. + +Attributes of Record Documents +------------------------------ + +Record documents have the following attributes: + +- .. include:: /includes/fact-document-max-size.rst + +- .. include:: /includes/fact-document-field-name-restrictions.rst + +.. include:: /includes/note-insert-id-field.rst + +The ``_id`` Field +----------------- + +The ``_id`` field has the following behavior and constraints: + +- In documents, the ``_id`` field is always indexed for regular + collections. + +- The ``_id`` field may contain values of any :doc:`BSON data type + `, other than an array. + + .. warning:: To ensure functioning replication, do not store values + that are of the BSON regular expression type in the ``_id`` + field. + + .. See :issue:`SERVER-9562` for more information. + +The following are common options for storing values for ``_id``: + +- Use an :doc:`ObjectId `. + +- Use a natural unique identifier, if available. This saves space and + avoids an additional index. + +- Generate an auto-incrementing number. See + :doc:`/tutorial/create-an-auto-incrementing-field`. + +- Generate a UUID in your application code. For a more efficient + storage of the UUID values in the collection and in the ``_id`` + index, store the UUID as a value of the BSON ``BinData`` type. + + .. include:: /includes/fact-bindata-storage-optimization.rst + +- Use your driver's BSON UUID facility to generate UUIDs. Be aware + that driver implementations may implement UUID serialization and + deserialization logic differently, which may not be fully compatible + with other drivers. See your :api:`driver documentation <>` for + information concerning UUID interoperability. diff --git a/source/core/document-type-sort-order.txt b/source/core/document-type-sort-order.txt new file mode 100644 index 00000000000..d5eee90f6f1 --- /dev/null +++ b/source/core/document-type-sort-order.txt @@ -0,0 +1,40 @@ +.. _documents-sort-order: + +================================== +Sort Order Specification Documents +================================== + +.. default-domain:: mongodb + +Sort order documents specify the order of documents that a +:method:`query() ` returns. Pass sort order +specification documents as an argument to the :method:`sort() +` method. See the :method:`sort() ` page +for more information on sorting. + +The sort order documents contain field and value pairs, in the following +form: + +.. code-block:: javascript + + { field: value } + +- ``field`` is the field by which to sort documents. + +- ``value`` is either 1 for ascending or -1 for descending. + +The following document specifies the sort order using the fields from a +sub-document ``name`` first sort by the ``last`` field ascending, then +by the ``first`` field also ascending: + +.. code-block:: javascript + + { 'name.last': 1, 'name.first': 1 } + +When passed as an argument to the :method:`sort() ` +method, the sort order document sorts the results of the +:method:`find() ` method: + +.. code-block:: javascript + + db.bios.find().sort( { 'name.last': 1, 'name.first': 1 } ) diff --git a/source/core/document-type-update-specification.txt b/source/core/document-type-update-specification.txt new file mode 100644 index 00000000000..faf5cb266a2 --- /dev/null +++ b/source/core/document-type-update-specification.txt @@ -0,0 +1,64 @@ +.. _documents-update-actions: + +============================== +Update Specification Documents +============================== + +.. default-domain:: mongodb + +Update documents specify the data modifications to perform during +an :method:`update() ` operation to modify +existing records in a collection. You can use :ref:`update operators +` to specify the exact actions to perform on the +document fields. + +The following is an example update document: + +.. code-block:: javascript + + { + $set: { 'name.middle': 'Warner' }, + $push: { awards: { award: 'IBM Fellow', + year: '1963', + by: 'IBM' } + } + } + +When passed as an argument to the :method:`update() +` method, the update actions document: + +- Modifies the field ``name`` whose value is another document. + Specifically, the :operator:`$set` operator updates the ``middle`` + field in the ``name`` subdocument. The document uses :ref:`dot + notation ` to access a field in a subdocument. + +- Adds an element to the field ``awards``, whose value is an array. + Specifically, the :operator:`$push` operator adds another document as + element to the field ``awards``. + +.. code-block:: javascript + + db.bios.update( + { _id: 1 }, + { + $set: { 'name.middle': 'Warner' }, + $push: { awards: { + award: 'IBM Fellow', + year: '1963', + by: 'IBM' + } + } + } + ) + +.. seealso:: + + - :ref:`update operators ` page for the available + update operators and syntax. + + - :doc:`update ` for more examples on + update documents. + +For additional examples of updates that involve array elements, +including where the elements are documents, see the :operator:`$` +positional operator. diff --git a/source/core/document.txt b/source/core/document.txt index edad04f172f..9d0b70ab725 100644 --- a/source/core/document.txt +++ b/source/core/document.txt @@ -1,50 +1,56 @@ -============== -BSON Documents -============== +========= +Documents +========= .. default-domain:: mongodb -MongoDB is a document-based database system, and as a result, all -records, or data, in MongoDB are documents. Documents are the default -representation of most user accessible data structures in the -database. Documents provide structure for data in the following -MongoDB contexts: +MongoDB stores all data in documents, which are JSON-style data +structures composed of field-and-value pairs: -- the :ref:`records ` stored in :term:`collections - ` +.. code-block:: javascript + + { "item": "pencil", "qty": 500, "type": "no.2" } + +Most user-accessible data structures in MongoDB are documents, +including: -- the :ref:`query selectors ` that determine - which records to select for read, update, and delete operations +- :ref:`Records `. MongoDB stores records in + :term:`collections ` -- the :ref:`update actions ` that specify - the particular field updates to perform during an update operation +- :ref:`Query selectors `, which define what + records to select for read, update, and delete operations. -- the specification of :ref:`indexes ` for - collection. +- :ref:`Update actions `, which define what + fields to modify during an update. -- arguments to several MongoDB methods and operators, including: +- :ref:`Index specifications `, which define what + fields to index. - - :ref:`sort order ` for the :method:`sort() - ` method. +- Arguments to many MongoDB methods and operators, such as the + :ref:`sort order document ` used by the + :method:`sort() ` method. - - :ref:`index specification ` for the - :method:`hint() ` method. +- Outputted data, such as the output of the :dbcommand:`collStats` + command. -- the output of a number of MongoDB commands and operations, including: +Document Format +--------------- - - the output - of :dbcommand:`collStats` command, and +MongoDB documents are stored on disk in the :term:`BSON` serialization +format. BSON is a binary representation of :term:`JSON` documents, +though contains more data types than does JSON. For the BSON spec, see +`bsonspec.org `_. See also +:doc:`/core/bson-types`. - - the :doc:`output ` of the - :dbcommand:`serverStatus` command. +The :program:`mongo` JavaScript shell and the :doc:`MongoDB language +drivers ` translate between BSON and the +language-specific document representation. -Structure ---------- +Document Structure +------------------ -The document structure in MongoDB are :term:`BSON` objects with -support for the full range of :term:`BSON types`; however, BSON -documents are conceptually, similar to :term:`JSON` objects, and have -the following structure: +MongoDB documents are composed of field-and-value pairs and have the +following structure: .. code-block:: javascript @@ -56,13 +62,9 @@ the following structure: fieldN: valueN } -Having support for the full range of BSON types, MongoDB documents may -contain field and value pairs where the value can be another document, -an array, an array of documents as well as the basic types such as -``Double``, ``String``, and ``Date``. See also -:ref:`document-bson-type-considerations`. - -Consider the following document that contains values of varying types: +The value of a field can be any of the BSON :doc:`data types +`, including other documents, arrays, and arrays of +documents. The following document contains values of varying types: .. code-block:: javascript @@ -75,77 +77,41 @@ Consider the following document that contains values of varying types: views : NumberLong(1250000) } -The document contains the following fields: - -- ``_id`` that holds an *ObjectId*. - -- ``name`` that holds a *subdocument* that contains the fields - ``first`` and ``last``. - -- ``birth`` and ``death``, which both have *Date* types. - -- ``contribs`` that holds an *array of strings*. - -- ``views`` that holds a value of *NumberLong* type. - -All field names are strings in :term:`BSON` documents. Be aware that -there are some :limit:`restrictions on field names -` for :term:`BSON` documents: field names -cannot contain null characters, dots (``.``), or dollar signs (``$``). - -.. note:: +The above fields have the following data types: - BSON documents may have more than one field with the same name; - however, most :doc:`MongoDB Interfaces ` - represent MongoDB with a structure (e.g. a hash table) that does - not support duplicate field names. If you need to manipulate - documents that have more than one field with the same name, see - your driver's documentation for more information. +- ``_id`` holds an *ObjectId*. - Some documents created by internal MongoDB processes may have - duplicate fields, but *no* MongoDB process will *ever* add - duplicate keys to an existing user document. +- ``name`` holds a *subdocument* that contains the fields ``first`` and + ``last``. -.. _document-type-operators: +- ``birth`` and ``death`` hold values of the *Date* type. -Type Operators -~~~~~~~~~~~~~~ +- ``contribs`` holds an *array of strings*. -To determine the type of fields, the :program:`mongo` shell provides -the following operators: +- ``views`` holds a value of the *NumberLong* type. -- ``instanceof`` returns a boolean to test if a value has - a specific type. +Field Names +----------- -- ``typeof`` returns the type of a field. +Field names are strings. Field names cannot contain null characters, +dots (``.``) or dollar signs (``$``). See +:ref:`faq-dollar-sign-escaping` for an alternate approach. -.. example:: +BSON documents may have more than one field with the same name. +Most :doc:`MongoDB interfaces `, however, represent MongoDB +with a structure (e.g. a hash table) that does not support duplicate +field names. If you need to manipulate documents that have more than one +field with the same name, see the :doc:`driver documentation +` for your driver. - Consider the following operations using ``instanceof`` and - ``typeof``: - - - The following operation tests whether the ``_id`` field is of type - ``ObjectId``: - - .. code-block:: javascript - - mydoc._id instanceof ObjectId - - The operation returns ``true``. - - - The following operation returns the type of the ``_id`` field: - - .. code-block:: javascript - - typeof mydoc._id - - In this case ``typeof`` will return the more generic ``object`` - type rather than ``ObjectId`` type. +Some documents created by internal MongoDB processes may have duplicate +fields, but *no* MongoDB process will *ever* add duplicate fields to an +existing user document. .. _document-dot-notation: Dot Notation -~~~~~~~~~~~~ +------------ .. include:: /includes/fact-dot-notation.rst @@ -157,426 +123,15 @@ Dot Notation - :ref:`read-operations-arrays` for dot notation examples with arrays. -Document Types in MongoDB -------------------------- - -.. _documents-records: - -Record Documents -~~~~~~~~~~~~~~~~ - -Most documents in MongoDB in :term:`collections ` store -data from users' applications. - -These documents have the following attributes: - -- .. include:: /includes/fact-document-max-size.rst - -- .. include:: /includes/fact-document-field-name-restrictions.rst - -.. include:: /includes/note-insert-id-field.rst - -The following document specifies a record in a collection: - - .. code-block:: javascript - - { - _id: 1, - name: { first: 'John', last: 'Backus' }, - birth: new Date('Dec 03, 1924'), - death: new Date('Mar 17, 2007'), - contribs: [ 'Fortran', 'ALGOL', 'Backus-Naur Form', 'FP' ], - awards: [ - { award: 'National Medal of Science', - year: 1975, - by: 'National Science Foundation' }, - { award: 'Turing Award', - year: 1977, - by: 'ACM' } - ] - } - -The document contains the following fields: - -- ``_id``, which must hold a unique value and is *immutable*. - -- ``name`` that holds another *document*. This sub-document contains - the fields ``first`` and ``last``, which both hold *strings*. - -- ``birth`` and ``death`` that both have *date* types. - -- ``contribs`` that holds an *array of strings*. - -- ``awards`` that holds an *array of documents*. - -Consider the following behavior and constraints of the ``_id`` field in -MongoDB documents: - -- In documents, the ``_id`` field is always indexed for regular - collections. - -- The ``_id`` field may contain values of any BSON data type other than - an array. - - .. warning:: To ensure functioning replication, do not store values - that are of the BSON regular expression type in the ``_id`` - field. - - .. See :issue:`SERVER-9562` for more information. - -Consider the following options for the value of an ``_id`` field: - -- Use an ``ObjectId``. See the :doc:`ObjectId ` - documentation. - - Although it is common to assign ``ObjectId`` values to ``_id`` - fields, if your objects have a natural unique identifier, consider - using that for the value of ``_id`` to save space and to avoid an - additional index. - -- Generate a sequence number for the documents in your collection in - your application and use this value for the ``_id`` value. See the - :doc:`/tutorial/create-an-auto-incrementing-field` tutorial for an - implementation pattern. - -- Generate a UUID in your application code. For a more efficient - storage of the UUID values in the collection and in the ``_id`` - index, store the UUID as a value of the BSON ``BinData`` type. - - .. include:: /includes/fact-bindata-storage-optimization.rst - -- Use your driver's BSON UUID facility to generate UUIDs. Be aware - that driver implementations may implement UUID serialization and - deserialization logic differently, which may not be fully compatible - with other drivers. See your :api:`driver documentation <>` for - information concerning UUID interoperability. - -.. _documents-query-selectors: -.. _mongodb-query-documents: -.. _mongodb-query-document: - -Query Specification Documents -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Query documents specify the conditions that determine which records to -select for read, update, and delete operations. You can use -``:`` expressions to specify the equality condition and -:doc:`query operator ` expressions to specify -additional conditions. - -When passed as an argument to methods such as the :method:`find() -` method, the :method:`remove() -` method, or the :method:`update() -` method, the query document selects documents -for MongoDB to return, remove, or update, as in the following: - -.. code-block:: javascript - - db.bios.find( { _id: 1 } ) - db.bios.remove( { _id: { $gt: 3 } } ) - db.bios.update( { _id: 1, name: { first: 'John', last: 'Backus' } }, - , - ) - -.. seealso:: - - - :ref:`read-operations-query-argument` and - :doc:`/core/read` for more examples on selecting documents - for reads. - - - :doc:`/core/update` for more examples on - selecting documents for updates. - - - :doc:`/core/delete` for more examples on selecting - documents for deletes. - -.. _documents-update-actions: - -Update Specification Documents -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Update documents specify the data modifications to perform during -an :method:`update() ` operation to modify -existing records in a collection. You can use :ref:`update operators -` to specify the exact actions to perform on the -document fields. - -Consider the update document example: - -.. code-block:: javascript - - { - $set: { 'name.middle': 'Warner' }, - $push: { awards: { award: 'IBM Fellow', - year: '1963', - by: 'IBM' } - } - } - -When passed as an argument to the :method:`update() -` method, the update actions document: - -- Modifies the field ``name`` whose value is another document. - Specifically, the :operator:`$set` operator updates the ``middle`` - field in the ``name`` subdocument. The document uses :ref:`dot - notation ` to access a field in a subdocument. - -- Adds an element to the field ``awards`` whose value is an array. - Specifically, the :operator:`$push` operator adds another document as - element to the field ``awards``. - -.. code-block:: javascript - - db.bios.update( - { _id: 1 }, - { - $set: { 'name.middle': 'Warner' }, - $push: { awards: { - award: 'IBM Fellow', - year: '1963', - by: 'IBM' - } - } - } - ) - -.. seealso:: - - - :ref:`update operators ` page for the available - update operators and syntax. - - - :doc:`update ` for more examples on - update documents. - -For additional examples of updates that involve array elements, -including where the elements are documents, see the :operator:`$` -positional operator. - -.. _documents-index: -.. _document-index-specification: - -Index Specification Documents -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Index specification documents describe the fields to index on during -the :doc:`index creation -`. See :doc:`indexes -` for an overview of indexes. [#index-def]_ - -Index documents contain field and value pairs, in the following form: - -.. code-block:: javascript - - { field: value } - -- ``field`` is the field in the documents to index. - -- ``value`` is either 1 for ascending or -1 for descending. - -The following document specifies the :ref:`multi-key index -` on the ``_id`` field and the ``last`` field -contained in the subdocument ``name`` field. The document uses -:ref:`dot notation ` to access a field in a -subdocument: - -.. code-block:: javascript - - { _id: 1, 'name.last': 1 } - -When passed as an argument to the :method:`ensureIndex() -` method, the index documents specifies -the index to create: - -.. code-block:: javascript - - db.bios.ensureIndex( { _id: 1, 'name.last': 1 } ) - -.. [#index-def] Indexes optimize a number of key :doc:`read - ` and :doc:`write ` - operations. - -.. _documents-sort-order: - -Sort Order Specification Documents -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Sort order documents specify the order of documents that a -:method:`query() ` returns. Pass sort order -specification documents as an argument to the :method:`sort() -` method. See the :method:`sort() ` page -for more information on sorting. - -The sort order documents contain field and value pairs, in the following -form: - -.. code-block:: javascript - - { field: value } - -- ``field`` is the field by which to sort documents. - -- ``value`` is either 1 for ascending or -1 for descending. - -The following document specifies the sort order using the fields from a -sub-document ``name`` first sort by the ``last`` field ascending, then -by the ``first`` field also ascending: - -.. code-block:: javascript - - { 'name.last': 1, 'name.first': 1 } - -When passed as an argument to the :method:`sort() ` -method, the sort order document sorts the results of the -:method:`find() ` method: - -.. code-block:: javascript - - db.bios.find().sort( { 'name.last': 1, 'name.first': 1 } ) - -.. _document-mongodb-type-considerations: -.. _document-bson-type-considerations: - -BSON Type Considerations ------------------------- - -The following BSON types require special consideration: - -.. _document-bson-type-object-id: - -ObjectId -~~~~~~~~ - -ObjectIds are: small, likely unique, fast to generate, and ordered. -These values consists of 12-bytes, where the first 4-bytes is a -timestamp that reflects the ObjectId's creation. Refer to the -:doc:`ObjectId ` documentation for more information. - -.. _document-bson-type-string: - -String -~~~~~~ - -BSON strings are UTF-8. In general, drivers for each programming -language convert from the language's string format to UTF-8 when -serializing and deserializing BSON. This makes it possible to store -most international characters in BSON strings with ease. -[#sort-string-internationalization]_ In addition, MongoDB -:operator:`$regex` queries support UTF-8 in the regex string. - -.. [#sort-string-internationalization] Given strings using UTF-8 - character sets, using :method:`sort() ` on strings - will be reasonably correct; however, because internally - :method:`sort() ` uses the C++ ``strcmp`` api, the - sort order may handle some characters incorrectly. - -.. _document-bson-type-timestamp: - -Timestamps -~~~~~~~~~~ - -BSON has a special timestamp type for *internal* MongoDB use and is -**not** associated with the regular :ref:`document-bson-type-date` -type. Timestamp values are a 64 bit value where: - -- the first 32 bits are a ``time_t`` value (seconds since the Unix epoch) - -- the second 32 bits are an incrementing ``ordinal`` for operations - within a given second. - -Within a single :program:`mongod` instance, timestamp values are -always unique. - -In replication, the :term:`oplog` has a ``ts`` field. The values in -this field reflect the operation time, which uses a BSON timestamp -value. - -.. note:: - - The BSON Timestamp type is for *internal* MongoDB use. For most - cases, in application development, you will want to use the BSON - date type. See :ref:`document-bson-type-date` for more - information. - -If you create a BSON Timestamp using the empty constructor (e.g. ``new -Timestamp()``), MongoDB will only generate a timestamp *if* you use -the constructor in the first field of the document. [#id_exception]_ -Otherwise, MongoDB will generate an empty timestamp value -(i.e. ``Timestamp(0, 0)``.) - -.. versionchanged:: 2.1 - :program:`mongo` shell displays the Timestamp value with the wrapper: - - .. code-block:: javascript - - Timestamp(, ) - - Prior to version 2.1, the :program:`mongo` shell display the - Timestamp value as a document: - - .. code-block:: javascript - - { t : , i : } - -.. [#id_exception] If the first field in the document is ``_id``, then - you can generate a timestamp in the *second* field - of a document. - - .. only:: html or dirhtml or singlehtml or epub or gettext - - In the following example, MongoDB will generate a Timestamp - value, even though the ``Timestamp()`` constructor is *not* in - the first field in the document: - - .. code-block:: javascript - - db.bios.insert( { _id: 9, last_updated: new Timestamp() } ) - -.. _document-bson-type-date: - -Date -~~~~ - -BSON Date is a 64-bit integer that represents the number of -milliseconds since the Unix epoch (Jan 1, 1970). The `official BSON -specification `_ refers to the -BSON Date type as the *UTC datetime*. - -.. versionchanged:: 2.0 - BSON Date type is signed. [#unsigned-date]_ Negative values - represent dates before 1970. - -Consider the following examples of BSON Date: - -- Construct a Date using the ``new Date()`` constructor in the - :program:`mongo` shell: - - .. code-block:: javascript - - var mydate1 = new Date() - -- Construct a Date using the ``ISODate()`` constructor in the - :program:`mongo` shell: - - .. code-block:: javascript - - var mydate2 = ISODate() - -- Return the ``Date`` value as string: - - .. code-block:: javascript - - mydate1.toString() - -- Return the month portion of the Date value; months are zero-indexed, - so that January is month ``0``: - - .. code-block:: javascript +.. class:: hidden - mydate1.getMonth() + .. toctree:: + :titlesonly: -.. [#unsigned-date] Prior to version 2.0, ``Date`` values were - incorrectly interpreted as *unsigned* integers, which affected - sorts, range queries, and indexes on ``Date`` fields. Because - indexes are not recreated when upgrading, please re-index if you - created an index on ``Date`` values with an earlier version, and - dates before 1970 are relevant to your application. + /core/document-type-record + /core/document-type-query-specification + /core/document-type-update-specification + /core/document-type-index-specification + /core/document-type-sort-order + /core/bson-types + /core/document-type-operators diff --git a/source/includes/fact-dot-notation.rst b/source/includes/fact-dot-notation.rst index f34573b70df..2c4bf98e945 100644 --- a/source/includes/fact-dot-notation.rst +++ b/source/includes/fact-dot-notation.rst @@ -1,7 +1,7 @@ MongoDB uses the *dot notation* to access the elements of an array and to access the fields of a subdocument. -To access an element of an array by the zero-based index position, you +To access an element of an array by the zero-based index position, concatenate the array name with the dot (``.``) and zero-based index position: @@ -9,7 +9,7 @@ position: '.' -To access a field of a subdocument with *dot-notation*, you concatenate +To access a field of a subdocument with *dot-notation*, concatenate the subdocument name with the dot (``.``) and the field name: .. code-block:: javascript diff --git a/source/reference/glossary.txt b/source/reference/glossary.txt index 03927b9a59a..d90f938a0f3 100644 --- a/source/reference/glossary.txt +++ b/source/reference/glossary.txt @@ -14,8 +14,8 @@ Glossary of JSON (JavaScript Object Notation) documents. For a detailed spec, see `bsonspec.org `_. - .. seealso:: The :ref:`bson-json-type-conversion-fidelity` - section. + .. seealso:: :doc:`/core/document`, :doc:`/core/bson-types` and + :ref:`bson-json-type-conversion-fidelity` database command Any MongoDB operation other than an insert, update, remove, @@ -179,30 +179,7 @@ Glossary BSON types The set of types supported by the :term:`BSON` serialization - format. The following types are available: - - ======================= ========== - **Type** **Number** - ----------------------- ---------- - Double 1 - String 2 - Object 3 - Array 4 - Binary data 5 - Object id 7 - Boolean 8 - Date 9 - Null 10 - Regular Expression 11 - JavaScript 13 - Symbol 14 - JavaScript (with scope) 15 - 32-bit integer 16 - Timestamp 17 - 64-bit integer 18 - Min key 255 - Max key 127 - ======================= ========== + format. For a list of BSON types, see :doc:`/core/bson-types`. master In conventional master/:term:`slave` replication, the master From 72bc68a65e4f54963e255e4e32954f70252f7ceb Mon Sep 17 00:00:00 2001 From: schmalliso Date: Wed, 10 Jul 2013 14:03:46 -0400 Subject: [PATCH 06/14] crud-reorg: edit top level page --- source/crud.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/source/crud.txt b/source/crud.txt index 8f2a4bede22..1d5d3f2e05f 100644 --- a/source/crud.txt +++ b/source/crud.txt @@ -26,7 +26,8 @@ deployments. The :doc:`/core/document` provides an overview of /core/read-operations /core/write-operations - /reference/write-concern + /core/write-concern + Fundamental Concepts for Document Databases ------------------------------------------- @@ -40,6 +41,7 @@ Fundamental Concepts for Document Databases /reference/object-id /core/gridfs /reference/database-references + /reference/write-concern .. _crud-documents: .. _crud-operations: From e0e3de4b175438bbb5e951a1bf81b14ab6d313c7 Mon Sep 17 00:00:00 2001 From: Sam Kleinman Date: Fri, 12 Jul 2013 16:11:34 -0400 Subject: [PATCH 07/14] crud-reorg: reference section edits --- source/core/read-operations.txt | 4 +- source/crud.txt | 37 ++++++------- source/reference.txt | 2 - source/reference/object-id.txt | 96 ++++++++++++++++++--------------- 4 files changed, 72 insertions(+), 67 deletions(-) diff --git a/source/core/read-operations.txt b/source/core/read-operations.txt index d73d8a061a4..9e643115cfb 100644 --- a/source/core/read-operations.txt +++ b/source/core/read-operations.txt @@ -24,11 +24,9 @@ queries applications use to request data from MongoDB and how different factors affect the efficiency of reads. .. toctree:: - :maxdepth: 1 + :titlesonly: /core/read-operations-queries /core/read-operations-indexes /core/read-operations-cursors /core/read-operations-architecture-considerations - - diff --git a/source/crud.txt b/source/crud.txt index 1d5d3f2e05f..c2c6969978c 100644 --- a/source/crud.txt +++ b/source/crud.txt @@ -22,40 +22,41 @@ deployments. The :doc:`/core/document` provides an overview of :term:`documents ` and document-orientation in MongoDB. .. toctree:: - :maxdepth: 1 + :titlesonly: /core/read-operations /core/write-operations /core/write-concern - - -Fundamental Concepts for Document Databases -------------------------------------------- - -.. todo insert signposting blurb here. - -.. toctree:: - :maxdepth: 1 - - /core/document - /reference/object-id /core/gridfs - /reference/database-references - /reference/write-concern .. _crud-documents: .. _crud-operations: -CRUD Operations for MongoDB ---------------------------- +MongoDB CRUD Operation Examples +------------------------------- These documents provide an overview and examples of common database operations, i.e. CRUD, in MongoDB. .. toctree:: - :maxdepth: 1 + :titlesonly: /core/create /core/read /core/update /core/delete + +Core Operations Reference +------------------------- + +.. todo insert signposting blurb here. + +.. toctree:: + :titlesonly: + + /core/document + /reference/write-concern + /reference/object-id + /reference/gridfs + /reference/database-references + diff --git a/source/reference.txt b/source/reference.txt index e04098df14f..f6b8a3f7477 100644 --- a/source/reference.txt +++ b/source/reference.txt @@ -54,8 +54,6 @@ General Reference reference/limits reference/connection-string reference/mongodb-extended-json - reference/database-references - reference/gridfs reference/glossary .. seealso:: The :ref:`genindex` may provide useful insight into the diff --git a/source/reference/object-id.txt b/source/reference/object-id.txt index ec43b54b710..53588bbc5e4 100644 --- a/source/reference/object-id.txt +++ b/source/reference/object-id.txt @@ -51,7 +51,7 @@ information on MongoDB's document orientation. ObjectId() ---------- -The :program:`mongo` shell provides the ``ObjectId()`` wrapper class to +The :program:`mongo` shell provides the ``ObjectId()`` wrapper class toy generate a new ObjectId, and to provide the following helper attribute and methods: @@ -87,81 +87,89 @@ Examples Consider the following uses ``ObjectId()`` class in the :program:`mongo` shell: -- To generate a new ObjectId, use the ``ObjectId()`` constructor with - no argument: +Generate a new ObjectId +~~~~~~~~~~~~~~~~~~~~~~~ - .. code-block:: javascript +To generate a new ObjectId, use the ``ObjectId()`` constructor with +no argument: - x = ObjectId() +.. code-block:: javascript - In this example, the value of ``x`` would be: + x = ObjectId() - .. code-block:: javascript +In this example, the value of ``x`` would be: - ObjectId("507f1f77bcf86cd799439011") +.. code-block:: javascript -- To generate a new ObjectId using the ``ObjectId()`` constructor with - a unique hexadecimal string: + ObjectId("507f1f77bcf86cd799439011") - .. code-block:: javascript +To generate a new ObjectId using the ``ObjectId()`` constructor with +a unique hexadecimal string: - y = ObjectId("507f191e810c19729de860ea") +.. code-block:: javascript - In this example, the value of ``y`` would be: + y = ObjectId("507f191e810c19729de860ea") - .. code-block:: javascript +In this example, the value of ``y`` would be: - ObjectId("507f191e810c19729de860ea") +.. code-block:: javascript -- To return the timestamp of an ``ObjectId()`` object, use the - :method:`getTimestamp() ` method as follows: + ObjectId("507f191e810c19729de860ea") - .. code-block:: javascript +Convert an ObjectId into a Timestamp +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ObjectId("507f191e810c19729de860ea").getTimestamp() +To return the timestamp of an ``ObjectId()`` object, use the +:method:`getTimestamp() ` method as follows: - This operation will return the following Date object: +.. code-block:: javascript - .. code-block:: javascript + ObjectId("507f191e810c19729de860ea").getTimestamp() - ISODate("2012-10-17T20:46:22Z") +This operation will return the following Date object: -- Access the ``str`` attribute of an ``ObjectId()`` object, as - follows: +.. code-block:: javascript - .. code-block:: javascript + ISODate("2012-10-17T20:46:22Z") - ObjectId("507f191e810c19729de860ea").str +Convert ObjectIds into Strings +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - This operation will return the following hexadecimal string: +Access the ``str`` attribute of an ``ObjectId()`` object, as follows: - .. code-block:: none +.. code-block:: javascript - 507f191e810c19729de860ea + ObjectId("507f191e810c19729de860ea").str -- To return the string representation of an ``ObjectId()`` object, use - the :method:`~ObjectId.toString()` method as follows: +This operation will return the following hexadecimal string: - .. code-block:: javascript +.. code-block:: none - ObjectId("507f191e810c19729de860ea").toString() + 507f191e810c19729de860ea - This operation will return the following output: +To return the value of an ``ObjectId()`` object as a hexadecimal +string, use the :method:`~ObjectId.valueOf()` method as +follows: - .. code-block:: javascript +.. code-block:: javascript - ObjectId("507f191e810c19729de860ea") + ObjectId("507f191e810c19729de860ea").valueOf() -- To return the value of an ``ObjectId()`` object as a hexadecimal - string, use the :method:`~ObjectId.valueOf()` method as - follows: +This operation returns the following output: - .. code-block:: javascript +.. code-block:: none - ObjectId("507f191e810c19729de860ea").valueOf() + 507f191e810c19729de860ea - This operation returns the following output: +To return the string representation of an ``ObjectId()`` object, use +the :method:`~ObjectId.toString()` method as follows: - .. code-block:: none +.. code-block:: javascript - 507f191e810c19729de860ea + ObjectId("507f191e810c19729de860ea").toString() + +This operation will return the following output: + +.. code-block:: javascript + + ObjectId("507f191e810c19729de860ea") From 187d6fcffca676e2735e59f147a177ebed8d3ceb Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Thu, 11 Jul 2013 17:26:40 -0400 Subject: [PATCH 08/14] move the create examples, create the bios collection --- source/core/create.txt | 531 ++---------------- source/crud.txt | 2 +- source/reference/bios-example-collection.txt | 287 ++++++++++ .../reference/method/db.collection.insert.txt | 384 ++++++++++--- .../reference/method/db.collection.save.txt | 147 +++-- .../method/db.collection.update-param.yaml | 32 +- .../reference/method/db.collection.update.txt | 317 ++++++++--- 7 files changed, 1008 insertions(+), 692 deletions(-) create mode 100644 source/reference/bios-example-collection.txt diff --git a/source/core/create.txt b/source/core/create.txt index f44d53773fb..477c2dd68df 100644 --- a/source/core/create.txt +++ b/source/core/create.txt @@ -4,55 +4,49 @@ Create .. default-domain:: mongodb -Of the four basic database operations (i.e. CRUD), *create* operations -are those that add new records or :term:`documents ` to a -:term:`collection` in MongoDB. For general information about write -operations and the factors that affect their performance, see -:doc:`/core/write-operations`; for documentation of the other CRUD -operations, see the :doc:`/crud` page. +*Create* operations add new :term:`documents ` to a +:term:`collection`. A document is the equivalent of an :term:`RDBMS` +record, and a collection is the equivalent of an RDBMS table. *Create* +operations are one of the three categories of +:doc:`/core/write-operations`, along with :doc:`/core/update` and +:doc:`/core/delete` operations. -.. contents:: - :backlinks: none - :local: +You can create documents in a MongoDB collection using either of the +following *create* operations: -Overview --------- +- :ref:`Inserts `, which are MongoDB's core create + operation. -You can create documents in a MongoDB collection using any of the -following basic operations: +- :ref:`Updates that include the upsert flag `. + These updates create a new document if no document matches the update + criteria. -- :ref:`insert ` +The create operations have the following behaviors: -- :ref:`updates with the upsert option ` - -All insert operations in MongoDB exhibit the following properties: - -- If you attempt to insert a document without the :term:`_id` field, - the client library *or* the :program:`mongod` instance will add an - ``_id`` field and populate the field with a unique :term:`ObjectId - `. +- If you insert a document *without* the :term:`_id` field, the client + library or :program:`mongod` instance adds an ``_id`` field and + populates the field with a unique :term:`ObjectId `. - For operations with :ref:`write concern `, if you - specify an ``_id`` field, the ``_id`` field must be unique within - the collection; otherwise the :program:`mongod` will return a - duplicate key exception. + specify an ``_id`` field, the field must be unique within the + collection. Otherwise :program:`mongod` returns a duplicate key + exception. -- .. include:: /includes/fact-document-max-size.rst + .. note:: -- .. include:: /includes/fact-document-field-name-restrictions.rst + .. include:: /includes/fact-write-concern.rst -.. note:: +.. include:: /includes/fact-document-max-size.rst - .. include:: /includes/fact-write-concern.rst +.. include:: /includes/fact-document-field-name-restrictions.rst .. _crud-create-insert: -``insert()`` ------------- +Insert Operations +----------------- -The :method:`~db.collection.insert()` is the primary method -to insert a document or documents into a MongoDB collection, and has -the following syntax: +The :method:`~db.collection.insert()` method is the primary method to +insert documents into a MongoDB collection and has the following syntax: .. code-block:: javascript @@ -63,462 +57,61 @@ the following syntax: The :method:`~db.collection.insert()` method is analogous to the ``INSERT`` statement. -Insert the First Document in a Collection -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If the collection does not exist [#show-collection]_, then the -:method:`~db.collection.insert()` method creates the -collection during the first insert. Specifically in the example, if the -collection ``bios`` does not exist , then the insert operation will -create this collection: - -.. code-block:: javascript - - db.bios.insert( - { - _id: 1, - name: { first: 'John', last: 'Backus' }, - birth: new Date('Dec 03, 1924'), - death: new Date('Mar 17, 2007'), - contribs: [ 'Fortran', 'ALGOL', 'Backus-Naur Form', 'FP' ], - awards: [ - { - award: 'W.W. McDowell Award', - year: 1967, - by: 'IEEE Computer Society' - }, - { - award: 'National Medal of Science', - year: 1975, - by: 'National Science Foundation' - }, - { - award: 'Turing Award', - year: 1977, - by: 'ACM' - }, - { - award: 'Draper Prize', - year: 1993, - by: 'National Academy of Engineering' - } - ] - } - ) - -You can confirm the insert by :doc:`querying ` -the ``bios`` collection: - -.. code-block:: javascript - - db.bios.find() - -This operation returns the following document from the ``bios`` -collection: - -.. code-block:: javascript +.. example:: The following inserts a document with three fields: - { - "_id" : 1, - "name" : { "first" : "John", "last" : "Backus" }, - "birth" : ISODate("1924-12-03T05:00:00Z"), - "death" : ISODate("2007-03-17T04:00:00Z"), - "contribs" : [ "Fortran", "ALGOL", "Backus-Naur Form", "FP" ], - "awards" : [ - { - "award" : "W.W. McDowell Award", - "year" : 1967, - "by" : "IEEE Computer Society" - }, - { - "award" : "National Medal of Science", - "year" : 1975, - "by" : "National Science Foundation" - }, - { - "award" : "Turing Award", - "year" : 1977, - "by" : "ACM" - }, - { "award" : "Draper Prize", - "year" : 1993, - "by" : "National Academy of Engineering" - } - ] - } - -Insert a Document without Specifying an ``_id`` Field -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If the new document does not contain an ``_id`` field, then the -:method:`~db.collection.insert()` method adds the ``_id`` -field to the document and generates a unique ``ObjectId`` for the value: + .. code-block:: javascript -.. code-block:: javascript + db.products.insert( { _id: 10, item: "card", qty: 15 } ) - db.bios.insert( - { - name: { first: 'John', last: 'McCarthy' }, - birth: new Date('Sep 04, 1927'), - death: new Date('Dec 24, 2011'), - contribs: [ 'Lisp', 'Artificial Intelligence', 'ALGOL' ], - awards: [ - { - award: 'Turing Award', - year: 1971, - by: 'ACM' - }, - { - award: 'Kyoto Prize', - year: 1988, - by: 'Inamori Foundation' - }, - { - award: 'National Medal of Science', - year: 1990, - by: 'National Science Foundation' - } - ] - } - ) - -You can verify the inserted document by the querying the ``bios`` -collection: +For further examples, see :method:`~db.collection.insert()` and +:ref:`crud-create-insert-save`. -.. code-block:: javascript +.. _crud-create-update: - db.bios.find( { name: { first: 'John', last: 'McCarthy' } } ) +Update Operations that Include the ``upsert`` Flag +-------------------------------------------------- -The returned document contains an ``_id`` field with the generated -``ObjectId`` value: +An :method:`~db.collection.update()` operation with the ``upsert`` flag +creates a new document *if no document matches the update's query +criteria*. The operation has the following syntax +[#previous-versions-upsert]_: .. code-block:: javascript - { - "_id" : ObjectId("50a1880488d113a4ae94a94a"), - "name" : { "first" : "John", "last" : "McCarthy" }, - "birth" : ISODate("1927-09-04T04:00:00Z"), - "death" : ISODate("2011-12-24T05:00:00Z"), - "contribs" : [ "Lisp", "Artificial Intelligence", "ALGOL" ], - "awards" : [ - { - "award" : "Turing Award", - "year" : 1971, - "by" : "ACM" - }, - { - "award" : "Kyoto Prize", - "year" :1988, - "by" : "Inamori Foundation" - }, - { - "award" : "National Medal of Science", - "year" : 1990, - "by" : "National Science Foundation" - } - ] - } - -Bulk Insert Multiple Documents -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you pass an array of documents to the -:method:`~db.collection.insert()` method, the -:method:`~db.collection.insert()` performs a bulk insert into a -collection. - -The following operation inserts three documents into the ``bios`` -collection. The operation also illustrates the *dynamic schema* -characteristic of MongoDB. Although the document with ``_id: 3`` -contains a field ``title`` which does not appear in the other -documents, MongoDB does not require the other documents to contain this -field: - -.. code-block:: javascript + db.collection.update( , , { upsert : true } ) - db.bios.insert( - [ - { - _id: 3, - name: { first: 'Grace', last: 'Hopper' }, - title: 'Rear Admiral', - birth: new Date('Dec 09, 1906'), - death: new Date('Jan 01, 1992'), - contribs: [ 'UNIVAC', 'compiler', 'FLOW-MATIC', 'COBOL' ], - awards: [ - { - award: 'Computer Sciences Man of the Year', - year: 1969, - by: 'Data Processing Management Association' - }, - { - award: 'Distinguished Fellow', - year: 1973, - by: ' British Computer Society' - }, - { - award: 'W. W. McDowell Award', - year: 1976, - by: 'IEEE Computer Society' - }, - { - award: 'National Medal of Technology', - year: 1991, - by: 'United States' - } - ] - }, - { - _id: 4, - name: { first: 'Kristen', last: 'Nygaard' }, - birth: new Date('Aug 27, 1926'), - death: new Date('Aug 10, 2002'), - contribs: [ 'OOP', 'Simula' ], - awards: [ - { - award: 'Rosing Prize', - year: 1999, - by: 'Norwegian Data Association' - }, - { - award: 'Turing Award', - year: 2001, - by: 'ACM' - }, - { - award: 'IEEE John von Neumann Medal', - year: 2001, - by: 'IEEE' - } - ] - }, - { - _id: 5, - name: { first: 'Ole-Johan', last: 'Dahl' }, - birth: new Date('Oct 12, 1931'), - death: new Date('Jun 29, 2002'), - contribs: [ 'OOP', 'Simula' ], - awards: [ - { - award: 'Rosing Prize', - year: 1999, - by: 'Norwegian Data Association' - }, - { - award: 'Turing Award', - year: 2001, - by: 'ACM' - }, - { - award: 'IEEE John von Neumann Medal', - year: 2001, - by: 'IEEE' - } - ] - } - ] - ) - -.. _crud-create-insert-save: - -Insert a Document with ``save()`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The :method:`~db.collection.save()` method performs an insert -if the document to save does not contain the ``_id`` field. - -The following :method:`~db.collection.save()` operation performs an -insert into the ``bios`` collection since the document does not contain -the ``_id`` field: +If no document matches the ```` criteria, the ``upsert`` creates +a new document with either: -.. code-block:: javascript +- the fields and values of the ```` parameter, or - db.bios.save( - { - name: { first: 'Guido', last: 'van Rossum'}, - birth: new Date('Jan 31, 1956'), - contribs: [ 'Python' ], - awards: [ - { - award: 'Award for the Advancement of Free Software', - year: 2001, - by: 'Free Software Foundation' - }, - { - award: 'NLUUG Award', - year: 2003, - by: 'NLUUG' - } - ] - } - ) - -.. [#show-collection] You can also view a list of the existing - collections in the database using the ``show collections`` - operation in the :program:`mongo` shell. +- the fields and values of the both the ```` and ```` + parameters. -.. _crud-create-update: +The ``upsert`` creates a document with data from both ```` and +```` if the ```` parameter *only uses update operators*. +See :ref:`update-operators`. -``update()`` Operations with the ``upsert`` Flag ------------------------------------------------- +.. example:: The following ``upsert`` operation either updates a + document or creates a new one: -The :method:`~db.collection.update()` operation in MongoDB accepts an -"``upsert``" flag that modifies the behavior of -:method:`~db.collection.update()` from :doc:`updating existing documents -`, to inserting data. + .. code-block:: javascript -These :method:`~db.collection.update()` operations with the upsert -flag eliminate the need to perform an additional operation to check -for existence of a record before performing either an update or an -insert operation. These update operations have the use ```` -argument to determine the write operation: + db.favorites.update( { author : "Dante" } , { $inc : { votes : 1 } } , { upsert : true } ) -- If the query matches an existing document(s), the operation is an - :doc:`update `. + If no document contains ``{ author : "Dante" }``, the ``upsert`` + creates a new document with an auto-generated ``_id``: -- If the query matches no document in the collection, the operation is - an :doc:`insert `. + .. code-block:: javascript -An upsert operation has the following syntax [#previous-versions-upsert]_: + { "_id" : ObjectId("51e01a2d9c6ae665454e301b"), "author" : "Dante", "votes" : 1 } -.. code-block:: javascript + If one or more documents exist with ``{ author : "Dante" }``, the + ``upsert`` does not create a new document but instead increments the + ``votes`` field in an existing document. - db.collection.update( , - , - { upsert: true } ) +For further examples, see :method:`~db.collection.update()` and +:ref:`crud-create-save`. .. [#previous-versions-upsert] .. include:: /includes/fact-upsert-multi-options.rst - -Insert a Document that Contains ``field`` and ``value`` Pairs -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If no document matches the ```` argument, the ``upsert`` -performs an insert. If the ```` argument includes only field -and value pairs, the new document contains the fields and values -specified in the ```` argument. If query does not include an -``_id`` field, the operation adds the ``_id`` field and generates a -unique ``ObjectId`` for its value. - -The following update inserts a new document into the -``bios`` collection [#previous-versions-upsert]_: - -.. code-block:: javascript - - db.bios.update( - { name: { first: 'Dennis', last: 'Ritchie'} }, - { - name: { first: 'Dennis', last: 'Ritchie'}, - birth: new Date('Sep 09, 1941'), - death: new Date('Oct 12, 2011'), - contribs: [ 'UNIX', 'C' ], - awards: [ - { - award: 'Turing Award', - year: 1983, - by: 'ACM' - }, - { - award: 'National Medal of Technology', - year: 1998, - by: 'United States' - }, - { - award: 'Japan Prize', - year: 2011, - by: 'The Japan Prize Foundation' - } - ] - }, - { upsert: true } - ) - -Insert a Document that Contains Update Operator Expressions -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If no document matches the ```` argument, the update operation -inserts a new document. If the ```` argument includes only -:ref:`update operators `, the new document contains -the fields and values from ```` argument with the operations -from the ```` argument applied. - -The following operation inserts a new document into the ``bios`` -collection [#previous-versions-upsert]_: - -.. code-block:: javascript - - db.bios.update( - { - _id: 7, - name: { first: 'Ken', last: 'Thompson' } - }, - { - $set: { - birth: new Date('Feb 04, 1943'), - contribs: [ 'UNIX', 'C', 'B', 'UTF-8' ], - awards: [ - { - award: 'Turing Award', - year: 1983, - by: 'ACM' - }, - { - award: 'IEEE Richard W. Hamming Medal', - year: 1990, - by: 'IEEE' - }, - { - award: 'National Medal of Technology', - year: 1998, - by: 'United States' - }, - { - award: 'Tsutomu Kanai Award', - year: 1999, - by: 'IEEE' - }, - { - award: 'Japan Prize', - year: 2011, - by: 'The Japan Prize Foundation' - } - ] - } - }, - { upsert: true } - ) - -.. _crud-create-save: - -Update operations with ``save()`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The :method:`~db.collection.save()` method is identical to an :ref:`update -operation with the upsert flag ` - -performs an upsert if the document to save contains the ``_id`` -field. To determine whether to perform an insert or an update, -:method:`~db.collection.save()` method queries documents on the -``_id`` field. - -The following operation performs an upsert that inserts a document into -the ``bios`` collection since no documents in the collection contains -an ``_id`` field with the value ``10``: - -.. code-block:: javascript - - db.bios.save( - { - _id: 10, - name: { first: 'Yukihiro', aka: 'Matz', last: 'Matsumoto'}, - birth: new Date('Apr 14, 1965'), - contribs: [ 'Ruby' ], - awards: [ - { - award: 'Award for the Advancement of Free Software', - year: '2011', - by: 'Free Software Foundation' - } - ] - } - ) diff --git a/source/crud.txt b/source/crud.txt index c2c6969978c..0e299b19726 100644 --- a/source/crud.txt +++ b/source/crud.txt @@ -59,4 +59,4 @@ Core Operations Reference /reference/object-id /reference/gridfs /reference/database-references - + /reference/bios-example-collection diff --git a/source/reference/bios-example-collection.txt b/source/reference/bios-example-collection.txt new file mode 100644 index 00000000000..b93ee582ea8 --- /dev/null +++ b/source/reference/bios-example-collection.txt @@ -0,0 +1,287 @@ +=============================== +The ``bios`` Example Collection +=============================== + +.. default-domain:: mongodb + +The ``bios`` collection provides example data for experimenting with +MongoDB. Many of this guide's examples on :method:`insert +`, :method:`update ` and +:method:`read ` operations create or query data +from the ``bios`` collection. + +The following documents comprise the ``bios`` collection. In the +examples, the data might be different, as the examples themselves make +changes to the data. + +.. code-block:: javascript + + { + "_id" : 1, + "name" : { + "first" : "John", + "last" : "Backus" + }, + "birth" : ISODate("1924-12-03T05:00:00Z"), + "death" : ISODate("2007-03-17T04:00:00Z"), + "contribs" : [ + "Fortran", + "ALGOL", + "Backus-Naur Form", + "FP" + ], + "awards" : [ + { + "award" : "W.W. McDowell Award", + "year" : 1967, + "by" : "IEEE Computer Society" + }, + { + "award" : "National Medal of Science", + "year" : 1975, + "by" : "National Science Foundation" + }, + { + "award" : "Turing Award", + "year" : 1977, + "by" : "ACM" + }, + { + "award" : "Draper Prize", + "year" : 1993, + "by" : "National Academy of Engineering" + } + ] + } + + { + "_id" : ObjectId("51df07b094c6acd67e492f41"), + "name" : { + "first" : "John", + "last" : "McCarthy" + }, + "birth" : ISODate("1927-09-04T04:00:00Z"), + "death" : ISODate("2011-12-24T05:00:00Z"), + "contribs" : [ + "Lisp", + "Artificial Intelligence", + "ALGOL" + ], + "awards" : [ + { + "award" : "Turing Award", + "year" : 1971, + "by" : "ACM" + }, + { + "award" : "Kyoto Prize", + "year" : 1988, + "by" : "Inamori Foundation" + }, + { + "award" : "National Medal of Science", + "year" : 1990, + "by" : "National Science Foundation" + } + ] + } + + { + "_id" : 3, + "name" : { + "first" : "Grace", + "last" : "Hopper" + }, + "title" : "Rear Admiral", + "birth" : ISODate("1906-12-09T05:00:00Z"), + "death" : ISODate("1992-01-01T05:00:00Z"), + "contribs" : [ + "UNIVAC", + "compiler", + "FLOW-MATIC", + "COBOL" + ], + "awards" : [ + { + "award" : "Computer Sciences Man of the Year", + "year" : 1969, + "by" : "Data Processing Management Association" + }, + { + "award" : "Distinguished Fellow", + "year" : 1973, + "by" : " British Computer Society" + }, + { + "award" : "W. W. McDowell Award", + "year" : 1976, + "by" : "IEEE Computer Society" + }, + { + "award" : "National Medal of Technology", + "year" : 1991, + "by" : "United States" + } + ] + } + + { + "_id" : 4, + "name" : { + "first" : "Kristen", + "last" : "Nygaard" + }, + "birth" : ISODate("1926-08-27T04:00:00Z"), + "death" : ISODate("2002-08-10T04:00:00Z"), + "contribs" : [ + "OOP", + "Simula" + ], + "awards" : [ + { + "award" : "Rosing Prize", + "year" : 1999, + "by" : "Norwegian Data Association" + }, + { + "award" : "Turing Award", + "year" : 2001, + "by" : "ACM" + }, + { + "award" : "IEEE John von Neumann Medal", + "year" : 2001, + "by" : "IEEE" + } + ] + } + + { + "_id" : 5, + "name" : { + "first" : "Ole-Johan", + "last" : "Dahl" + }, + "birth" : ISODate("1931-10-12T04:00:00Z"), + "death" : ISODate("2002-06-29T04:00:00Z"), + "contribs" : [ + "OOP", + "Simula" + ], + "awards" : [ + { + "award" : "Rosing Prize", + "year" : 1999, + "by" : "Norwegian Data Association" + }, + { + "award" : "Turing Award", + "year" : 2001, + "by" : "ACM" + }, + { + "award" : "IEEE John von Neumann Medal", + "year" : 2001, + "by" : "IEEE" + } + ] + } + + { + "_id" : 6, + "name" : { + "first" : "Guido", + "last" : "van Rossum" + }, + "birth" : ISODate("1956-01-31T05:00:00Z"), + "contribs" : [ + "Python" + ], + "awards" : [ + { + "award" : "Award for the Advancement of Free Software", + "year" : 2001, + "by" : "Free Software Foundation" + }, + { + "award" : "NLUUG Award", + "year" : 2003, + "by" : "NLUUG" + } + ] + } + + { + "_id" : 10, + "name" : { + "first" : "Yukihiro", + "aka" : "Matz", + "last" : "Matsumoto" + }, + "birth" : ISODate("1965-04-14T04:00:00Z"), + "contribs" : [ + "Ruby" + ], + "awards" : [ + { + "award" : "Award for the Advancement of Free Software", + "year" : "2011", + "by" : "Free Software Foundation" + } + ] + } + + { + "_id" : 11, + "name" : { + "first" : "James", + "last" : "Gosling" + }, + "birth" : ISODate("1955-05-19T04:00:00Z"), + "contribs" : [ + "Java" + ], + "awards" : [ + { + "award" : "The Economist Innovation Award", + "year" : 2002, + "by" : "The Economist" + }, + { + "award" : "Officer of the Order of Canada", + "year" : 2007, + "by" : "Canada" + } + ] + } + + { + "_id" : 99, + "name" : { + "first" : "Ken", + "last" : "Iverson" + }, + "birth" : ISODate("1941-12-17T05:00:00Z"), + "died" : ISODate("2004-10-19T04:00:00Z"), + "contribs" : [ + "APL", + "J" + ], + "awards" : [ + { + "award" : "Turing Award", + "year" : 1979, + "by" : "ACM" + }, + { + "award" : "Harry H. Goode Memorial Award", + "year" : 1975, + "by" : "IEEE Computer Society" + }, + { + "award" : "IBM Fellow", + "year" : 1970, + "by" : "IBM" + } + ] + } diff --git a/source/reference/method/db.collection.insert.txt b/source/reference/method/db.collection.insert.txt index d65b178b466..9b67f34a2db 100644 --- a/source/reference/method/db.collection.insert.txt +++ b/source/reference/method/db.collection.insert.txt @@ -39,69 +39,321 @@ Examples -------- The following are examples of the :method:`insert() -` method: - -- To insert a single document and have MongoDB generate the unique - ``_id``, omit the ``_id`` field in the document and pass the document - to the :method:`insert() ` method as in the - following: - - .. code-block:: javascript - - db.products.insert( { item: "card", qty: 15 } ) - - This operation inserts a new document into the ``products`` collection - with the ``item`` field set to ``card``, the ``qty`` field set to - ``15``, and the ``_id`` field set to a unique ``ObjectId``: - - .. code-block:: javascript - - { "_id" : ObjectId("5063114bd386d8fadbd6b004"), "item" : "card", "qty" : 15 } - - .. include:: /includes/note-insert-id-field.rst - -- To insert a single document, with a custom ``_id`` field, include the - ``_id`` field set to a unique identifier and pass the document to the - :method:`insert() ` method as follows: - - .. code-block:: javascript - - db.products.insert( { _id: 10, item: "box", qty: 20 } ) - - This operation inserts a new document in the ``products`` collection - with the ``_id`` field set to ``10``, the ``item`` field set to - ``box``, the ``qty`` field set to ``20``: - - .. code-block:: javascript - - { "_id" : 10, "item" : "box", "qty" : 20 } - - .. include:: /includes/note-insert-id-field.rst - -- To insert multiple documents, pass an array of documents to the - :method:`insert() ` method as in the - following: - - .. code-block:: javascript - - db.products.insert( [ { _id: 11, item: "pencil", qty: 50, type: "no.2" }, - { item: "pen", qty: 20 }, - { item: "eraser", qty: 25 } ] ) - - The operation will insert three documents into the ``products`` - collection: - - - A document with the fields ``_id`` set to ``11``, ``item`` set to - ``pencil``, ``qty`` set to ``50``, and the ``type`` set to ``no.2``. - - - A document with the fields ``_id`` set to a unique ``objectid``, - ``item`` set to ``pen``, and ``qty`` set to ``20``. - - - A document with the fields ``_id`` set to a unique ``objectid``, - ``item`` set to ``eraser``, and ``qty`` set to ``25``. - - .. code-block:: javascript - - { "_id" : 11, "item" : "pencil", "qty" : 50, "type" : "no.2" } - { "_id" : ObjectId("50631bc0be4617f17bb159ca"), "item" : "pen", "qty" : 20 } - { "_id" : ObjectId("50631bc0be4617f17bb159cb"), "item" : "eraser", "qty" : 25 } +` method. + +Insert a Document +~~~~~~~~~~~~~~~~~ + +In the following examples, the document passed to the :method:`insert() +` method omits the ``_id`` field, which lets +MongoDB generate a unique ``ObjectId`` for ``_id``: + +The first example creates a document in the ``products`` collection: + +.. code-block:: javascript + + db.products.insert( { item: "card", qty: 15 } ) + +This creates a new document with an ``_id`` field set to a unique +``ObjectId``: + +.. code-block:: javascript + + { "_id" : ObjectId("5063114bd386d8fadbd6b004"), "item" : "card", "qty" : 15 } + +The second example inserts a document in the +:doc:`/reference/bios-example-collection`. If the ``bios`` collection +does not exist, the :method:`~db.collection.insert()` method creates the +collection: + +.. code-block:: javascript + + db.bios.insert( + { + name: { first: 'John', last: 'McCarthy' }, + birth: new Date('Sep 04, 1927'), + death: new Date('Dec 24, 2011'), + contribs: [ 'Lisp', 'Artificial Intelligence', 'ALGOL' ], + awards: [ + { + award: 'Turing Award', + year: 1971, + by: 'ACM' + }, + { + award: 'Kyoto Prize', + year: 1988, + by: 'Inamori Foundation' + }, + { + award: 'National Medal of Science', + year: 1990, + by: 'National Science Foundation' + } + ] + } + ) + +To verify the inserted document, query the ``bios`` collection: + +.. code-block:: javascript + + db.bios.find( { name: { first: 'John', last: 'McCarthy' } } ) + +The returned document contains an ``_id`` field with the generated +``ObjectId`` value: + +.. code-block:: javascript + + { + "_id" : ObjectId("50a1880488d113a4ae94a94a"), + "name" : { "first" : "John", "last" : "McCarthy" }, + "birth" : ISODate("1927-09-04T04:00:00Z"), + "death" : ISODate("2011-12-24T05:00:00Z"), + "contribs" : [ "Lisp", "Artificial Intelligence", "ALGOL" ], + "awards" : [ + { + "award" : "Turing Award", + "year" : 1971, + "by" : "ACM" + }, + { + "award" : "Kyoto Prize", + "year" :1988, + "by" : "Inamori Foundation" + }, + { + "award" : "National Medal of Science", + "year" : 1990, + "by" : "National Science Foundation" + } + ] + } + +.. include:: /includes/note-insert-id-field.rst + +Insert a Document with a Custom ID +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the following examples, the document passed to the :method:`insert() +` method includes a unique value for the ``_id`` +field. The value of ``_id`` must be unique within the collection. + +The first example creates a document with an ``_id`` value of ``10``, +which is unique value for ``_id`` within the ``products`` collection. If +the value were not unique, the insert would fail: + +.. code-block:: javascript + + db.products.insert( { _id: 10, item: "box", qty: 20 } ) + +The insert operation creates the following document: + +.. code-block:: javascript + + { "_id" : 10, "item" : "box", "qty" : 20 } + +The second example inserts a document in the +:doc:`/reference/bios-example-collection` and gives it an ``_id`` value +of ``1``. If the ``bios`` collection does not exist, the +:method:`~db.collection.insert()` method creates the collection +[#show-collection]_: + +.. code-block:: javascript + + db.bios.insert( + { + _id: 1, + name: { first: 'John', last: 'Backus' }, + birth: new Date('Dec 03, 1924'), + death: new Date('Mar 17, 2007'), + contribs: [ 'Fortran', 'ALGOL', 'Backus-Naur Form', 'FP' ], + awards: [ + { + award: 'W.W. McDowell Award', + year: 1967, + by: 'IEEE Computer Society' + }, + { + award: 'National Medal of Science', + year: 1975, + by: 'National Science Foundation' + }, + { + award: 'Turing Award', + year: 1977, + by: 'ACM' + }, + { + award: 'Draper Prize', + year: 1993, + by: 'National Academy of Engineering' + } + ] + } + ) + +To confirm the insert, :doc:`query ` the ``bios`` +collection: + +.. code-block:: javascript + + db.bios.find() + +If the ``bios`` collection contains only the document you just inserted, +the query returns only the one document: + +.. code-block:: javascript + + { + "_id" : 1, + "name" : { "first" : "John", "last" : "Backus" }, + "birth" : ISODate("1924-12-03T05:00:00Z"), + "death" : ISODate("2007-03-17T04:00:00Z"), + "contribs" : [ "Fortran", "ALGOL", "Backus-Naur Form", "FP" ], + "awards" : [ + { + "award" : "W.W. McDowell Award", + "year" : 1967, + "by" : "IEEE Computer Society" + }, + { + "award" : "National Medal of Science", + "year" : 1975, + "by" : "National Science Foundation" + }, + { + "award" : "Turing Award", + "year" : 1977, + "by" : "ACM" + }, + { "award" : "Draper Prize", + "year" : 1993, + "by" : "National Academy of Engineering" + } + ] + } + +.. include:: /includes/note-insert-id-field.rst + +Insert Multiple Documents +~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following examples insert multiple documents by passing an array of +documents to the :method:`insert() ` method. +Such inserts are called *bulk inserts*. + +The first example creates three documents in the ``products`` +collection, one with a custom ``_id``: + +.. code-block:: javascript + + db.products.insert( [ { _id: 11, item: "pencil", qty: 50, type: "no.2" }, + { item: "pen", qty: 20 }, + { item: "eraser", qty: 25 } + ] ) + +The operation creates the following three documents. The ``ObjectId`` +values are specific to the machine and time when the operation is run: + +.. code-block:: javascript + + { "_id" : 11, "item" : "pencil", "qty" : 50, "type" : "no.2" } + { "_id" : ObjectId("51e0373c6f35bd826f47e9a0"), "item" : "pen", "qty" : 20 } + { "_id" : ObjectId("51e0373c6f35bd826f47e9a1"), "item" : "eraser", "qty" : 25 } + +The second example creates three documents in the +:doc:`/reference/bios-example-collection`. The example illustrates the +*dynamic schema* characteristic of MongoDB. The document with ``_id: 3`` +contains a field named ``title`` that does not appear in the other +documents. MongoDB does not require the other documents to contain this +field: + +.. code-block:: javascript + + db.bios.insert( + [ + { + _id: 3, + name: { first: 'Grace', last: 'Hopper' }, + title: 'Rear Admiral', + birth: new Date('Dec 09, 1906'), + death: new Date('Jan 01, 1992'), + contribs: [ 'UNIVAC', 'compiler', 'FLOW-MATIC', 'COBOL' ], + awards: [ + { + award: 'Computer Sciences Man of the Year', + year: 1969, + by: 'Data Processing Management Association' + }, + { + award: 'Distinguished Fellow', + year: 1973, + by: ' British Computer Society' + }, + { + award: 'W. W. McDowell Award', + year: 1976, + by: 'IEEE Computer Society' + }, + { + award: 'National Medal of Technology', + year: 1991, + by: 'United States' + } + ] + }, + { + _id: 4, + name: { first: 'Kristen', last: 'Nygaard' }, + birth: new Date('Aug 27, 1926'), + death: new Date('Aug 10, 2002'), + contribs: [ 'OOP', 'Simula' ], + awards: [ + { + award: 'Rosing Prize', + year: 1999, + by: 'Norwegian Data Association' + }, + { + award: 'Turing Award', + year: 2001, + by: 'ACM' + }, + { + award: 'IEEE John von Neumann Medal', + year: 2001, + by: 'IEEE' + } + ] + }, + { + _id: 5, + name: { first: 'Ole-Johan', last: 'Dahl' }, + birth: new Date('Oct 12, 1931'), + death: new Date('Jun 29, 2002'), + contribs: [ 'OOP', 'Simula' ], + awards: [ + { + award: 'Rosing Prize', + year: 1999, + by: 'Norwegian Data Association' + }, + { + award: 'Turing Award', + year: 2001, + by: 'ACM' + }, + { + award: 'IEEE John von Neumann Medal', + year: 2001, + by: 'IEEE' + } + ] + } + ] + ) + +.. [#show-collection] You can also view a list of the existing + collections in the database using the ``show collections`` + operation in the :program:`mongo` shell. diff --git a/source/reference/method/db.collection.save.txt b/source/reference/method/db.collection.save.txt index 4f2298ecbff..6d194bef428 100644 --- a/source/reference/method/db.collection.save.txt +++ b/source/reference/method/db.collection.save.txt @@ -4,6 +4,9 @@ db.collection.save() .. default-domain:: mongodb +Definition +---------- + .. method:: db.collection.save(document) Updates an existing document or inserts a new document, depending @@ -37,60 +40,130 @@ Examples The following are examples of the :method:`~db.collection.save()` method. -.. TODO we should break these examples up using headings. +.. _crud-create-insert-save: + +Save a New Document with a Generated ID +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following examples each pass the :method:`~db.collection.save()` +method a document without an ``_id`` field so that MongoDB *inserts* a +new document and generates the ``_id``. + +The first example inserts a document with the ``item`` field set to +``book`` and ``qty`` field set to ``40``: + +.. code-block:: javascript + + db.products.save( { item: "book", qty: 40 } ) + +The operation creates a document with an ``_id`` field set to a unique +``ObjectId``: + +.. code-block:: javascript + + { "_id" : ObjectId("50691737d386d8fadbd6b01d"), "item" : "book", "qty" : 40 } + +The second example inserts a new document into the +:doc:`/reference/bios-example-collection`: + +.. code-block:: javascript + + db.bios.save( + { + name: { first: 'Guido', last: 'van Rossum'}, + birth: new Date('Jan 31, 1956'), + contribs: [ 'Python' ], + awards: [ + { + award: 'Award for the Advancement of Free Software', + year: 2001, + by: 'Free Software Foundation' + }, + { + award: 'NLUUG Award', + year: 2003, + by: 'NLUUG' + } + ] + } + ) + +This operation creates a new document with the above fields and with a +unique ``ObjectId``. + +.. include:: /includes/note-insert-id-field.rst + +.. _crud-create-save: + +Save a New Document with a Custom ID +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following examples each pass the :method:`~db.collection.save()` +method a document *with* an ``_id`` field that holds a value that *does +not* exist in the collection. The operations insert new documents. The +results of these operations are identical to an :ref:`update operation +with the upsert flag ` set to ``true`` or ``1``. -.. example:: Pass to the :method:`~db.collection.save()` method - a document without an ``_id`` field, so that to insert the document - into the collection and have MongoDB generate the unique ``_id`` as - in the following: +The first example creates a new document with the ``_id`` field set to +``100``, the ``item`` field set to ``water``, and the ``qty`` field set +to ``30``: - .. code-block:: javascript +.. code-block:: javascript - db.products.save( { item: "book", qty: 40 } ) + db.products.save( { _id: 100, item: "water", qty: 30 } ) - This operation inserts a new document into the ``products`` - collection with the ``item`` field set to ``book``, the ``qty`` - field set to ``40``, and the ``_id`` field set to a unique - ``ObjectId``: +The operation creates the following new document : - .. code-block:: javascript +.. code-block:: javascript - { "_id" : ObjectId("50691737d386d8fadbd6b01d"), "item" : "book", "qty" : 40 } + { "_id" : 100, "item" : "water", "qty" : 30 } - .. include:: /includes/note-insert-id-field.rst +The second example creates a new document in the +:doc:`/reference/bios-example-collection`. The +:method:`~db.collection.save()` method performs an ``upsert`` because +the document to save contains an ``_id`` field with a unique value. No +documents in the collection contain the value: -.. example:: Pass to the :method:`~db.collection.save()` method a - document with an ``_id`` field that holds a value that does not - exist in the collection to insert the document with that value in - the ``_id`` value into the collection, as in the following: +.. code-block:: javascript - .. code-block:: javascript + db.bios.save( + { + _id: 10, + name: { first: 'Yukihiro', aka: 'Matz', last: 'Matsumoto'}, + birth: new Date('Apr 14, 1965'), + contribs: [ 'Ruby' ], + awards: [ + { + award: 'Award for the Advancement of Free Software', + year: '2011', + by: 'Free Software Foundation' + } + ] + } + ) - db.products.save( { _id: 100, item: "water", qty: 30 } ) +.. include:: /includes/note-insert-id-field.rst - This operation inserts a new document into the ``products`` - collection with the ``_id`` field set to ``100``, the ``item`` - field set to ``water``, and the field ``qty`` set to ``30``: +Replace an Existing Document +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - .. code-block:: javascript +This example replaces the data found in the following existing document +in the ``products`` collection: - { "_id" : 100, "item" : "water", "qty" : 30 } +.. code-block:: javascript - .. include:: /includes/note-insert-id-field.rst + { "_id" : 100, "item" : "water", "qty" : 30 } -.. example:: Pass to the :method:`~db.collection.save()` method a - document with the ``_id`` field set to a value in the - ``collection`` to replace all fields and values of the matching - document with the new fields and values, as in the following: +To replace the document's data, pass the :method:`~db.collection.save()` +method a document with new data and with the ``_id`` field set to the +``_id`` value of the existing document: - .. code-block:: javascript +.. code-block:: javascript - db.products.save( { _id:100, item:"juice" } ) + db.products.save( { _id : 100, item : "juice" } ) - This operation replaces the existing document with a value of - ``100`` in the ``_id`` field. The updated document will resemble - the following: +The operation replaces the existing document with the following: - .. code-block:: javascript +.. code-block:: javascript - { "_id" : 100, "item" : "juice" } + { "_id" : 100, "item" : "juice" } diff --git a/source/reference/method/db.collection.update-param.yaml b/source/reference/method/db.collection.update-param.yaml index 849bb2c6e57..e16ad505fc8 100644 --- a/source/reference/method/db.collection.update-param.yaml +++ b/source/reference/method/db.collection.update-param.yaml @@ -25,20 +25,6 @@ description: | The modifications to apply. For details see :ref:`update-parameter` after this table. --- -object: - name: db.collection.update() - type: method -field: - optional: true - type: param -name: options -type: document -position: 3 -description: | - New in version 2.2: Specifies whether to perform an :term:`upsert` - and/or a multiple update. Use the ``options`` parameter instead of the - individual ``upsert`` and ``multi`` parameters. ---- object: name: db.collection.update() type: method @@ -49,14 +35,10 @@ name: upsert type: Boolean position: 4 description: | - Specifies whether to create a new document if no document matches the - query criteria. When ``upsert`` is set to ``true``, the - :method:`~db.collection.update()` method either updates an existing - document, or, if no document matches the criteria, inserts a new - document. When ``upsert`` is set to ``false``, - :method:`~db.collection.update()` does not insert a new document if no - match is found. The default value is ``false``. For additional - information, see :ref:`upsert-parameter`. + If set to ``true``, creates a new document if no document matches the + query criteria. If set to ``false``, :method:`~db.collection.update()` + does not insert a new document if no match is found. The default value + is ``false``. For additional information, see :ref:`upsert-parameter`. --- object: name: db.collection.update() @@ -68,10 +50,8 @@ name: multi type: Boolean position: 5 description: | - Specifies whether to update multiple documents that meet the ``query`` - criteria. When ``multi`` is set to ``true``, the - :method:`~db.collection.update()` method updates all documents that - meet the criteria. When ``multi`` is set to ``false``, the method + If set to ``true``, updates multiple documents that meet the ``query`` + criteria. If set to ``false``, :method:`~db.collection.update()` updates one document. The default value is ``false``. For additional information, see :ref:`multi-parameter`. ... diff --git a/source/reference/method/db.collection.update.txt b/source/reference/method/db.collection.update.txt index b190764c85d..f5400064e1d 100644 --- a/source/reference/method/db.collection.update.txt +++ b/source/reference/method/db.collection.update.txt @@ -9,44 +9,43 @@ Definition .. method:: db.collection.update(query, update, [options]) - Modifies an existing document or documents in a collection. - :method:`~db.collection.update()` takes the following parameters: + Modifies an existing document or documents in a collection. The + :method:`~db.collection.update()` method updates a single document by + default. - .. include:: /reference/method/db.collection.update-param.rst + .. versionchanged:: 2.2 + The :method:`~db.collection.update()` method has the following form: - :method:`~db.collection.update()` updates a single document by - default. To update all documents in the collection that match the - update query criteria, specify the ``multi`` option. To insert a - document if no document matches the update query criteria, specify - the ``upsert`` option. + .. code-block:: javascript - .. versionchanged:: 2.2 - The :program:`mongo` shell provides an updated interface that - accepts the options parameter in a document format to specify - ``multi`` and ``upsert`` options. + db.collection.update( , , { , } ) - Prior to version 2.2, in the :program:`mongo` shell, ``upsert`` and - ``multi`` were positional boolean options: + Prior to version 2.2, the :method:`~db.collection.update()` method + has the following form: .. code-block:: javascript - db.collection.update(query, update, , ) + db.collection.update( , , , ) + + The :method:`~db.collection.update()` method takes the following + parameters: - Although the update operation may apply mostly to updating the values - of the fields, :method:`update() ` can also - modify the name of the ``field`` in a document using the - :operator:`$rename` operator. + .. include:: /reference/method/db.collection.update-param.rst + + The :method:`update() ` operation applies + mostly to updating the values of the fields but can also modify the + name of a ``field`` using the :operator:`$rename` operator. .. _update-parameter: Update Parameter ~~~~~~~~~~~~~~~~ -If the ``update`` parameter contains any :ref:`update operator -` expressions, such as the :operator:`$set` operator -expression, then: +If the ```` parameter contains any :ref:`update operator +` expressions, such as an expression using the +:operator:`$set` operator, then: -- the ``update`` parameter must contain only update operator +- the ```` parameter must contain *only* update operator expressions. .. versionadded:: 2.2 @@ -54,16 +53,17 @@ expression, then: - :method:`~db.collection.update()` updates only the corresponding fields in the document. -If the ``update`` parameter consists only of ``field: value`` +If the ```` parameter consists *only* of ``field: value`` expressions, then: -- :method:`~db.collection.update()` *replaces* the document with the - ``updates`` document. If the ``updates`` document is missing the - :term:`_id` field, MongoDB will add the ``_id`` field and assign to it - a unique :term:`objectid` . +- the :method:`~db.collection.update()` operation *replaces* the + document with the ``update`` document. If the ``update`` document is + missing the :term:`_id` field, MongoDB will add the ``_id`` field and + assign to it a unique :term:`objectid`. For examples, see + :ref:`example-update-replace-fields`. -- :method:`~db.collection.update()` updates cannot update multiple - documents. +- the :method:`~db.collection.update()` operation *cannot* use ``multi`` + to update multiple documents. .. _upsert-parameter: @@ -71,114 +71,245 @@ Upsert Parameter ~~~~~~~~~~~~~~~~ If ``upsert`` is set to ``true`` and no document matches the criteria, -:method:`~db.collection.update()` inserts a new document with the fields -and values of the ``update`` parameter. If the ``update`` included only -update operators, :method:`~db.collection.update()` inserts the -``query`` parameter as well . +:method:`~db.collection.update()` inserts a *single* document. The +``upsert`` creates a new document with either: + +- The fields and values of the ```` parameter, or -In version 2.2 of the :program:`mongo` shell, you may also specify -``upsert`` in the ``options`` parameter. +- the fields and values of the both the ```` and ```` + parameters. -With ``upsert``, :method:`~db.collection.update()` inserts a *single* -document. +The ``upsert`` creates a document with data from both ```` and +```` if the ```` parameter *only uses update operators*. +See :ref:`update-operators`. .. _multi-parameter: Multi Parameter ~~~~~~~~~~~~~~~ -When ``multi`` is set to ``true``, the :method:`~db.collection.update()` -method updates all documents that meet the ``query`` criteria. In -version 2.2 of the :program:`mongo` shell, you may also specify -``multi`` in the ``options`` parameter. - -The ``multi`` update operation may interleave with other write -operations. For unsharded collections, you can override this behavior -with the :operator:`$isolated` isolation operator, which isolates the -update operation and blocks other write operations during the update. +If ``multi`` is set to ``true``, the :method:`~db.collection.update()` +method updates all documents that meet the ```` criteria. The +``multi`` update operation may interleave with other write operations. +For unsharded collections, you can override this behavior with the +:operator:`$isolated` isolation operator, which isolates the update +operation and blocks other write operations during the update. For an +example, see :ref:`example-update-multi`. Examples -------- -The following examples use the 2.2 interface to specify options in the -document form. +The following examples use the **MongoDB version 2.2 interface** to +specify options in the document form. Update Specific Fields in a Document ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Call :method:`~db.collection.update()` with an ``update`` parameter -using ``field: value`` pairs and expressions using :ref:`update -operators ` as in the following: +To update specific fields, call :method:`~db.collection.update()` with +an ```` parameter that uses ``field: value`` pairs in combination +with :ref:`update operators `. For example, given a +``books`` collection with the following document: + +.. code-block:: javascript + + { "_id" : 826, "item" : "Divine Comedy", "stock" : 2 } + +The following operation updates the document by adding a ``price`` and +incrementing ``stock`` by ``5``. + +.. code-block:: javascript + + db.books.update( { item: "Divine Comedy" }, { $set: { price: 18 } , $inc: { stock: 5 } } ) + +The updated document is now the following: .. code-block:: javascript - db.products.update( { item: "book", qty: { $gt: 5 } }, { $set: { x: 6 }, $inc: { y: 5} } ) + { "_id" : 826, "item" : "Divine Comedy", "price" : 18, "stock" : 7 } -This operation updates a document in the ``products`` -collection that matches the query criteria and sets the value of -the field ``x`` to ``6``, and increment the value of the field -``y`` by ``5``. All other fields of the document remain the same. +.. _example-update-replace-fields: Replace All Fields in a Document ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -To replace all the fields in a document with the document as -specified in the ``update`` parameter, call the :method:`update() -` method with an ``update`` parameter -that consists of *only* ``key: value`` expressions, as in the following: +The following examples replace *all* the fields in a document except for +the ``_id`` field. When the ```` argument includes only field +and value pairs, the update replaces all the fields in the existing +document with the fields specified in the ```` argument. To +replace all fields, do not use update operators. + +For example, given the following document in the ``books`` collection: + +.. code-block:: javascript + + { "_id" : 444, "item" : "The Banquet", "author" : "Dante", "price" : 20, "stock" : 4 } + +The following operation passes an ```` parameter that replaces +all the fields in the original document, except for the ``_id`` field. +Notice that both the ```` document and the ```` document both +contain the ``item`` field: .. code-block:: javascript - db.products.update( { item: "book", qty: { $gt: 5 } }, { x: 6, y: 15 } ) + db.books.update( { item: "The Banquet" }, { item: "The Banquet", price: 22 , stock: 3 } ) -This operation selects a document from the ``products`` -collection that matches the query criteria sets the value of the -field ``x`` to ``6`` and the value of the field ``y`` to -``15``. All other fields of the matched document are *removed*, -except the :term:`_id` field. +The operation updates the ``price`` and ``stock`` values and *removes* +the ``auth`` field + +.. code-block:: javascript + + { "_id" : 444, "item" : "The Banquet", "price" : 22, "stock" : 3 } + +In the next example, the :method:`update() +` method includes the ``upsert`` option. The command issues the update +to the :doc:`/reference/bios-example-collection`. + +If a document with ``name`` set to ``{ first: 'Dennis', last: +'Ritchie'}`` exists, MongoDB replaces all the matching document's data, +except the ``_id`` field, with the fields and values in the ```` +parameter. If no document matches the ```` argument, MongoDB +creates a new document with the fields and values specified in the +```` argument. The query does not include an ``_id`` field, so +the operation adds the ``_id`` field and generates a unique ``ObjectId`` +for its value. + +.. code-block:: javascript + + db.bios.update( + { name: { first: 'Dennis', last: 'Ritchie'} }, + { + name: { first: 'Dennis', last: 'Ritchie'}, + birth: new Date('Sep 09, 1941'), + death: new Date('Oct 12, 2011'), + contribs: [ 'UNIX', 'C' ], + awards: [ + { + award: 'Turing Award', + year: 1983, + by: 'ACM' + }, + { + award: 'National Medal of Technology', + year: 1998, + by: 'United States' + }, + { + award: 'Japan Prize', + year: 2011, + by: 'The Japan Prize Foundation' + } + ] + }, + { upsert: true } + ) + +.. _example-update-multi: Update Multiple Documents ~~~~~~~~~~~~~~~~~~~~~~~~~ -Call the :method:`update() ` method and specify -the ``multi`` option, as in the following: +To update multiple documents, call the :method:`update() +` method and specify the ``multi`` option. For +example, given a ``books`` collection with the following documents: .. code-block:: javascript - db.products.update( { item: "book", qty: { $gt: 5 } }, { $set: { x: 6, y: 15 } }, { multi: true } ) + { "_id" : 1, "author" : "Dante", "item" : "Divine Comedy", "price" : 18 } + { "_id" : 2, "author" : "Dante", "item" : "The Banquet", "price" : 22 } + { "_id" : 3, "author" : "Dante", "item" : "Monarchia", "price" : 14 } + +The following operation uses the ``multi`` option and a ``query`` for +``author: "Dante"`` to update all matching documents: + +.. code-block:: javascript -This operation updates *all* documents in the ``products`` -collection that match the query criteria by setting the value of -the field ``x`` to ``6`` and the value of the field ``y`` to -``15``. This operation does not affect any other fields in -documents in the ``products`` collection. + db.books.update( { author: "Dante" }, { $set: { born: "Florence", died: "Ravenna" } }, { multi: true } ) -You can perform the same operation by calling the -:method:`~db.collection.update()` method with the ``multi`` -parameter: +This operation does not affect any other fields in documents in the +``books`` collection: .. code-block:: javascript - db.products.update( { item: "book", qty: { $gt: 5 } }, { $set: { x: 6, y: 15 } }, false, true ) + { "_id" : 1, "author" : "Dante", "born" : "Florence", "died" : "Ravenna", "item" : "Divine Comedy", "price" : 18 } + { "_id" : 2, "author" : "Dante", "born" : "Florence", "died" : "Ravenna", "item" : "The Banquet", "price" : 22 } + { "_id" : 3, "author" : "Dante", "born" : "Florence", "died" : "Ravenna", "item" : "Monarchia", "price" : 14 } -Upsert -~~~~~~ +Insert a New Document (Upsert) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -To update a document or to insert a new document if no document matches -the query criteria, call the :method:`~db.collection.update()` and -specify the ``upsert`` option, as in the following: +To insert a new document if no document matches the query criteria, call +the :method:`~db.collection.update()` and specify the ``upsert`` option. +The following examples use the ``upsert`` option: + +For example, given the following document: + +.. code-block:: javascript + + db.books.update( { "item": "The New Life" }, { $set: { author: "Dante", "price": 15 } }, { upsert: true } ) + +If one or more documents with ``item: "The New Life"`` exists, this +operation updates *one* of those documents with the values in the +``author`` and ``price`` fields. If the fields do not exist, the +operation adds them. + +If a document with ``item: "New Life"`` does not exist, this operation +creates a new document with an ``_id`` field set to a unique +``ObjectId``: .. code-block:: javascript - db.products.update( { item: "magazine", qty: { $gt: 5 } }, { $set: { x: 25, y: 50 } }, { upsert: true } ) + { "_id" : ObjectId("51e0570a9c6ae665454e301c"), "author" : "Dante", "item" : "New Life", "price" : 15 } + +In the next example, no document matches the ```` argument, +so the ``upsert`` performs an insert. The example inserts a new document +into the :doc:`/reference/bios-example-collection`. + +The ```` argument includes only :ref:`update operators +`, so the new document contains the fields and values +from ```` argument with the operations from the ```` +argument applied. + +.. code-block:: javascript -This operation will: + db.bios.update( + { + _id: 7, + name: { first: 'Ken', last: 'Thompson' } + }, + { + $set: { + birth: new Date('Feb 04, 1943'), + contribs: [ 'UNIX', 'C', 'B', 'UTF-8' ], + awards: [ + { + award: 'Turing Award', + year: 1983, + by: 'ACM' + }, + { + award: 'IEEE Richard W. Hamming Medal', + year: 1990, + by: 'IEEE' + }, + { + award: 'National Medal of Technology', + year: 1998, + by: 'United States' + }, + { + award: 'Tsutomu Kanai Award', + year: 1999, + by: 'IEEE' + }, + { + award: 'Japan Prize', + year: 2011, + by: 'The Japan Prize Foundation' + } + ] + } + }, + { upsert: true } + ) -- update a single document in the ``products`` collection that - matches the query criteria, setting the value of the field - ``x`` to ``25`` and the value of the field ``y`` to ``50``, *or* -- if no matching document exists, insert a document in the - ``products`` collection, with the field ``item`` set to - ``magazine``, the field ``x`` set to ``25``, and the field ``y`` - set to ``50``. From 6d3caf147d6a391697dc231da6ccc8d676b21543 Mon Sep 17 00:00:00 2001 From: Sam Kleinman Date: Fri, 12 Jul 2013 16:48:52 -0400 Subject: [PATCH 09/14] crud-reorg: migrate document pages ord.txt --- source/core/document-type-operators.txt | 36 --------- source/core/document-type-record.txt | 95 ----------------------- source/core/document.txt | 54 ++++++++++++- source/core/shell-types.txt | 45 ++++++++++- source/reference.txt | 1 + source/{core => reference}/bson-types.txt | 0 6 files changed, 93 insertions(+), 138 deletions(-) delete mode 100644 source/core/document-type-operators.txt delete mode 100644 source/core/document-type-record.txt rename source/{core => reference}/bson-types.txt (100%) diff --git a/source/core/document-type-operators.txt b/source/core/document-type-operators.txt deleted file mode 100644 index c4d95df6008..00000000000 --- a/source/core/document-type-operators.txt +++ /dev/null @@ -1,36 +0,0 @@ -==================== -Determine Field Type -==================== - -.. default-domain:: mongodb - -To determine the type of fields, the :program:`mongo` shell provides -the following operators: - -- ``instanceof`` returns a boolean to test if a value has - a specific type. - -- ``typeof`` returns the type of a field. - -.. example:: - - Consider the following operations using ``instanceof`` and - ``typeof``: - - - The following operation tests whether the ``_id`` field is of type - ``ObjectId``: - - .. code-block:: javascript - - mydoc._id instanceof ObjectId - - The operation returns ``true``. - - - The following operation returns the type of the ``_id`` field: - - .. code-block:: javascript - - typeof mydoc._id - - In this case ``typeof`` will return the more generic ``object`` - type rather than ``ObjectId`` type. diff --git a/source/core/document-type-record.txt b/source/core/document-type-record.txt deleted file mode 100644 index 100234fffc5..00000000000 --- a/source/core/document-type-record.txt +++ /dev/null @@ -1,95 +0,0 @@ -.. _documents-records: - -================ -Record Documents -================ - -.. default-domain:: mongodb - -Record documents store the database's data. Each record document is -equivalent to a row in an :term:`RDBMS`. Record documents are grouped -into :term:`collections `, which are the equivalent of RDBMS -tables. - -The following is an example record document: - - .. code-block:: javascript - - { - _id: 1, - name: { first: 'John', last: 'Backus' }, - birth: new Date('Dec 03, 1924'), - death: new Date('Mar 17, 2007'), - contribs: [ 'Fortran', 'ALGOL', 'Backus-Naur Form', 'FP' ], - awards: [ - { award: 'National Medal of Science', - year: 1975, - by: 'National Science Foundation' }, - { award: 'Turing Award', - year: 1977, - by: 'ACM' } - ] - } - -The document contains the following fields: - -- ``_id``, which must hold a unique value and is *immutable*. - -- ``name`` that holds another *document*. This sub-document contains - the fields ``first`` and ``last``, which both hold *strings*. - -- ``birth`` and ``death`` that both have *date* types. - -- ``contribs`` that holds an *array of strings*. - -- ``awards`` that holds an *array of documents*. - -Attributes of Record Documents ------------------------------- - -Record documents have the following attributes: - -- .. include:: /includes/fact-document-max-size.rst - -- .. include:: /includes/fact-document-field-name-restrictions.rst - -.. include:: /includes/note-insert-id-field.rst - -The ``_id`` Field ------------------ - -The ``_id`` field has the following behavior and constraints: - -- In documents, the ``_id`` field is always indexed for regular - collections. - -- The ``_id`` field may contain values of any :doc:`BSON data type - `, other than an array. - - .. warning:: To ensure functioning replication, do not store values - that are of the BSON regular expression type in the ``_id`` - field. - - .. See :issue:`SERVER-9562` for more information. - -The following are common options for storing values for ``_id``: - -- Use an :doc:`ObjectId `. - -- Use a natural unique identifier, if available. This saves space and - avoids an additional index. - -- Generate an auto-incrementing number. See - :doc:`/tutorial/create-an-auto-incrementing-field`. - -- Generate a UUID in your application code. For a more efficient - storage of the UUID values in the collection and in the ``_id`` - index, store the UUID as a value of the BSON ``BinData`` type. - - .. include:: /includes/fact-bindata-storage-optimization.rst - -- Use your driver's BSON UUID facility to generate UUIDs. Be aware - that driver implementations may implement UUID serialization and - deserialization logic differently, which may not be fully compatible - with other drivers. See your :api:`driver documentation <>` for - information concerning UUID interoperability. diff --git a/source/core/document.txt b/source/core/document.txt index 9d0b70ab725..d63b3b211fd 100644 --- a/source/core/document.txt +++ b/source/core/document.txt @@ -46,6 +46,7 @@ The :program:`mongo` JavaScript shell and the :doc:`MongoDB language drivers ` translate between BSON and the language-specific document representation. + Document Structure ------------------ @@ -110,6 +111,56 @@ existing user document. .. _document-dot-notation: +Document Limitations +-------------------- + +Record documents have the following attributes: + +- .. include:: /includes/fact-document-max-size.rst + +- .. include:: /includes/fact-document-field-name-restrictions.rst + +The ``_id`` Field +----------------- + +The ``_id`` field has the following behavior and constraints: + +- In documents, the ``_id`` field is always indexed for regular + collections. + +- The ``_id`` field may contain values of any :doc:`BSON data type + `, other than an array. + + .. warning:: To ensure functioning replication, do not store values + that are of the BSON regular expression type in the ``_id`` + field. + + .. See :issue:`SERVER-9562` for more information. + +The following are common options for storing values for ``_id``: + +- Use an :doc:`ObjectId `. + +- Use a natural unique identifier, if available. This saves space and + avoids an additional index. + +- Generate an auto-incrementing number. See + :doc:`/tutorial/create-an-auto-incrementing-field`. + +- Generate a UUID in your application code. For a more efficient + storage of the UUID values in the collection and in the ``_id`` + index, store the UUID as a value of the BSON ``BinData`` type. + + .. include:: /includes/fact-bindata-storage-optimization.rst + +- Use your driver's BSON UUID facility to generate UUIDs. Be aware + that driver implementations may implement UUID serialization and + deserialization logic differently, which may not be fully compatible + with other drivers. See your :api:`driver documentation <>` for + information concerning UUID interoperability. + +.. include:: /includes/note-insert-id-field.rst + Dot Notation ------------ @@ -128,10 +179,7 @@ Dot Notation .. toctree:: :titlesonly: - /core/document-type-record /core/document-type-query-specification /core/document-type-update-specification /core/document-type-index-specification /core/document-type-sort-order - /core/bson-types - /core/document-type-operators diff --git a/source/core/shell-types.txt b/source/core/shell-types.txt index 83992d4be29..7b807b169b4 100644 --- a/source/core/shell-types.txt +++ b/source/core/shell-types.txt @@ -14,8 +14,11 @@ information. .. _mongo-shell-data-type: +Types +----- + Date ----- +~~~~ The :program:`mongo` shell provides various options to return the date, either as a string or as an object: @@ -123,7 +126,7 @@ Consider the following examples: The operation returns ``object``. ObjectId --------- +~~~~~~~~ The :program:`mongo` shell provides the ``ObjectId()`` wrapper class around :term:`ObjectId` data types. To generate a new ObjectId, use @@ -139,7 +142,7 @@ the following operation in the :program:`mongo` shell: .. _shell-type-long: NumberLong ----------- +~~~~~~~~~~ By default, the :program:`mongo` shell treats all numbers as floating-point values. The :program:`mongo` shell provides the @@ -203,9 +206,43 @@ to a floating point value, as in the following example: .. _shell-type-int: NumberInt ----------- +~~~~~~~~~ By default, the :program:`mongo` shell treats all numbers as floating-point values. The :program:`mongo` shell provides the ``NumberInt()`` constructor to explicitly specify 32-bit integers. +Check Types in the ``mongo`` Shell +---------------------------------- + +To determine the type of fields, the :program:`mongo` shell provides +the following operators: + +- ``instanceof`` returns a boolean to test if a value has + a specific type. + +- ``typeof`` returns the type of a field. + +.. example:: + + Consider the following operations using ``instanceof`` and + ``typeof``: + + - The following operation tests whether the ``_id`` field is of type + ``ObjectId``: + + .. code-block:: javascript + + mydoc._id instanceof ObjectId + + The operation returns ``true``. + + - The following operation returns the type of the ``_id`` field: + + .. code-block:: javascript + + typeof mydoc._id + + In this case ``typeof`` will return the more generic ``object`` + type rather than ``ObjectId`` type. + diff --git a/source/reference.txt b/source/reference.txt index f6b8a3f7477..001b65216a4 100644 --- a/source/reference.txt +++ b/source/reference.txt @@ -54,6 +54,7 @@ General Reference reference/limits reference/connection-string reference/mongodb-extended-json + reference/bson-types reference/glossary .. seealso:: The :ref:`genindex` may provide useful insight into the diff --git a/source/core/bson-types.txt b/source/reference/bson-types.txt similarity index 100% rename from source/core/bson-types.txt rename to source/reference/bson-types.txt From 42b128bed57c7f8deaef219f873e11b8a8894a27 Mon Sep 17 00:00:00 2001 From: Sam Kleinman Date: Fri, 12 Jul 2013 16:57:55 -0400 Subject: [PATCH 10/14] crud-reorg: cross reference fixes --- source/core/document.txt | 4 ++-- source/core/shell-types.txt | 2 ++ source/reference/bson-types.txt | 2 +- source/reference/glossary.txt | 4 ++-- 4 files changed, 7 insertions(+), 5 deletions(-) diff --git a/source/core/document.txt b/source/core/document.txt index d63b3b211fd..515d391e180 100644 --- a/source/core/document.txt +++ b/source/core/document.txt @@ -40,7 +40,7 @@ MongoDB documents are stored on disk in the :term:`BSON` serialization format. BSON is a binary representation of :term:`JSON` documents, though contains more data types than does JSON. For the BSON spec, see `bsonspec.org `_. See also -:doc:`/core/bson-types`. +:doc:`/reference/bson-types`. The :program:`mongo` JavaScript shell and the :doc:`MongoDB language drivers ` translate between BSON and the @@ -64,7 +64,7 @@ following structure: } The value of a field can be any of the BSON :doc:`data types -`, including other documents, arrays, and arrays of +`, including other documents, arrays, and arrays of documents. The following document contains values of varying types: .. code-block:: javascript diff --git a/source/core/shell-types.txt b/source/core/shell-types.txt index 7b807b169b4..489fe210895 100644 --- a/source/core/shell-types.txt +++ b/source/core/shell-types.txt @@ -212,6 +212,8 @@ By default, the :program:`mongo` shell treats all numbers as floating-point values. The :program:`mongo` shell provides the ``NumberInt()`` constructor to explicitly specify 32-bit integers. +.. _check-types-in-shell: + Check Types in the ``mongo`` Shell ---------------------------------- diff --git a/source/reference/bson-types.txt b/source/reference/bson-types.txt index 10a3484060f..9f239b4182c 100644 --- a/source/reference/bson-types.txt +++ b/source/reference/bson-types.txt @@ -39,7 +39,7 @@ Min key 255 Max key 127 ======================= ========== -To determine a field's type, see :doc:`/core/document-type-operators`. +To determine a field's type, see :ref:`check-types-in-shell`. If you convert BSON to JSON, see :ref:`bson-json-type-conversion-fidelity` for more information. diff --git a/source/reference/glossary.txt b/source/reference/glossary.txt index d90f938a0f3..fa01e569427 100644 --- a/source/reference/glossary.txt +++ b/source/reference/glossary.txt @@ -14,7 +14,7 @@ Glossary of JSON (JavaScript Object Notation) documents. For a detailed spec, see `bsonspec.org `_. - .. seealso:: :doc:`/core/document`, :doc:`/core/bson-types` and + .. seealso:: :doc:`/core/document`, :doc:`/reference/bson-types` and :ref:`bson-json-type-conversion-fidelity` database command @@ -179,7 +179,7 @@ Glossary BSON types The set of types supported by the :term:`BSON` serialization - format. For a list of BSON types, see :doc:`/core/bson-types`. + format. For a list of BSON types, see :doc:`/reference/bson-types`. master In conventional master/:term:`slave` replication, the master From 4cee0508ec9ebdaf690dacbc2e388a1d0bf9483c Mon Sep 17 00:00:00 2001 From: Sam Kleinman Date: Fri, 12 Jul 2013 17:06:57 -0400 Subject: [PATCH 11/14] crud-reorg: fixing link and fixing example document --- source/core/document.txt | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/source/core/document.txt b/source/core/document.txt index 515d391e180..d70e50da059 100644 --- a/source/core/document.txt +++ b/source/core/document.txt @@ -14,24 +14,20 @@ structures composed of field-and-value pairs: Most user-accessible data structures in MongoDB are documents, including: -- :ref:`Records `. MongoDB stores records in - :term:`collections ` +- All database records. -- :ref:`Query selectors `, which define what - records to select for read, update, and delete operations. +- :doc:`Query selectors `, which define what records to + select for read, update, and delete operations. -- :ref:`Update actions `, which define what - fields to modify during an update. +- :doc:`Update definitions `, which define what fields + to modify during an update. -- :ref:`Index specifications `, which define what +- :doc:`Index specifications `, which define what fields to index. -- Arguments to many MongoDB methods and operators, such as the - :ref:`sort order document ` used by the - :method:`sort() ` method. - -- Outputted data, such as the output of the :dbcommand:`collStats` - command. +- Data output by MongoDB for reporting and configuration, such as the + output of the :dbcommand:`serverStatus` and the :ref:`replica set + configuration document `. Document Format --------------- @@ -129,7 +125,7 @@ The ``_id`` field has the following behavior and constraints: collections. - The ``_id`` field may contain values of any :doc:`BSON data type - `, other than an array. + `, other than an array. .. warning:: To ensure functioning replication, do not store values that are of the BSON regular expression type in the ``_id`` From a746075883c536388bbda1140320e3136c814e75 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Fri, 12 Jul 2013 17:05:34 -0400 Subject: [PATCH 12/14] crud reorg: remove 'document-type' files, move 'sort' info --- .../document-type-index-specification.txt | 45 ------------- .../document-type-query-specification.txt | 40 ------------ source/core/document-type-sort-order.txt | 40 ------------ .../document-type-update-specification.txt | 64 ------------------- source/reference/method/cursor.sort.txt | 26 ++++++-- 5 files changed, 21 insertions(+), 194 deletions(-) delete mode 100644 source/core/document-type-index-specification.txt delete mode 100644 source/core/document-type-query-specification.txt delete mode 100644 source/core/document-type-sort-order.txt delete mode 100644 source/core/document-type-update-specification.txt diff --git a/source/core/document-type-index-specification.txt b/source/core/document-type-index-specification.txt deleted file mode 100644 index 838d4ccea20..00000000000 --- a/source/core/document-type-index-specification.txt +++ /dev/null @@ -1,45 +0,0 @@ -.. _documents-index: -.. _document-index-specification: - -============================= -Index Specification Documents -============================= - -.. default-domain:: mongodb - -Index specification documents describe the fields to index on during -the :doc:`index creation -`. See :doc:`indexes -` for an overview of indexes. [#index-def]_ - -Index documents contain field and value pairs, in the following form: - -.. code-block:: javascript - - { field: value } - -- ``field`` is the field in the documents to index. - -- ``value`` is either 1 for ascending or -1 for descending. - -The following document specifies the :ref:`multi-key index -` on the ``_id`` field and the ``last`` field -contained in the subdocument ``name`` field. The document uses -:ref:`dot notation ` to access a field in a -subdocument: - -.. code-block:: javascript - - { _id: 1, 'name.last': 1 } - -When passed as an argument to the :method:`ensureIndex() -` method, the index documents specifies -the index to create: - -.. code-block:: javascript - - db.bios.ensureIndex( { _id: 1, 'name.last': 1 } ) - -.. [#index-def] Indexes optimize a number of key :doc:`read - ` and :doc:`write ` - operations. diff --git a/source/core/document-type-query-specification.txt b/source/core/document-type-query-specification.txt deleted file mode 100644 index 524a699586f..00000000000 --- a/source/core/document-type-query-specification.txt +++ /dev/null @@ -1,40 +0,0 @@ -.. _documents-query-selectors: -.. _mongodb-query-document: - -============================= -Query Specification Documents -============================= - -.. default-domain:: mongodb - -Query documents specify the conditions that determine which records to -select for read, update, and delete operations. You can use -``:`` expressions to specify the equality condition and -:doc:`query operator ` expressions to specify -additional conditions. - -When passed as an argument to methods such as the :method:`find() -` method, the :method:`remove() -` method, or the :method:`update() -` method, the query document selects documents -for MongoDB to return, remove, or update, as in the following: - -.. code-block:: javascript - - db.bios.find( { _id: 1 } ) - db.bios.remove( { _id: { $gt: 3 } } ) - db.bios.update( { _id: 1, name: { first: 'John', last: 'Backus' } }, - , - ) - -.. seealso:: - - - :ref:`read-operations-query-argument` and - :doc:`/core/read` for more examples on selecting documents - for reads. - - - :doc:`/core/update` for more examples on - selecting documents for updates. - - - :doc:`/core/delete` for more examples on selecting - documents for deletes. diff --git a/source/core/document-type-sort-order.txt b/source/core/document-type-sort-order.txt deleted file mode 100644 index d5eee90f6f1..00000000000 --- a/source/core/document-type-sort-order.txt +++ /dev/null @@ -1,40 +0,0 @@ -.. _documents-sort-order: - -================================== -Sort Order Specification Documents -================================== - -.. default-domain:: mongodb - -Sort order documents specify the order of documents that a -:method:`query() ` returns. Pass sort order -specification documents as an argument to the :method:`sort() -` method. See the :method:`sort() ` page -for more information on sorting. - -The sort order documents contain field and value pairs, in the following -form: - -.. code-block:: javascript - - { field: value } - -- ``field`` is the field by which to sort documents. - -- ``value`` is either 1 for ascending or -1 for descending. - -The following document specifies the sort order using the fields from a -sub-document ``name`` first sort by the ``last`` field ascending, then -by the ``first`` field also ascending: - -.. code-block:: javascript - - { 'name.last': 1, 'name.first': 1 } - -When passed as an argument to the :method:`sort() ` -method, the sort order document sorts the results of the -:method:`find() ` method: - -.. code-block:: javascript - - db.bios.find().sort( { 'name.last': 1, 'name.first': 1 } ) diff --git a/source/core/document-type-update-specification.txt b/source/core/document-type-update-specification.txt deleted file mode 100644 index faf5cb266a2..00000000000 --- a/source/core/document-type-update-specification.txt +++ /dev/null @@ -1,64 +0,0 @@ -.. _documents-update-actions: - -============================== -Update Specification Documents -============================== - -.. default-domain:: mongodb - -Update documents specify the data modifications to perform during -an :method:`update() ` operation to modify -existing records in a collection. You can use :ref:`update operators -` to specify the exact actions to perform on the -document fields. - -The following is an example update document: - -.. code-block:: javascript - - { - $set: { 'name.middle': 'Warner' }, - $push: { awards: { award: 'IBM Fellow', - year: '1963', - by: 'IBM' } - } - } - -When passed as an argument to the :method:`update() -` method, the update actions document: - -- Modifies the field ``name`` whose value is another document. - Specifically, the :operator:`$set` operator updates the ``middle`` - field in the ``name`` subdocument. The document uses :ref:`dot - notation ` to access a field in a subdocument. - -- Adds an element to the field ``awards``, whose value is an array. - Specifically, the :operator:`$push` operator adds another document as - element to the field ``awards``. - -.. code-block:: javascript - - db.bios.update( - { _id: 1 }, - { - $set: { 'name.middle': 'Warner' }, - $push: { awards: { - award: 'IBM Fellow', - year: '1963', - by: 'IBM' - } - } - } - ) - -.. seealso:: - - - :ref:`update operators ` page for the available - update operators and syntax. - - - :doc:`update ` for more examples on - update documents. - -For additional examples of updates that involve array elements, -including where the elements are documents, see the :operator:`$` -positional operator. diff --git a/source/reference/method/cursor.sort.txt b/source/reference/method/cursor.sort.txt index ccbabe2c970..c925837b87c 100644 --- a/source/reference/method/cursor.sort.txt +++ b/source/reference/method/cursor.sort.txt @@ -20,6 +20,17 @@ Definition .. include:: /reference/method/cursor.sort-param.rst + The ``sort`` parameter contains field and value pairs, in the + following form: + + .. code-block:: javascript + + { field: value } + + - ``field`` is the field by which to sort documents. + + - ``value`` is either 1 for ascending or -1 for descending. + .. note:: You must apply :method:`cursor.limit()` to the cursor before @@ -27,8 +38,8 @@ Definition .. todo: verify "must," seems a bit strong. -Example -------- +Examples +-------- The following query returns all documents in ``collection`` sorted by the ``age`` field in descending order. @@ -37,9 +48,14 @@ the ``age`` field in descending order. db.collection.find().sort( { age: -1 } ); -Specify a value of negative -one (e.g. ``-1``), as above, to sort in descending order or a -positive value (e.g. ``1``) to sort in ascending order. +The following query specifies the sort order using the fields from a +sub-document ``name``. The query sorts first by the ``last`` field and +then by the ``first`` field. The query sorts both fields in ascending +order: + +.. code-block:: javascript + + db.bios.find().sort( { 'name.last': 1, 'name.first': 1 } ) Limit Results ------------- From ec386834003dd4bc66606bc62a265f8a6dd36c80 Mon Sep 17 00:00:00 2001 From: Sam Kleinman Date: Fri, 12 Jul 2013 17:24:04 -0400 Subject: [PATCH 13/14] crud-reorg: document example cleanup --- source/core/document.txt | 10 ------- .../reference/method/db.collection.update.txt | 30 +++++++++++++++++++ 2 files changed, 30 insertions(+), 10 deletions(-) diff --git a/source/core/document.txt b/source/core/document.txt index d70e50da059..a031cfcfc65 100644 --- a/source/core/document.txt +++ b/source/core/document.txt @@ -169,13 +169,3 @@ Dot Notation - :ref:`read-operations-arrays` for dot notation examples with arrays. - -.. class:: hidden - - .. toctree:: - :titlesonly: - - /core/document-type-query-specification - /core/document-type-update-specification - /core/document-type-index-specification - /core/document-type-sort-order diff --git a/source/reference/method/db.collection.update.txt b/source/reference/method/db.collection.update.txt index f5400064e1d..e5d99afaf02 100644 --- a/source/reference/method/db.collection.update.txt +++ b/source/reference/method/db.collection.update.txt @@ -313,3 +313,33 @@ argument applied. ) +Update with ``$set`` and ``$push`` Operators +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Consider the following operation: + +.. code-block:: javascript + + db.bios.update( + { _id: 1 }, + { + $set: { 'name.middle': 'Warner' }, + $push: { awards: { + award: 'IBM Fellow', + year: '1963', + by: 'IBM' + } + } + } + ) + +This :method:`~db.collection.update()` operation: + +- Modifies the field ``name`` whose value is another document. + Specifically, the :operator:`$set` operator updates the ``middle`` + field in the ``name`` subdocument. The document uses :ref:`dot + notation ` to access a field in a subdocument. + +- Adds an element to the field ``awards``, whose value is an array. + Specifically, the :operator:`$push` operator adds another document as + element to the field ``awards``. From e803f8ceb6d9d822c573dff8029c5668d8e08368 Mon Sep 17 00:00:00 2001 From: Bob Grabar Date: Mon, 15 Jul 2013 11:39:16 -0400 Subject: [PATCH 14/14] fix links to 'query document' --- source/reference/command/text.txt | 4 ++-- source/reference/operator/positional.txt | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/source/reference/command/text.txt b/source/reference/command/text.txt index 34030aeb38f..a47121c92bf 100644 --- a/source/reference/command/text.txt +++ b/source/reference/command/text.txt @@ -57,7 +57,7 @@ Definition :param document filter: - Optional. A :ref:`query document ` to + Optional. A :ref:`query document ` to further limit the results of the query using another database field. You can use any valid MongoDB query in the filter document, except if the index includes an ascending or @@ -188,7 +188,7 @@ Definition } ) - - The ``filter`` :ref:`query document ` + - The ``filter`` :ref:`query document ` may use any of the available :doc:`query operators `. diff --git a/source/reference/operator/positional.txt b/source/reference/operator/positional.txt index f9df37ed316..c62f8d420e4 100644 --- a/source/reference/operator/positional.txt +++ b/source/reference/operator/positional.txt @@ -19,7 +19,7 @@ - the positional :operator:`$` operator acts as a placeholder for the **first** element that matches the :ref:`query document - `, and + `, and - the ``array`` field **must** appear as part of the ``query document``. @@ -46,7 +46,7 @@ Remember that the positional :operator:`$` operator acts as a placeholder for the **first match** of the update :ref:`query - document `. + document `. The positional :operator:`$` operator facilitates updates to arrays that contain embedded documents. Use the positional :operator:`$`