DOCS-1111 clarify sort operations with index

source/tutorial/sort-results-with-indexes.txt

@@ -7,54 +7,178 @@ Use Indexes to Sort Query Results
 
 .. default-domain:: mongodb
 
-For the fastest performance when sorting query results by a given field,
-create a sorted index on that field.
+For the fastest performance when sorting query results by a given
+field, create a sorted index on that field. In MongoDB, as in other
+databases, indexes are sorted, and access to the indexes returns the
+results in that order.
 
-To sort query results on multiple fields, create a :ref:`compound
-index <index-type-compound>`. MongoDB sorts results based on the field
-order in the index. For queries that include a sort that uses a
-compound index, ensure that all fields before the first sorted field
-are equality matches.
+To sort on multiple fields, create a :ref:`compound index
+<index-type-compound>`. With compound indexes, the results can be in
+the sorted order of either the full index or an index prefix. An index
+prefix is a subset of a compound index; the subset consists of one or
+more fields at the start of the index, in order. For example, given an
+index ``{ a:1, b: 1, c: 1, d: 1 }``, the following subsets are index
+prefixes:
 
-.. example::
+.. code-block:: javascript
 
-   If you create the following index:
+   { a: 1 }
+   { a: 1, b: 1 }
+   { a: 1, b: 1, c: 1 }
 
-   .. code-block:: javascript
+For more information on sorting by index prefixes, see
+:ref:`sort-index-prefix`.
 
-      { a: 1, b: 1, c: 1, d: 1 }
+If the query includes **equality** match conditions on an index prefix,
+you can sort on a subset of the index that starts after or overlaps with
+the prefix. For example, given an index ``{ a: 1, b: 1, c: 1, d: 1 }``,
+if the query condition includes equality match conditions on ``a`` and
+``b``, you can specify a sort on the subsets ``{ c: 1 }`` or ``{ c: 1,
+d: 1 }``:
 
-   The following query and sort operations can use the index:
+.. code-block:: javascript
 
-   .. code-block:: javascript
+   db.collection.find( { a: 5, b: 3 } ).sort( { c: 1 } )
+   db.collection.find( { a: 5, b: 3 } ).sort( { c: 1, d: 1 } )
 
-      db.collection.find().sort( { a:1 } )
-      db.collection.find().sort( { a:1, b:1 } )
+In these operations, the equality match and the sort documents together
+cover the index prefixes ``{ a: 1, b: 1, c: 1 }`` and ``{ a: 1, b: 1,
+c: 1, d: 1 }`` respectively.
 
-      db.collection.find( { a:4 } ).sort( { a:1, b:1 } )
-      db.collection.find( { b:5 } ).sort( { a:1, b:1 } )
+You can also specify a sort order that includes the prefix; however,
+since the query condition specifies equality matches on these fields,
+they are constant in the resulting documents and do not contribute to
+the sort order:
 
-      db.collection.find( { a:5 } ).sort( { b:1, c:1 } )
+.. code-block:: javascript
 
-      db.collection.find( { a:5, c:4, b:3 } ).sort( { d:1 } )
+   db.collection.find( { a: 5, b: 3 } ).sort( { a: 1, b: 1, c: 1 } )
+   db.collection.find( { a: 5, b: 3 } ).sort( { a: 1, b: 1, c: 1, d: 1 } )
 
-      db.collection.find( { a: { $gt:4 } } ).sort( { a:1, b:1 } )
-      db.collection.find( { a: { $gt:5 } } ).sort( { a:1, b:1 } )
-
-      db.collection.find( { a:5, b:3, d:{ $gt:4 } } ).sort( { c:1 } )
-      db.collection.find( { a:5, b:3, c:{ $lt:2 }, d:{ $gt:4 } } ).sort( { c:1 } )
-
-   However, the following queries cannot sort the results using the
-   index:
-
-   .. code-block:: javascript
-
-      db.collection.find().sort( { b:1 } )
-      db.collection.find( { b:5 } ).sort( { b:1 } )
+For more information on sorting by index subsets that are not prefixes,
+see :ref:`sort-equality-match`.
 
 .. note::
 
    For in-memory sorts that do not use an index, the :method:`sort()
    <cursor.sort()>` operation is significantly slower. The
    :method:`~cursor.sort()` operation will abort when it uses 32
    megabytes of memory.
+
+Sort With a Subset of Compound Index
+------------------------------------
+
+If the sort document contains a subset of the compound index fields,
+the subset can determine whether MongoDB can use the index efficiently
+to both retrieve and sort the query results. If MongoDB can efficiently
+use the index to both retrieve and sort the query results, the output
+from the :method:`~cursor.explain()` will display
+:data:`~explain.scanAndOrder` as ``false`` or ``0``. If MongoDB can
+only use the index for retrieving documents that meet the query
+criteria, MongoDB must manually sort the resulting documents without
+the use of the index. For in-memory sort operations,
+:method:`~cursor.explain()` will display :data:`~explain.scanAndOrder`
+as ``true`` or ``1``.
+
+.. _sort-index-prefix:
+
+Sort Subset Starts at the Index Beginning
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the sort document is a subset of a compound index and starts from
+the beginning of the index, MongoDB can use the index to both retrieve
+and sort the query results.
+
+For example, the collection ``collection`` has the following index:
+
+.. code-block:: javascript
+
+   { a: 1, b: 1, c: 1, d: 1 }
+
+The following operations include a sort with a subset of the index.
+Because the sort subset starts at beginning of the index, the
+operations can use the index for both the query retrieval and sort:
+
+.. code-block:: javascript
+
+   db.collection.find().sort( { a:1 } )
+   db.collection.find().sort( { a:1, b:1 } )
+   db.collection.find().sort( { a:1, b:1, c:1 } )
+
+   db.collection.find( { a: 4 } ).sort( { a: 1, b: 1 } )
+   db.collection.find( { a: { $gt: 4 } } ).sort( { a: 1, b: 1 } )
+
+   db.collection.find( { b: 5 } ).sort( { a: 1, b: 1 } )
+   db.collection.find( { b: { $gt:5 }, c: { $gt: 1 } } ).sort( { a: 1, b: 1 } )
+
+The last two operations include query conditions on the field ``b`` but
+does not include a query condition on the field ``a``:
+
+.. code-block:: javascript
+
+   db.collection.find( { b: 5 } ).sort( { a: 1, b: 1 } )
+   db.collection.find( { b: { $gt:5 }, c: { $gt: 1 } } ).sort( { a: 1, b: 1 } )
+
+Consider the case where the collection has the index ``{ b: 1 }`` in
+addition to the ``{ a: 1, b: 1, c: 1, d: 1 }`` index. Because of the
+query condition on ``b``, it is not immediately obvious which index
+MongoDB may select as the "best" index. To explicitly specify the index
+to use, see :method:`~cursor.hint()`.
+
+.. _sort-equality-match:
+
+Sort Subset Does Not Start at the Index Beginning
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The sort document can be a subset of a compound index that does **not**
+start from the beginning of the index. For instance, ``{ c: 1 }`` is a
+subset of the index ``{ a: 1, b: 1, c: 1, d: 1 }`` that omits the
+preceding index fields ``a`` and ``b``. MongoDB can use the index
+efficiently **if** the the query document includes all the preceding
+fields of the index, in this case ``a`` and ``b``, in **equality**
+conditions. In other words, the equality conditions in the query
+document and the subset in the sort document **contiguously** cover a
+prefix of the index.
+
+For example, the collection ``collection`` has the following index:
+
+.. code-block:: javascript
+
+   { a: 1, b: 1, c: 1, d: 1 }
+
+Then following operations can use the index efficiently:
+
+.. code-block:: javascript
+
+   db.collection.find( { a: 5 } ).sort( { b: 1, c: 1 } )
+   db.collection.find( { a: 5, c: 4, b: 3 } ).sort( { d: 1 } )
+
+- In the first operation, the query document ``{ a: 5 }`` with the sort
+  document ``{ b: 1, c: 1 }`` cover the prefix ``{ a:1 , b: 1, c: 1 }``
+  of the index.
+
+- In the second operation, the query document ``{ a: 5, c: 4, b: 3 }``
+  with the sort document ``{ d: 1 }`` covers the full index.
+
+Only the index fields preceding the sort subset must have the equality
+conditions in the query document. The other index fields may have other
+conditions. The following operations can efficiently use the index
+since the equality conditions in the query document and the subset in
+the sort document **contiguously** cover a prefix of the index:
+
+.. code-block:: javascript
+
+   db.collection.find( { a: 5, b: 3 } ).sort( { c: 1 } )
+   db.collection.find( { a: 5, b: 3, c: { $lt: 4 } } ).sort( { c: 1 } )
+
+The following operations specify a sort document of ``{ c: 1 }``, but
+the query documents do not contain equality matches on the
+**preceding** index fields ``a`` and ``b``:
+
+.. code-block:: javascript
+
+   db.collection.find( { a: { $gt: 2 } } ).sort( { c: 1 } )
+   db.collection.find( { c: 5 } ).sort( { c: 1 } )
+
+These operations **will not** efficiently use the index ``{ a: 1, b: 1,
+c: 1, d: 1 }`` and may not even use the index to retrieve the documents.