Skip to content

Covered index 956 #634

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 12, 2013
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 93 additions & 26 deletions source/applications/indexes.txt
Original file line number Diff line number Diff line change
Expand Up @@ -117,32 +117,99 @@ can support all the queries that search a "prefix" subset of those fields.
Create Indexes that Support Covered Queries
-------------------------------------------

A covered index query is a query in which all the queried fields are
part of an index. They are "covered queries" because an index "covers" the query.
MongoDB can fulfill the query by using *only* the index. MongoDB need
not scan documents from the database.

Querying *only* the index is much faster than querying documents. Index
keys are typically smaller than the documents they catalog, and indexes
are typically stored in RAM or located sequentially on disk.

Mongod automatically uses a covered query when possible. To ensure use
of a covered query, create an index that includes all the fields listed
in the query result. This means that the :term:`projection` document
given to a query (to specify which fields MongoDB returns from
the result set) must
explicitly exclude the ``_id`` field from the result set, unless the
index includes ``_id``.

MongoDB cannot use a covered query if any of the indexed fields in any
of the documents in the collection includes an array. If an indexed field
is an array, the index becomes a :ref:`multi-key index
<index-type-multikey>` index and cannot support a covered query.

To test whether MongoDB used a covered query, use
:method:`explain() <cursor.explain()>`. If the output displays ``true``
for the ``indexOnly`` field, MongoDB used a covered query. For
more information see :ref:`indexes-measuring-use`.
A covered query is a query in which:

- all the fields in the :ref:`query <read-operations-query-document>`
are part of an index, **and**

- all the fields returned in the results are in the same index.

Because the index "covers" the query, MongoDB can both match the
:ref:`query conditions <read-operations-query-document>` **and** return
the results using only the index; MongoDB does not need to look at
the documents, only the index, to fulfill the query.

Querying *only* the index can be much faster than querying documents
outside of the index. Index keys are typically smaller than the
documents they catalog, and indexes are typically available in RAM or
located sequentially on disk.

MongoDB automatically uses an index that covers a query when possible.
To ensure that a query is covered, create an index that includes all
the fields listed in the query result and the :ref:`query document
<read-operations-query-document>`. This means that if the index does
**not** include the ``_id`` field, the :term:`projection` document,
which specifies the fields MongoDB returns, must explicitly exclude the
``_id`` field from the result set.

Consider the following example where the collection ``user`` has
an index on the fields ``user`` and ``status``:

.. code-block:: javascript

{ status: 1, user: 1 }

Then, the following query which queries on the ``status`` field and
returns only the ``user`` field is covered:

.. code-block:: javascript

db.users.find( { status: "A" }, { user: 1, _id: 0 } )

However, the following query that uses the index to match documents is
**not** covered by the index because it returns both the ``user`` field
**and** the ``_id`` field:

.. code-block:: javascript

db.users.find( { status: "A" }, { user: 1 } )

An index **cannot** cover a query if:

- any of the indexed fields in any of the documents in the collection
includes an array. If an indexed field is an array, the index becomes
a :ref:`multi-key index <index-type-multikey>` index and cannot
support a covered query.

- any of the indexed fields are fields in subdocuments. To index fields
in subdocuments, use :term:`dot notation`. For example, consider
a collection ``users`` with documents of the following form:

.. code-block:: javascript

{ _id: 1, user: { login: "tester" } }

The collection has the following indexes:

.. code-block:: none

{ user: 1 }

{ "user.login": 1 }

The ``{ user: 1 }`` index covers the following query:

.. code-block:: none

db.users.find( { user: { login: "tester" } }, { user: 1, _id: 0 } )

However, the ``{ "user.login": 1 }`` index does **not** cover the
following query:

.. code-block:: none

db.users.find( { "user.login": "tester" }, { "user.login": 1, _id: 0 } )

The query, however, does use the ``{ "user.login": 1 }`` index to
find matching documents.

To determine whether a query is a covered query, use the
:method:`~cursor.explain()` method. If the :method:`~cursor.explain()`
output displays ``true`` for the :data:`indexOnly` field, the query is
covered by an index, and MongoDB queries only that index to match the
query **and** return the results.

For more information see :ref:`indexes-measuring-use`.

.. _index-sort:
.. _sorting-with-indexes:
Expand Down
21 changes: 16 additions & 5 deletions source/core/indexes.txt
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,22 @@ MongoDB indexes have the following core features:
with the best response time for each query type. You can override
the query optimizer using the :method:`cursor.hint()` method.

- An index "covers" a query if the index keys store all the data that
the query must return. When an index covers a query, the database
returns results more quickly than for queries that have to scan many
individual documents. See :ref:`indexes-covered-queries` for more
information.
- An index "covers" a query if:

- all the fields in the :ref:`query <read-operations-query-document>`
are part of that index, **and**

- all the fields returned in the documents that match the query are
in the same index.

When an index covers a query, the server can both match the
:ref:`query conditions <read-operations-query-document>` **and**
return the results using only the index; MongoDB does not need to
look at the documents, only the index, to fulfill the query.
Querying the index can be faster than querying the documents outside
of the index.

See :ref:`indexes-covered-queries` for more information.

- Using queries with good index coverage reduces the number of full
documents that MongoDB needs to store in memory, thus maximizing database
Expand Down
64 changes: 58 additions & 6 deletions source/core/read-operations.txt
Original file line number Diff line number Diff line change
Expand Up @@ -354,9 +354,9 @@ Result Projections
~~~~~~~~~~~~~~~~~~

The :term:`projection` specification limits the fields to return for
all matching documents. Constraining the result set by restricting the
fields to return can minimize network transit costs and the costs of
deserializing documents in the application layer.
all matching documents. Restricting the fields to return can minimize
network transit costs and the costs of deserializing documents in the
application layer.

The second argument to the :method:`find() <db.collection.find()>`
method is a projection, and it takes the form of a :term:`document` with
Expand Down Expand Up @@ -458,6 +458,54 @@ operation:
<index-ascending-and-descending>` to support sort operations in
compound queries.

.. _read-operations-covered-query:

Covering a Query
~~~~~~~~~~~~~~~~

An index :ref:`covers <indexes-covered-queries>` a query, or a query is
a "covered" query, when:

- all the fields in the :ref:`query <read-operations-query-document>`
are part of that index, **and**

- all the fields returned in the documents that match the query are in
the same index.

When an index "covers" the query, MongoDB can both match the
:ref:`query conditions <read-operations-query-document>` **and** return
the results using only the index; MongoDB does not need to look at
documents outside of the index. Querying the index can be faster than
querying the documents outside of the index.

Consider the following example where the collection ``inventory`` has
an index on the fields ``type`` and ``item``.

.. code-block:: sh

{ type: 1, item: 1 }

Then the following query that queries on the ``type`` and ``item``
fields and returns only the ``item`` field is covered by the ``{type:
1, item: 1}`` index:

.. code-block:: javascript

db.inventory.find( { type: "food", item:/^c/ },
{ item: 1, _id: 0 } )

However, the following query that uses the ``{ type: 1, item: 1 }`` index
to match documents is **not** covered by the index because the query
returns the ``item`` field **and** the ``_id`` field:

.. code-block:: javascript

db.inventory.find( { type: "food", item:/^c/ },
{ item: 1 } )

See :ref:`indexes-covered-queries` for more information, including when
indexes cannot cover a query.

Measuring Index Use
~~~~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -552,9 +600,13 @@ the query used an index. This query:
- then read 5 full documents from the collection, as indicated by
the :data:`nscannedObjects` field.

This indicates that the query was not "covered," or able to complete
only using the index, as reflected in the :data:`indexOnly`. See
:ref:`indexes-covered-queries` for more information.
Although the query uses an index to find the matching documents, the
:data:`indexOnly:false <indexOnly>` indicates that this index did
not :ref:`cover <read-operations-covered-query>` the query; i.e.
MongoDB could not both match the :ref:`query conditions
<read-operations-query-document>` **and** return the results using
only this index. See :ref:`indexes-covered-queries` for more
information.

.. index:: query optimizer
.. _read-operations-query-optimization:
Expand Down
22 changes: 16 additions & 6 deletions source/reference/explain.txt
Original file line number Diff line number Diff line change
Expand Up @@ -155,16 +155,19 @@ Core Explain Output

Specifies the total number of documents scanned during the query.
The :data:`nscannedObjects` may be lower than :data:`nscanned`, such
as if the index is a covered index.
as if the index :ref:`covers <indexes-covered-queries>` a query. See
:data:`indexOnly`. Additionally, the :data:`nscannedObjects` may be
lower than :data:`nscanned` in the case of multikey index on an
array field with duplicate documents.

.. data:: nscanned

Specifies the total number of documents or index entries scanned
during the database operation. You want :data:`n` and
:data:`nscanned` to be close in value as possible. The
:data:`nscanned` value may be higher than the
:data:`nscannedObjects` value, such as if the index is a covered
index.
:data:`nscannedObjects` value, such as if the index :ref:`covers
<indexes-covered-queries>` a query. See :data:`indexOnly`.

.. data:: nscannedObjectsAllPlans

Expand Down Expand Up @@ -194,9 +197,16 @@ Core Explain Output
.. data:: indexOnly

:data:`indexOnly` is a boolean value that returns ``true`` when the
query is :ref:`covered <indexes-covered-queries>` by the index. In
*covered* queries, the index contains all data that MongoDB needs
to fulfill the query.
the query is :ref:`covered <indexes-covered-queries>` by the index
indicated in the :data:`cursor` field. When an index covers a query,
MongoDB can both match the :ref:`query conditions
<read-operations-query-document>` **and** return the results using
only the index because:

- all the fields in the :ref:`query
<read-operations-query-document>` are part of that index, **and**

- all the fields returned in the results set are in the same index.

.. data:: nYields

Expand Down