diff --git a/source/applications.txt b/source/applications.txt index 231826f8fb2..9aa476af39e 100644 --- a/source/applications.txt +++ b/source/applications.txt @@ -57,4 +57,5 @@ The following documents provide patterns for developing application features: tutorial/isolate-sequence-of-operations tutorial/create-an-auto-incrementing-field tutorial/expire-data + tutorial/create-tailable-cursor diff --git a/source/core/capped-collections.txt b/source/core/capped-collections.txt index df6efbb9a54..c9c53454028 100644 --- a/source/core/capped-collections.txt +++ b/source/core/capped-collections.txt @@ -161,3 +161,15 @@ value of a date-typed field and a TTL value for the index. :doc:`TTL Collections ` are not compatible with capped collections. + +Tailable Cursor +~~~~~~~~~~~~~~~ + +You can use a tailable cursor with capped collections. Similar to the +Unix ``tail -f`` command, the tailable cursor "tails" the end of a +capped collection. As new documents are inserted into the capped +collection, you can use the tailable cursor to continue retrieving +documents. + +See :doc:`/tutorial/create-tailable-cursor` for information on creating +a tailable cursor. diff --git a/source/tutorial/create-tailable-cursor.txt b/source/tutorial/create-tailable-cursor.txt new file mode 100644 index 00000000000..00128956d3f --- /dev/null +++ b/source/tutorial/create-tailable-cursor.txt @@ -0,0 +1,176 @@ +====================== +Create Tailable Cursor +====================== + +.. default-domain:: mongodb + +Overview +-------- + +By default, MongoDB will automatically close a cursor if the cursor has +been exhausted. However, with :doc:`capped collections +`, you can create a *Tailable Cursor* which +remains open after the cursor has been exhausted. Similar to the Unix +``tail -f`` command, the tailable cursor "tails" the end of a capped +collection. After new documents are inserted into the capped +collection, you can use the tailable cursor to continue retrieving +documents. + +Use tailable cursors on capped collections with high numbers of write +operations for which an index would be too expensive. For instance, +MongoDB :doc:`replication ` uses tailable cursors to +tail the primary's :term:`oplog`. + +If your query is on an indexed field, do not use tailable cursors, but +instead, use a regular cursor. Keep track of the last value of the +indexed field returned by the query. To retrieve the newly added +documents, query the collection again using the last value of the +indexed field in the query criteria, as in the following example: + +.. code-block:: javascript + + db.mycollection.find( { indexedField: { $gt: } } ) + +Consider the following behaviors related to tailable cursors: + +- Tailable cursors do not use indexes and return documents in + :term:`natural order`. + +- Because tailable cursors do not use indexes, the initial scan for the + query may be expensive; but, after initially exhausting the cursor, + subsequent retrievals of the newly added documents are inexpensive. + +- Tailable cursors may become *dead*, or invalid, if either: + + - the query returned no match. + + - the last returned object is at the end of the collection and is + deleted. + + A *dead* cursor has an id of ``0``. + +See your :doc:`driver documentation ` for the +driver-specific method to specify the tailable cursor. +[#tailable-cursor-wire-protocol]_ + +.. [#tailable-cursor-wire-protocol] For more information on the details + of specifying a tailable cursor, see :wiki:`Mongo wire protocol ` documentation. + +C++ Example +----------- + +The ``tail`` function uses a tailable cursor to output the results from +a query to a capped collection: + +- The function handles the case of the dead cursor by having the query + be inside a loop. + +- To periodically check for new data, the ``cursor->more()`` statement + is also inside a loop. + +.. code-block:: cpp + + #include "client/dbclient.h" + + using namespace mongo; + + /* + * Example of a tailable cursor. + * The function "tails" the capped collection (ns) and output elements as they are added. + * The function also handles the possibility of a dead cursor by tracking the field 'insertDate'. + * New documents are added with increasing values of 'insertDate'. + */ + + void tail(DBClientBase& conn, const char *ns) { + + BSONElement lastValue = minKey.firstElement(); + + Query query = Query().hint( BSON( "$natural" << 1 ) ); + + while ( 1 ) { + auto_ptr c = + conn.query(ns, query, 0, 0, 0, + QueryOption_CursorTailable | QueryOption_AwaitData ); + + while ( 1 ) { + if ( !c->more() ) { + + if ( c->isDead() ) { + break; + } + + continue; + } + + BSONObj o = c->next(); + lastValue = o["insertDate"]; + cout << o.toString() << endl; + } + + query = QUERY( "insertDate" << GT << lastValue ).hint( BSON( "$natural" << 1 ) ); + } + } + +The ``tail`` function performs the following actions: + +- Initialize the ``lastValue`` variable, which tracks the last accessed + value and will be used if the cursor becomes *dead* and we need to + requery. Use ``hint()`` to ensure that the ``$natural`` order is used. + +- In an outer ``while(1)`` loop, + + - Query the capped collection and return a tailable cursor that + blocks for several seconds waiting for new documents + + .. code-block:: cpp + + auto_ptr c = + conn.query(ns, query, 0, 0, 0, + QueryOption_CursorTailable | QueryOption_AwaitData ); + + - The capped collection is specified by ``ns`` and is an argument + to the function. + + - The function sets the ``QueryOption_CursorTailable`` option to + specify that the returned cursor is a tailable cursor. + + - The function sets the ``QueryOption_AwaitData`` option to specify + that the returned cursor should block for a few seconds to wait + for data. + + - In an inner ``while (1)`` loop, read the documents from the cursor: + + - If the cursor has no more documents and is not dead, loop the + inner ``while`` loop to recheck for more documents. + + - If the cursor has no more documents and is dead, break the inner + ``while`` loop. + + - If the cursor has documents, + + - output the document, + + - update the ``lastValue`` value, + + - and loop the inner ``while (1)`` loop to recheck for more + documents. + + - If the logic breaks out of the inner ``while (1)`` loop (i.e. the + cursor is dead): + + - Use the ``lastValue`` value to create a new query condition that + matches documents added after the ``lastValue``. Explicitly + ensure ``$natural`` order with the ``hint()`` method: + + .. code-block:: cpp + + query = QUERY( "insertDate" << GT << lastValue ).hint( BSON( "$natural" << 1 ) ); + + - Loop through the outer ``while (1)`` loop to re-query with the new query + condition and repeat. + +.. seealso:: + + `Detailed blog post on tailable cursor + `_