DOCS-13668: Expose statistics which indicate how many collection scans have executed

Author: jwilliams-mongo

source/includes/extracts-server-status-projection-base.yaml

@@ -67,6 +67,12 @@ content: |
     - Added new metrics to track aggregation pipeline stage use in
       :serverstatus:`metrics.aggStageCounters`.
 
+    - Added new metrics to track queries that perform a collection scan:
+
+      - :serverstatus:`metrics.queryExecutor.collectionScans`
+      - :serverstatus:`metrics.queryExecutor.collectionScans.nonTailable`
+      - :serverstatus:`metrics.queryExecutor.collectionScans.total`
+
   - Starting in MongoDB 4.2.2, {{operationName}}:
 
     - Added new transaction metrics in :serverstatus:`transactions` for

source/includes/fact-queryexecstats-reference.rst

@@ -0,0 +1,20 @@
+The ``collectionScans`` field contains an embedded document bearing the
+following fields:
+
+.. list-table::
+    :header-rows: 1
+    :widths: 30 70
+
+    * - Field Name
+      - Description
+
+    * - ``total``
+      - A 64-bit integer giving the total number of queries that
+        performed a collection scan. The total consists of queries that
+        did and did not use a :doc:`tailable cursor
+        </core/tailable-cursors>`.
+
+    * - ``nonTailable``
+      - A 64-bit integer giving the number of queries that performed a
+        collection scan that did not use a :doc:`tailable cursor
+        </core/tailable-cursors>`.

source/reference/command/serverStatus.txt

@@ -3915,7 +3915,11 @@ metrics
       },
       "queryExecutor": {
             "scanned" : NumberLong(<num>),
-            "scannedObjects" : NumberLong(<num>)
+            "scannedObjects" : NumberLong(<num>),
+            "collectionScans" : {
+                "nonTailable" : NumbeLong(<num>),
+                "total" : NumberLong(<num>)
+            }
       },
       "record" : {
             "moves" : NumberLong(<num>)
@@ -4153,6 +4157,28 @@ metrics
    :data:`~explain.executionStats.totalDocsExamined` in the output of
    :method:`~cursor.explain()`.
 
+.. serverstatus:: metrics.queryExecutor.collectionScans
+
+   A document that reports on the number of queries that performed a
+   collection scan.
+
+   .. versionadded:: 4.4
+
+.. serverstatus:: metrics.queryExecutor.collectionScans.nonTailable
+
+   The number of queries that performed a collection scan that did not
+   use a :doc:`tailable cursor </core/tailable-cursors>`.
+
+   .. versionadded:: 4.4
+
+.. serverstatus:: metrics.queryExecutor.collectionScans.total
+
+   The total number queries that performed a collection scan. The total
+   consists of queries that did and did not use a :doc:`tailable cursor
+   </core/tailable-cursors>`.
+
+   .. versionadded:: 4.4
+
 .. serverstatus:: metrics.record
 
    A document that reports on data related to record allocation in the

source/reference/operator/aggregation/collStats.txt

@@ -28,7 +28,8 @@ Definition
           {
             latencyStats: { histograms: <boolean> },
             storageStats: { scale: <number> },
-            count: {}
+            count: {},
+            queryExecStats: {}
           }
       }
 
@@ -90,6 +91,12 @@ Definition
 
           .. versionadded:: 3.6
 
+      * - ``queryExecStats``
+        - Adds :ref:`query execution statistics
+          <collstat-queryexecstats>` to the return document.
+
+          .. versionadded:: 4.4
+
    For a collection in a replica set or a
    :ref:`non-sharded collection<sharded-vs-non-sharded-collections>` in
    a cluster, ``$collStats`` outputs a single document. For a
@@ -127,16 +134,16 @@ Definition
         milliseconds since the :term:`Unix epoch`.
 
     * - ``latencyStats``
-      - A collection of statistics related to request latency for a
-        collection or :doc:`view </core/views>`. See
-        :ref:`latency-stats-document` for details on this document.
+      - Statistics related to request latency for a collection or
+        :doc:`view </core/views>`. See :ref:`latency-stats-document`
+        for details on this document.
 
         Only present when the ``latencyStats: {}`` option is specified.
 
     * - ``storageStats``
 
-      - A collection of statistics related to a collection's storage
-        engine. See :ref:`storage-stats-document` for details on this
+      - Statistics related to a collection's storage engine. See
+        :ref:`storage-stats-document` for details on this
         document.
 
         The various size data is scaled by the specified factor (with
@@ -160,6 +167,13 @@ Definition
         Only present when the ``count: {}`` option is specified. Returns
         an error if applied to a :doc:`view </core/views>`.
 
+    * - ``queryExecStats``
+      - Statistics related to query execution for the collection. 
+
+        Only present when the ``queryExecStats: {}`` option is
+        specified. Returns an error if applied to a :doc:`view
+        </core/views>`. 
+
 Behavior
 --------
 
@@ -339,6 +353,40 @@ The total number of documents in the collection is also available as
 ``storageStats.count`` when ``storageStats: {}`` is specified. For more
 information, see :ref:`storage-stats-document`.
 
+.. _collstat-queryexecstats:
+
+``queryExecStats`` Document
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: 4.4
+
+The ``queryExecStats`` embedded document only exists in the output if
+you specify the ``queryExecStats`` option.
+
+.. include:: /includes/fact-queryexecstats-reference.rst
+
+For example, if you run ``$collStats`` with the ``queryExecStats: {}``
+option on a ``matrices`` collection:
+
+.. code-block:: javascript
+
+   db.matrices.aggregate( [ { $collStats: { queryExecStats: { } } } ] )
+
+The query returns a result similar to the following:
+
+.. code-block:: javascript
+
+   {
+     "ns": "test.matrices",
+     "host": "mongo.example.net:27017",
+     "localTime": ISODate("2020-06-03T14:23:29.711Z"),
+     "queryExecStats": {
+         "collectionScans": {
+             "total": NumberLong(33),
+             "nonTailable": NumberLong(31)
+         }
+     }
+   }
 
 ``$collStats`` on Sharded Collections
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

source/release-notes/4.4.txt

@@ -157,6 +157,15 @@ Starting in MongoDB 4.4 (also available starting in 4.2.4),
 
 .. include:: /includes/extracts/4.4-changes-planCache-changes.rst
 
+``$collStats`` Changes
+``````````````````````
+
+Starting in MongoDB 4.4, :pipeline:`$collStats` accepts the 
+``queryExecStats`` field as an argument document. Providing this field
+returns the following fields in the output:
+
+.. include:: /includes/fact-queryexecstats-reference.rst
+
 Replica Sets
 ------------
 
@@ -1297,6 +1306,12 @@ output:
              | :serverstatus:`mirroredReads.seen`
              | :serverstatus:`mirroredReads.sent`
 
+   Query Execution Metrics
+      .. list-table::
+
+         * - | :serverstatus:`metrics.queryExecutor.collectionScans`
+             | :serverstatus:`metrics.queryExecutor.collectionScans.nonTailable`
+             | :serverstatus:`metrics.queryExecutor.collectionScans.total`
 
    Replication Metrics
       .. list-table::