diff --git a/source/includes/covariance-behavior.rst b/source/includes/covariance-behavior.rst new file mode 100644 index 00000000000..7b9375e0bbd --- /dev/null +++ b/source/includes/covariance-behavior.rst @@ -0,0 +1,33 @@ +- Ignores non-numeric values, ``null`` values, and missing fields in a + window. + +- If the window contains one document, returns ``null``. + (Compare to |covariance-method|, which returns |covariance-value| if + the window contains one document.) + +- If the window is empty, returns ``null``. + +- If the window contains a ``NaN`` value, returns ``NaN``. + +- If the window contains one or more ``Infinity`` value(s) that are + all positive or all negative, returns ``Infinity``. The returned + ``Infinity`` value has the same sign as the ``Infinity`` values in the + window. + +- If the window contains ``Infinity`` values with different signs, + returns ``NaN``. + +- If the window contains a ``decimal`` value, returns a ``decimal`` + value. + +- If none of the previous points apply, returns a ``double`` value. + +The returned values in order of precedence are as follows: + +- ``NaN`` + +- ``Infinity`` + +- ``decimal`` + +- ``double`` \ No newline at end of file diff --git a/source/includes/expMovingAvg-N-or-alpha.rst b/source/includes/expMovingAvg-N-or-alpha.rst new file mode 100644 index 00000000000..948b9f3acd2 --- /dev/null +++ b/source/includes/expMovingAvg-N-or-alpha.rst @@ -0,0 +1,2 @@ +You must specify either :ref:`N ` or :ref:`alpha +`. You cannot specify both. \ No newline at end of file diff --git a/source/includes/extracts-agg-operators.yaml b/source/includes/extracts-agg-operators.yaml index 422a9a36b3f..752f1370303 100644 --- a/source/includes/extracts-agg-operators.yaml +++ b/source/includes/extracts-agg-operators.yaml @@ -930,6 +930,8 @@ content: | ref: agg-operators-accumulators-group content: | + .. versionchanged:: 5.0 + .. list-table:: :header-rows: 1 :widths: 20 80 @@ -943,36 +945,62 @@ content: | * - :group:`$addToSet` - - Returns an array of *unique* expression values for each - group. Order of the array elements is undefined. + - Returns an array of *unique* expression values for each group. + Order of the array elements is undefined. + + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. * - :group:`$avg` - Returns an average of numerical values. Ignores non-numeric values. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + * - :group:`$count` - - Returns the count of the documents in the stage. May be used in - place of ``{ $sum : 1 }``. + - Returns the number of documents in a group. + + Distinct from the :pipeline:`$count` pipeline stage. + + .. versionadded:: 5.0 + + Available in :pipeline:`$group` and + :pipeline:`$setWindowFields` stages. * - :group:`$first` - Returns a value from the first document for each group. Order - is only defined if the documents are in a defined order. + is only defined if the documents are sorted. Distinct from the :expression:`$first` array operator. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + * - :group:`$last` - Returns a value from the last document for each group. Order - is only defined if the documents are in a defined order. + is only defined if the documents are sorted. Distinct from the :expression:`$last` array operator. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + * - :group:`$max` - Returns the highest expression value for each group. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + * - :expression:`$mergeObjects` - Returns a document created by combining the input documents @@ -982,32 +1010,52 @@ content: | - Returns the lowest expression value for each group. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + * - :group:`$push` - - Returns an array of expression values for each group. + - Returns an array of expression values for documents in each + group. + + .. versionchanged:: 5.0 + Available in :pipeline:`$setWindowFields` stage. * - :group:`$stdDevPop` - Returns the population standard deviation of the input values. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + * - :group:`$stdDevSamp` - Returns the sample standard deviation of the input values. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + * - :group:`$sum` - Returns a sum of numerical values. Ignores non-numeric values. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. --- ref: agg-operators-accumulators-project content: | - .. versionchanged:: 3.2 + .. versionchanged:: 5.0 The following accumulator operators are also available in the - :pipeline:`$project`, :pipeline:`$addFields`, and :pipeline:`$set` stages. + :pipeline:`$project`, :pipeline:`$addFields`, :pipeline:`$set`, and, + starting in MongoDB 5.0, the :pipeline:`$setWindowFields` stages. .. list-table:: :header-rows: 1 @@ -1021,7 +1069,6 @@ content: | - Returns an average of the specified expression or list of expressions for each document. Ignores non-numeric values. - * - :group:`$max` - Returns the maximum of the specified expression or list of @@ -1044,4 +1091,200 @@ content: | - Returns a sum of numerical values. Ignores non-numeric values. +--- +ref: agg-operators-window +content: | + + .. versionadded:: 5.0 + + Window operators return values from a defined span of documents from + a collection, known as a *window*. A :ref:`window + ` is defined in the + :pipeline:`$setWindowFields` stage, available starting in MongoDB + 5.0. + + The following window operators are available in the + :pipeline:`$setWindowFields` stage. + + .. list-table:: + :header-rows: 1 + :widths: 20 80 + + * - Name + - Description + + * - :group:`$addToSet` + + - Returns an array of all unique values that results from + applying an :ref:`expression ` to + each document. + + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :group:`$avg` + + - Returns the average for the specified :ref:`expression + `. Ignores non-numeric values. + + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :group:`$count` + + - Returns the number of documents in the group or window. + + Distinct from the :pipeline:`$count` pipeline stage. + + .. versionadded:: 5.0 + + * - :group:`$covariancePop` + + - Returns the population covariance of two numeric + :ref:`expressions `. + + .. versionadded:: 5.0 + + * - :group:`$covarianceSamp` + + - Returns the sample covariance of two numeric + :ref:`expressions `. + + .. versionadded:: 5.0 + + * - :group:`$denseRank` + + - Returns the document position (known as the rank) relative to + other documents in the :pipeline:`$setWindowFields` stage + :ref:`partition `. There are no + gaps in the ranks. Ties receive the same rank. + + .. versionadded:: 5.0 + + * - :group:`$derivative` + + - Returns the mathematical derivative of the numeric + :ref:`expression `. + + .. versionadded:: 5.0 + + * - :group:`$documentNumber` + + - Returns the position of a document (known as the document + number) in the :pipeline:`$setWindowFields` stage + :ref:`partition `. Ties result in + different adjacent document numbers. + + .. versionadded:: 5.0 + + * - :group:`$expMovingAvg` + + - Returns the exponential moving average for the numeric + :ref:`expression `. + + .. versionadded:: 5.0 + + * - :group:`$first` + + - Returns the value that results from applying an + :ref:`expression ` to the first + document in a group or :ref:`window `. + + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :group:`$integral` + + - Returns the mathematical integral of the numeric + :ref:`expression `. + + .. versionadded:: 5.0 + + * - :group:`$last` + + - Returns the value that results from applying an + :ref:`expression ` to the last + document in a group or :ref:`window `. + + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :group:`$max` + + - Returns the maximum value that results from applying an + :ref:`expression ` to each document. + + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :group:`$min` + + - Returns the minimum value that results from applying an + :ref:`expression ` to each document. + + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :group:`$push` + + - Returns an array of values that result from applying an + :ref:`expression ` to each document. + + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :group:`$rank` + + - Returns the document position (known as the rank) relative to + other documents in the :pipeline:`$setWindowFields` stage + :ref:`partition `. + + .. versionadded:: 5.0 + + * - :group:`$shift` + + - Returns the value from an :ref:`expression + ` applied to a document in a + specified position relative to the current document in the + :pipeline:`$setWindowFields` stage :ref:`partition + `. + + .. versionadded:: 5.0 + + * - :group:`$stdDevPop` + + - Returns the population standard deviation that results from + applying a numeric :ref:`expression ` + to each document. + + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :group:`$stdDevSamp` + + - Returns the sample standard deviation that results from + applying a numeric :ref:`expression ` + to each document. + + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :group:`$sum` + + - Returns the sum that results from applying a numeric + :ref:`expression ` to each document. + + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + ... diff --git a/source/includes/extracts-agg-stages.yaml b/source/includes/extracts-agg-stages.yaml index 47dfb4db4ab..e99c6648bfc 100644 --- a/source/includes/extracts-agg-stages.yaml +++ b/source/includes/extracts-agg-stages.yaml @@ -40,6 +40,8 @@ content: | - Returns a count of the number of documents at this stage of the aggregation pipeline. + Distinct from the :group:`$count` aggregation accumulator. + * - :pipeline:`$facet` - Processes multiple :ref:`aggregation pipelines @@ -186,6 +188,13 @@ content: | :pipeline:`$set` is an alias for :pipeline:`$addFields` stage. + * - :pipeline:`$setWindowFields` + + - Groups documents into windows and applies one or more + operators to the documents in each window. + + .. versionadded:: 5.0 + * - :pipeline:`$skip` - Skips the first *n* documents where *n* is the specified skip diff --git a/source/includes/extracts-fact-aggregation-accumulator-base.yaml b/source/includes/extracts-fact-aggregation-accumulator-base.yaml index 0a5df666e08..ee675357958 100644 --- a/source/includes/extracts-fact-aggregation-accumulator-base.yaml +++ b/source/includes/extracts-fact-aggregation-accumulator-base.yaml @@ -1,7 +1,7 @@ ref: _fact-agg-accumulator content: | - {{accumulator}} is only available in the - :pipeline:`$group` stage. + {{accumulator}} is available in the :pipeline:`$group` stage and, + starting in MongoDB 5.0, the :pipeline:`$setWindowFields` stage. --- ref: _fact-agg-accumulator-group-project-availability content: | @@ -22,6 +22,8 @@ content: | - :pipeline:`$match` stage that includes an :query:`$expr` expression + - :pipeline:`$setWindowFields` (Available starting in MongoDB 5.0) + --- ref: _fact-agg-accumulator-availability-change content: | @@ -42,6 +44,8 @@ content: | - :pipeline:`$match` stage that includes an :query:`$expr` expression + - :pipeline:`$setWindowFields` (Available starting in MongoDB 5.0) + In MongoDB 3.2 and earlier, {{accumulator}} is available in the :pipeline:`$group` stage only. diff --git a/source/includes/note-group-and-window-behavior.rst b/source/includes/note-group-and-window-behavior.rst new file mode 100644 index 00000000000..634449f0652 --- /dev/null +++ b/source/includes/note-group-and-window-behavior.rst @@ -0,0 +1,20 @@ +To define the document order for |method| with the: + +- :pipeline:`$group` stage, add a :pipeline:`$sort` stage after the + :pipeline:`$group` stage. + +- :pipeline:`$setWindowFields` stage, set the :ref:`sortBy + ` field. + +.. note:: + + Although the :pipeline:`$sort` stage passes ordered documents as + input to the :pipeline:`$group` and :pipeline:`$setWindowFields` + stages, those stages are not guaranteed to maintain the sort order in + their own output. + +When used with :pipeline:`$setWindowFields`, |method| returns ``null`` +for empty :ref:`windows `. An example empty +window is a ``{ documents: [ -1, -1 ] }`` :ref:`documents +` window on the first document of a +:ref:`partition `. \ No newline at end of file diff --git a/source/includes/outputUnit-behavior.rst b/source/includes/outputUnit-behavior.rst new file mode 100644 index 00000000000..1291ecdbc61 --- /dev/null +++ b/source/includes/outputUnit-behavior.rst @@ -0,0 +1,18 @@ +A ``string`` that specifies the time unit. Use one of these +strings: + +- ``"week"`` + +- ``"day"`` + +- ``"hour"`` + +- ``"minute"`` + +- ``"second"`` + +- ``"millisecond"`` + +If the :ref:`sortBy ` field is not a date, you +must omit a ``unit``. If you specify a ``unit``, you must specify a date +in the :ref:`sortBy ` field. \ No newline at end of file diff --git a/source/includes/rank-and-denseRank-behavior.rst b/source/includes/rank-and-denseRank-behavior.rst new file mode 100644 index 00000000000..89a7d39c220 --- /dev/null +++ b/source/includes/rank-and-denseRank-behavior.rst @@ -0,0 +1,16 @@ +:group:`$rank` and :group:`$denseRank` differ in how they rank duplicate +:ref:`sortBy ` field values. For example, with +:ref:`sortBy ` field values of 7, 9, 9, and 10: + +- :group:`$denseRank` ranks the values as 1, 2, 2, and 3. The + duplicate 9 values have a rank of 2, and 10 has a rank of 3. There is + no gap in the ranks. + +- :group:`$rank` ranks the values as 1, 2, 2, and 4. The duplicate 9 + values have a rank of 2, and 10 has a rank of 4. There is a gap in the + ranks for 3. + +Documents with a ``null`` value for a :ref:`sortBy +` field or documents missing the :ref:`sortBy +` field are assigned a rank based on the +:ref:`BSON comparison order `. \ No newline at end of file diff --git a/source/includes/setWindowFields-duplicates-example-collection.rst b/source/includes/setWindowFields-duplicates-example-collection.rst new file mode 100644 index 00000000000..177e415a22d --- /dev/null +++ b/source/includes/setWindowFields-duplicates-example-collection.rst @@ -0,0 +1,42 @@ +Create a ``cakeSalesWithDuplicates`` collection where: + +- Cake sales are placed in the state of California (``CA``) and + Washington (``WA``). + +- Documents 6 to 8 have the same ``quantity`` and ``state`` as document + 5. + +- Document 9 has the same ``quantity`` and ``state`` as document 4. + +- Document 10 has a ``null`` ``quantity``. + +- Document 11 is missing the ``quantity``. + +.. code-block:: javascript + + db.cakeSalesWithDuplicates.insertMany( [ + { _id: 0, type: "chocolate", orderDate: new Date("2020-05-18T14:10:30Z"), + state: "CA", price: 13, quantity: 120 }, + { _id: 1, type: "chocolate", orderDate: new Date("2021-03-20T11:30:05Z"), + state: "WA", price: 14, quantity: 140 }, + { _id: 2, type: "vanilla", orderDate: new Date("2021-01-11T06:31:15Z"), + state: "CA", price: 12, quantity: 145 }, + { _id: 3, type: "vanilla", orderDate: new Date("2020-02-08T13:13:23Z"), + state: "WA", price: 13, quantity: 104 }, + { _id: 4, type: "strawberry", orderDate: new Date("2019-05-18T16:09:01Z"), + state: "CA", price: 41, quantity: 162 }, + { _id: 5, type: "strawberry", orderDate: new Date("2019-01-08T06:12:03Z"), + state: "WA", price: 43, quantity: 134 }, + { _id: 6, type: "strawberry", orderDate: new Date("2020-01-08T06:12:03Z"), + state: "WA", price: 41, quantity: 134 }, + { _id: 7, type: "strawberry", orderDate: new Date("2020-01-01T06:12:03Z"), + state: "WA", price: 34, quantity: 134 }, + { _id: 8, type: "strawberry", orderDate: new Date("2020-01-02T06:12:03Z"), + state: "WA", price: 40, quantity: 134 }, + { _id: 9, type: "strawberry", orderDate: new Date("2020-05-11T16:09:01Z"), + state: "CA", price: 39, quantity: 162 }, + { _id: 10, type: "strawberry", orderDate: new Date("2020-05-11T16:09:01Z"), + state: "CA", price: 39, quantity: null }, + { _id: 11, type: "strawberry", orderDate: new Date("2020-05-11T16:09:01Z"), + state: "CA", price: 39 } + ] ) \ No newline at end of file diff --git a/source/includes/setWindowFields-example-collection.rst b/source/includes/setWindowFields-example-collection.rst new file mode 100644 index 00000000000..f5601267498 --- /dev/null +++ b/source/includes/setWindowFields-example-collection.rst @@ -0,0 +1,19 @@ +Create a ``cakeSales`` collection that contains cake sales in the states +of California (``CA``) and Washington (``WA``): + +.. code-block:: javascript + + db.cakeSales.insertMany( [ + { _id: 0, type: "chocolate", orderDate: new Date("2020-05-18T14:10:30Z"), + state: "CA", price: 13, quantity: 120 }, + { _id: 1, type: "chocolate", orderDate: new Date("2021-03-20T11:30:05Z"), + state: "WA", price: 14, quantity: 140 }, + { _id: 2, type: "vanilla", orderDate: new Date("2021-01-11T06:31:15Z"), + state: "CA", price: 12, quantity: 145 }, + { _id: 3, type: "vanilla", orderDate: new Date("2020-02-08T13:13:23Z"), + state: "WA", price: 13, quantity: 104 }, + { _id: 4, type: "strawberry", orderDate: new Date("2019-05-18T16:09:01Z"), + state: "CA", price: 41, quantity: 162 }, + { _id: 5, type: "strawberry", orderDate: new Date("2019-01-08T06:12:03Z"), + state: "WA", price: 43, quantity: 134 } + ] ) \ No newline at end of file diff --git a/source/includes/setWindowFields-introduction-examples.rst b/source/includes/setWindowFields-introduction-examples.rst new file mode 100644 index 00000000000..746f3697da4 --- /dev/null +++ b/source/includes/setWindowFields-introduction-examples.rst @@ -0,0 +1,11 @@ +For example, you can use the :pipeline:`$setWindowFields` stage to +output the: + +- Difference in sales between two documents in a collection. + +- Sales rankings. + +- Cumulative sales totals. + +- Analysis of complex time series information without exporting the + data to an external database. \ No newline at end of file diff --git a/source/includes/setWindowFields-operators.rst b/source/includes/setWindowFields-operators.rst new file mode 100644 index 00000000000..fff9a2002ef --- /dev/null +++ b/source/includes/setWindowFields-operators.rst @@ -0,0 +1,18 @@ +These operators can be used with the :pipeline:`$setWindowFields` stage: + +.. _setWindowFields-accumulator-operators: + +- Accumulator operators: :group:`$addToSet`, :group:`$avg`, + :group:`$count`, :group:`$covariancePop`, :group:`$covarianceSamp`, + :group:`$derivative`, :group:`$expMovingAvg`, :group:`$integral`, + :group:`$max`, :group:`$min`, :group:`$push`, :group:`$stdDevSamp`, + :group:`$stdDevPop`, and :group:`$sum`. + +.. _setWindowFields-order-operators: + +- Order operators: :group:`$first`, :group:`$last`, and :group:`$shift`. + +.. _setWindowFields-rank-operators: + +- Rank operators: :group:`$denseRank`, :group:`$documentNumber`, and + :group:`$rank`. \ No newline at end of file diff --git a/source/includes/setWindowFields-partition-sort-date.rst b/source/includes/setWindowFields-partition-sort-date.rst new file mode 100644 index 00000000000..13694f7d606 --- /dev/null +++ b/source/includes/setWindowFields-partition-sort-date.rst @@ -0,0 +1,8 @@ +- ``partitionBy: "$state"`` :ref:`partitions + ` the documents in the collection by + ``state``. There are partitions for ``CA`` and ``WA``. + +- ``sortBy: { orderDate: 1 }`` :ref:`sorts + ` the documents in each partition by + ``orderDate`` in ascending order (``1``), so the earliest + ``orderDate`` is first. \ No newline at end of file diff --git a/source/includes/setWindowFields-partition-sort-quantity.rst b/source/includes/setWindowFields-partition-sort-quantity.rst new file mode 100644 index 00000000000..1d8936541da --- /dev/null +++ b/source/includes/setWindowFields-partition-sort-quantity.rst @@ -0,0 +1,8 @@ +- ``partitionBy: "$state"`` :ref:`partitions + ` the documents in the collection by + ``state``. There are partitions for ``CA`` and ``WA``. + +- ``sortBy: { quantity: -1 }`` :ref:`sorts + ` the documents in each partition by + ``quantity`` in descending order (``-1``), so the highest ``quantity`` + is first. \ No newline at end of file diff --git a/source/includes/stdDev-behavior.rst b/source/includes/stdDev-behavior.rst new file mode 100644 index 00000000000..96004329200 --- /dev/null +++ b/source/includes/stdDev-behavior.rst @@ -0,0 +1,13 @@ +Behavior with values in a :pipeline:`$setWindowFields` stage +:ref:`window `: + +- Ignores non-numeric values, ``null`` values, and missing fields in a + window. + +- If the window is empty, returns ``null``. + +- If the window contains a ``NaN`` value, returns ``null``. + +- If the window contains ``Infinity`` values, returns ``null``. + +- If none of the previous points apply, returns a ``double`` value. \ No newline at end of file diff --git a/source/includes/time-range-window-example.rst b/source/includes/time-range-window-example.rst new file mode 100644 index 00000000000..8d83b97b2d3 --- /dev/null +++ b/source/includes/time-range-window-example.rst @@ -0,0 +1,20 @@ +In the example: + +- ``partitionBy: "$state"`` :ref:`partitions + ` the documents in the collection by + ``state``. There are partitions for ``CA`` and ``WA``. + +- ``sortBy: { orderDate: 1 }`` :ref:`sorts + ` the documents in each partition by + ``orderDate`` in ascending order (``1``), so the earliest + ``orderDate`` is first. + +- ``output``: + + - Sets the ``orderDateArrayForState`` array field to ``orderDate`` + values for the documents in each ``state``. The array elements are + expanded with additions to the previous elements in the array. + + - Uses :group:`$push` to return an array of ``orderDate`` + values from the documents in a :ref:`range ` + window. \ No newline at end of file diff --git a/source/meta/aggregation-quick-reference.txt b/source/meta/aggregation-quick-reference.txt index e155be44dfe..371c17c234f 100644 --- a/source/meta/aggregation-quick-reference.txt +++ b/source/meta/aggregation-quick-reference.txt @@ -324,6 +324,11 @@ Variable Expression Operators .. include:: /includes/extracts/agg-operators-variable-project.rst +Window Operators +~~~~~~~~~~~~~~~~ + +.. include:: /includes/extracts/agg-operators-window.rst + Index of Expression Operators ----------------------------- diff --git a/source/reference/operator/aggregation-pipeline.txt b/source/reference/operator/aggregation-pipeline.txt index 178d32612db..a066d8a7890 100644 --- a/source/reference/operator/aggregation-pipeline.txt +++ b/source/reference/operator/aggregation-pipeline.txt @@ -97,6 +97,8 @@ Alphabetical Listing of Stages - Returns a count of the number of documents at this stage of the aggregation pipeline. + Distinct from the :group:`$count` aggregation accumulator. + * - :pipeline:`$currentOp` @@ -262,6 +264,14 @@ Alphabetical Listing of Stages Alias for :pipeline:`$addFields`. + * - :pipeline:`$setWindowFields` + + - Groups documents into windows and applies one or more operators + to the documents in each window. + + .. versionadded:: 5.0 + + * - :pipeline:`$skip` - Skips the first *n* documents where *n* is the specified skip number @@ -335,6 +345,7 @@ Alphabetical Listing of Stages /reference/operator/aggregation/sample /reference/operator/aggregation/search /reference/operator/aggregation/set + /reference/operator/aggregation/setWindowFields /reference/operator/aggregation/skip /reference/operator/aggregation/sort /reference/operator/aggregation/sortByCount diff --git a/source/reference/operator/aggregation.txt b/source/reference/operator/aggregation.txt index 149d5fb21e7..b1a4a33b7de 100644 --- a/source/reference/operator/aggregation.txt +++ b/source/reference/operator/aggregation.txt @@ -138,6 +138,11 @@ Variable Expression Operators .. include:: /includes/extracts/agg-operators-variable-project.rst +Window Operators +~~~~~~~~~~~~~~~~ + +.. include:: /includes/extracts/agg-operators-window.rst + Alphabetical Listing of Expression Operators -------------------------------------------- @@ -176,10 +181,12 @@ Alphabetical Listing of Expression Operators * - :group:`$addToSet` - - Returns an array of *unique* expression values for each group. Order of the - array elements is undefined. + - Returns an array of *unique* expression values for each group. + Order of the array elements is undefined. - Available in :pipeline:`$group` stage only. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. * - :expression:`$allElementsTrue` @@ -241,9 +248,9 @@ Alphabetical Listing of Expression Operators - Returns an average of numerical values. Ignores non-numeric values. - .. versionchanged:: 3.2 - Available in both :pipeline:`$group` and :pipeline:`$project` - stages. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. * - :expression:`$binarySize` @@ -300,11 +307,44 @@ Alphabetical Listing of Expression Operators * - :expression:`$cos` - Returns the cosine of a value that is measured in radians. - + + * - :expression:`$cosh` - Returns the hyperbolic cosine of a value that is measured in radians. + + + * - :group:`$count` + + - Returns the number of documents in the group or window. + + .. versionadded:: 5.0 + + Available in :pipeline:`$group` and + :pipeline:`$setWindowFields` stages. + + Distinct from the :pipeline:`$count` pipeline stage. + + + * - :group:`$covariancePop` + + - Returns the population covariance of two numeric + :ref:`expressions `. + + .. versionadded:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + + * - :group:`$covarianceSamp` + + - Returns the sample covariance of two numeric + :ref:`expressions `. + + .. versionadded:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. * - :expression:`$dateAdd` @@ -313,7 +353,7 @@ Alphabetical Listing of Expression Operators * - :expression:`$dateDiff` - Returns the difference between two dates. - + * - :expression:`$dateFromParts` - Constructs a BSON Date object given the date's constituent @@ -360,12 +400,42 @@ Alphabetical Listing of Expression Operators - Converts a value from degrees to radians. + * - :group:`$denseRank` + + - Returns the relative position of a sorted document. There are + no gaps in the positions. + + .. versionadded:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + + * - :group:`$derivative` + + - Returns the mathematical derivative for the numeric + :ref:`expression `. + + .. versionadded:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :expression:`$divide` - Returns the result of dividing the first number by the second. Accepts two argument expressions. + * - :group:`$documentNumber` + + - Returns the position of a document (known as the document number) + in the :ref:`partition `. + + .. versionadded:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :expression:`$eq` - Returns ``true`` if the values are @@ -376,6 +446,17 @@ Alphabetical Listing of Expression Operators - Raises *e* to the specified exponent. + + * - :group:`$expMovingAvg` + + - Returns the exponential moving average for the numeric + :ref:`expression `. + + .. versionadded:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :expression:`$filter` - Selects a subset of the array to return an array with only the elements @@ -385,10 +466,14 @@ Alphabetical Listing of Expression Operators * - :group:`$first` - Returns a value from the first document for each group. Order is - only defined if the documents are in a defined order. + only defined if the documents are sorted. - Available in :pipeline:`$group` stage only. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + Distinct from the :expression:`$first` array operator. + * - :expression:`$first` @@ -396,6 +481,7 @@ Alphabetical Listing of Expression Operators .. versionadded:: 4.4 + Distinct from the :group:`$first` accumulator. * - :expression:`$floor` @@ -459,6 +545,16 @@ Alphabetical Listing of Expression Operators substring is not found, returns ``-1``. + * - :group:`$integral` + + - Returns the mathematical integral for the numeric + :ref:`expression `. + + .. versionadded:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :expression:`$isArray` - Determines if the operand is an array. Returns a boolean. @@ -494,9 +590,11 @@ Alphabetical Listing of Expression Operators * - :group:`$last` - Returns a value from the last document for each group. Order is - only defined if the documents are in a defined order. + only defined if the documents are sorted. - Available in :pipeline:`$group` stage only. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. Distinct from the :expression:`$last` array operator. @@ -504,11 +602,11 @@ Alphabetical Listing of Expression Operators * - :expression:`$last` - Returns the last array element. - - Distinct from the :group:`$last` accumulator. - + .. versionadded:: 4.4 + Distinct from the :group:`$last` accumulator. + * - :expression:`$let` @@ -565,9 +663,9 @@ Alphabetical Listing of Expression Operators - Returns the highest expression value for each group. - .. versionchanged:: 3.2 - Available in both :pipeline:`$group` and :pipeline:`$project` - stages. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. * - :expression:`$mergeObjects` @@ -585,9 +683,9 @@ Alphabetical Listing of Expression Operators - Returns the lowest expression value for each group. - .. versionchanged:: 3.2 - Available in both :pipeline:`$group` and :pipeline:`$project` - stages. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. * - :expression:`$millisecond` @@ -646,11 +744,15 @@ Alphabetical Listing of Expression Operators - Raises a number to the specified exponent. + * - :group:`$push` - - Returns an array of expression values for each group. + - Returns an array of expression values for documents in each + group. - Available in :pipeline:`$group` stage only. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. * - :expression:`$radiansToDegrees` @@ -667,7 +769,17 @@ Alphabetical Listing of Expression Operators - Outputs an array containing a sequence of integers according to user-defined inputs. + + + * - :group:`$rank` + + - Returns the document position (known as the rank) relative to + other sorted documents. + .. versionadded:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + * - :expression:`$reduce` @@ -677,22 +789,19 @@ Alphabetical Listing of Expression Operators * - :expression:`$regexFind` - - - Applies a regular expression (regex) to a string and returns + - Applies a regular expression (regex) to a string and returns information on the *first* matched substring. * - :expression:`$regexFindAll` - - - Applies a regular expression (regex) to a string and returns + - Applies a regular expression (regex) to a string and returns information on the all matched substrings. * - :expression:`$regexMatch` - - - Applies a regular expression (regex) to a string and returns + - Applies a regular expression (regex) to a string and returns a boolean that indicates if a match is found or not. @@ -773,6 +882,17 @@ Alphabetical Listing of Expression Operators - Returns a set with elements that appear in *any* of the input sets. + * - :group:`$shift` + + - Returns the value from an :ref:`expression + ` applied to a document in a specified + position relative to the current document in the output. + + .. versionadded:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. + + * - :expression:`$size` - Returns the number of elements in the array. Accepts a single @@ -809,18 +929,18 @@ Alphabetical Listing of Expression Operators - Returns the population standard deviation of the input values. - .. versionchanged:: 3.2 - Available in both :pipeline:`$group` and :pipeline:`$project` - stages. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. * - :group:`$stdDevSamp` - Returns the sample standard deviation of the input values. - .. versionchanged:: 3.2 - Available in both :pipeline:`$group` and :pipeline:`$project` - stages. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. * - :expression:`$strcasecmp` @@ -876,9 +996,9 @@ Alphabetical Listing of Expression Operators - Returns a sum of numerical values. Ignores non-numeric values. - .. versionchanged:: 3.2 - Available in both :pipeline:`$group` and :pipeline:`$project` - stages. + .. versionchanged:: 5.0 + + Available in :pipeline:`$setWindowFields` stage. * - :expression:`$switch` @@ -1012,9 +1132,15 @@ Alphabetical Listing of Expression Operators /reference/operator/aggregation/convert /reference/operator/aggregation/cos /reference/operator/aggregation/cosh +<<<<<<< HEAD /reference/operator/aggregation/count-accum /reference/operator/aggregation/dateAdd /reference/operator/aggregation/dateDiff +======= + /reference/operator/aggregation/count-accumulator + /reference/operator/aggregation/covariancePop + /reference/operator/aggregation/covarianceSamp +>>>>>>> f7253052c... DOCSP-10258 window functions /reference/operator/aggregation/dateFromParts /reference/operator/aggregation/dateSubtract /reference/operator/aggregation/dateToParts @@ -1024,9 +1150,13 @@ Alphabetical Listing of Expression Operators /reference/operator/aggregation/dayOfWeek /reference/operator/aggregation/dayOfYear /reference/operator/aggregation/degreesToRadians + /reference/operator/aggregation/denseRank + /reference/operator/aggregation/derivative /reference/operator/aggregation/divide + /reference/operator/aggregation/documentNumber /reference/operator/aggregation/eq /reference/operator/aggregation/exp + /reference/operator/aggregation/expMovingAvg /reference/operator/aggregation/filter /reference/operator/aggregation/first /reference/operator/aggregation/first-array-element @@ -1040,6 +1170,7 @@ Alphabetical Listing of Expression Operators /reference/operator/aggregation/indexOfArray /reference/operator/aggregation/indexOfBytes /reference/operator/aggregation/indexOfCP + /reference/operator/aggregation/integral /reference/operator/aggregation/isArray /reference/operator/aggregation/isNumber /reference/operator/aggregation/isoDayOfWeek @@ -1074,6 +1205,7 @@ Alphabetical Listing of Expression Operators /reference/operator/aggregation/radiansToDegrees /reference/operator/aggregation/rand /reference/operator/aggregation/range + /reference/operator/aggregation/rank /reference/operator/aggregation/reduce /reference/operator/aggregation/regexFind /reference/operator/aggregation/regexFindAll @@ -1090,6 +1222,7 @@ Alphabetical Listing of Expression Operators /reference/operator/aggregation/setIntersection /reference/operator/aggregation/setIsSubset /reference/operator/aggregation/setUnion + /reference/operator/aggregation/shift /reference/operator/aggregation/size /reference/operator/aggregation/sin /reference/operator/aggregation/sinh diff --git a/source/reference/operator/aggregation/addToSet.txt b/source/reference/operator/aggregation/addToSet.txt index 7226c1e9a76..43b523cc14e 100644 --- a/source/reference/operator/aggregation/addToSet.txt +++ b/source/reference/operator/aggregation/addToSet.txt @@ -13,18 +13,28 @@ $addToSet (aggregation) Definition ---------- +.. versionchanged:: 5.0 + .. group:: $addToSet - Returns an array of all *unique* values that results from applying - an expression to each document in a group of documents that share - the same group by key. Order of the elements in the output array is - unspecified. + :group:`$addToSet` returns an array of all *unique* values that + results from applying an expression to each document in a group. You + can use ``$addToSet`` in: + + - A :pipeline:`$group` stage, which groups documents that share the + same group key. + + - A :pipeline:`$setWindowFields` stage, which groups documents in a + :ref:`window `. + + The order of the elements in the returned array is unspecified. .. include:: /includes/extracts/fact-aggregation-accumulator-addToSet.rst :group:`$addToSet` has the following syntax: - .. code-block:: javascript + .. code-block:: none + :copyable: false { $addToSet: } @@ -38,12 +48,15 @@ If the value of the expression is an array, :group:`$addToSet` appends the whole array as a *single* element. If the value of the expression is a document, MongoDB determines that -the document is a duplicate if another document in the array matches -the to-be-added document exactly; i.e. the existing document has the -exact same fields and values in the exact same order. +the document is a duplicate if another document in the array matches the +to-be-added document exactly. Specifically, the existing document has +the exact same fields and values in the exact same order. + +Examples +-------- -Example -------- +Use in ``$group`` Stage +~~~~~~~~~~~~~~~~~~~~~~~ Consider a ``sales`` collection with the following documents: @@ -76,7 +89,76 @@ compute the list of unique items sold for each group: The operation returns the following results: .. code-block:: javascript + :copyable: false { "_id" : { "day" : 46, "year" : 2014 }, "itemsSold" : [ "xyz", "abc" ] } { "_id" : { "day" : 34, "year" : 2014 }, "itemsSold" : [ "xyz", "jkl" ] } { "_id" : { "day" : 1, "year" : 2014 }, "itemsSold" : [ "abc" ] } + +Use in ``$setWindowFields`` Stage +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. versionadded:: 5.0 + +.. include:: /includes/setWindowFields-example-collection.rst + +This example uses :group:`$addToSet` in the :pipeline:`$setWindowFields` +stage to output the unique cake ``type`` sales for each ``state``: + +.. code-block:: javascript + + db.cakeSales.aggregate( [ + { + $setWindowFields: { + partitionBy: "$state", + sortBy: { orderDate: 1 }, + output: { + cakeTypesForState: { + $addToSet: "$type", + window: { + documents: [ "unbounded", "current" ] + } + } + } + } + } + ] ) + +In the example: + +.. include:: /includes/setWindowFields-partition-sort-date.rst + +- ``output`` adds each unique cake ``type`` to the ``cakeTypesForState`` + array field using :group:`$addToSet` that is run in a :ref:`documents + ` window. + + The :ref:`window ` contains documents between + an ``unbounded`` lower limit and the ``current`` document. This means + :group:`$addToSet` returns an array containing the unique cake + ``type`` fields for the documents between the beginning of the + partition and the current document. + +In this example output, the cake ``type`` array for ``CA`` and ``WA`` is +shown in the ``cakeTypesForState`` field: + +.. code-block:: javascript + :copyable: false + + { "_id" : 4, "type" : "strawberry", "orderDate" : ISODate("2019-05-18T16:09:01Z"), + "state" : "CA", "price" : 41, "quantity" : 162, + "cakeTypesForState" : [ "strawberry" ] } + { "_id" : 0, "type" : "chocolate", "orderDate" : ISODate("2020-05-18T14:10:30Z"), + "state" : "CA", "price" : 13, "quantity" : 120, + "cakeTypesForState" : [ "strawberry", "chocolate" ] } + { "_id" : 2, "type" : "vanilla", "orderDate" : ISODate("2021-01-11T06:31:15Z"), + "state" : "CA", "price" : 12, "quantity" : 145, + "cakeTypesForState" : [ "strawberry", "vanilla", "chocolate" ] } + { "_id" : 5, "type" : "strawberry", "orderDate" : ISODate("2019-01-08T06:12:03Z"), + "state" : "WA", "price" : 43, "quantity" : 134, + "cakeTypesForState" : [ "strawberry" ] } + { "_id" : 3, "type" : "vanilla", "orderDate" : ISODate("2020-02-08T13:13:23Z"), + "state" : "WA", "price" : 13, "quantity" : 104, + "cakeTypesForState" : [ "vanilla", "strawberry" ] } + { "_id" : 1, "type" : "chocolate", "orderDate" : ISODate("2021-03-20T11:30:05Z"), + "state" : "WA", "price" : 14, "quantity" : 140, + "cakeTypesForState" : [ "vanilla", "chocolate", "strawberry" ] } diff --git a/source/reference/operator/aggregation/avg.txt b/source/reference/operator/aggregation/avg.txt index 5451373c425..e166bf5ea27 100644 --- a/source/reference/operator/aggregation/avg.txt +++ b/source/reference/operator/aggregation/avg.txt @@ -13,6 +13,8 @@ $avg (aggregation) Definition ---------- +.. versionchanged:: 5.0 + .. group:: $avg Returns the average value of the numeric values. :group:`$avg` @@ -20,12 +22,18 @@ Definition .. include:: /includes/extracts/fact-aggregation-accumulator-avg.rst - When used in the :pipeline:`$group` stage, :group:`$avg` has the - following syntax and returns the collective average of all the numeric - values that result from applying a specified expression to each - document in a group of documents that share the same group by key: + When used in the :pipeline:`$group` or :pipeline:`$setWindowFields` + stage, :group:`$avg` has the following syntax and respectively + returns the collective average of all the numeric values from + applying an expression to each document in a: + + - Group of documents that share the same group by key. + + - Set of documents defined by a :ref:`window + `. - .. code-block:: javascript + .. code-block:: none + :copyable: false { $avg: } @@ -35,14 +43,16 @@ Definition - :group:`$avg` has one specified expression as its operand: - .. code-block:: javascript + .. code-block:: none + :copyable: false { $avg: } - :group:`$avg` has a list of specified expressions as its operand: - .. code-block:: javascript + .. code-block:: none + :copyable: false { $avg: [ , ... ] } @@ -105,6 +115,7 @@ average quantity for each grouping. The operation returns the following results: .. code-block:: javascript + :copyable: false { "_id" : "xyz", "avgAmount" : 37.5, "avgQuantity" : 7.5 } { "_id" : "jkl", "avgAmount" : 20, "avgQuantity" : 1 } @@ -134,9 +145,71 @@ average lab scores, and the average of the final and the midterm: The operation results in the following documents: .. code-block:: javascript + :copyable: false { "_id" : 1, "quizAvg" : 7.666666666666667, "labAvg" : 6.5, "examAvg" : 77.5 } { "_id" : 2, "quizAvg" : 9.5, "labAvg" : 8, "examAvg" : 87.5 } { "_id" : 3, "quizAvg" : 4.666666666666667, "labAvg" : 5.5, "examAvg" : 74 } -.. include:: /includes/extracts/fact-agg-accumulator-array-operand-in-project-avg.rst +Use in ``$setWindowFields`` Stage +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. versionadded:: 5.0 + +.. include:: /includes/setWindowFields-example-collection.rst + +This example uses :group:`$avg` in the :pipeline:`$setWindowFields` +stage to output the moving average for the cake sales ``quantity`` for +each ``state``: + +.. code-block:: javascript + + db.cakeSales.aggregate( [ + { + $setWindowFields: { + partitionBy: "$state", + sortBy: { orderDate: 1 }, + output: { + averageQuantityForState: { + $avg: "$quantity", + window: { + documents: [ "unbounded", "current" ] + } + } + } + } + } + ] ) + +In the example: + +.. include:: /includes/setWindowFields-partition-sort-date.rst + +- ``output`` sets the ``averageQuantityForState`` field to the moving + average ``quantity`` using :group:`$avg` for the documents in a + :ref:`documents ` window. + + The :ref:`window ` contains documents between + an ``unbounded`` lower limit and the ``current`` document in the + output. This means :group:`$avg` returns the moving average + ``quantity`` for the documents between the beginning of the partition + and the current document. + +In this output, the moving average ``quantity`` for ``CA`` and ``WA`` +is shown in the ``averageQuantityForState`` field: + +.. code-block:: javascript + :copyable: false + + { "_id" : 4, "type" : "strawberry", "orderDate" : ISODate("2019-05-18T16:09:01Z"), + "state" : "CA", "price" : 41, "quantity" : 162, "averageQuantityForState" : 162 } + { "_id" : 0, "type" : "chocolate", "orderDate" : ISODate("2020-05-18T14:10:30Z"), + "state" : "CA", "price" : 13, "quantity" : 120, "averageQuantityForState" : 141 } + { "_id" : 2, "type" : "vanilla", "orderDate" : ISODate("2021-01-11T06:31:15Z"), + "state" : "CA", "price" : 12, "quantity" : 145, "averageQuantityForState" : 142.33333333333334 } + { "_id" : 5, "type" : "strawberry", "orderDate" : ISODate("2019-01-08T06:12:03Z"), + "state" : "WA", "price" : 43, "quantity" : 134, "averageQuantityForState" : 134 } + { "_id" : 3, "type" : "vanilla", "orderDate" : ISODate("2020-02-08T13:13:23Z"), + "state" : "WA", "price" : 13, "quantity" : 104, "averageQuantityForState" : 119 } + { "_id" : 1, "type" : "chocolate", "orderDate" : ISODate("2021-03-20T11:30:05Z"), + "state" : "WA", "price" : 14, "quantity" : 140, "averageQuantityForState" : 126 } diff --git a/source/reference/operator/aggregation/count-accumulator.txt b/source/reference/operator/aggregation/count-accumulator.txt new file mode 100644 index 00000000000..49be7222b09 --- /dev/null +++ b/source/reference/operator/aggregation/count-accumulator.txt @@ -0,0 +1,155 @@ +================================ +$count (aggregation accumulator) +================================ + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +Definition +---------- + +.. versionadded:: 5.0 + +.. group:: $count + +Returns the number of documents. + +.. admonition:: Disambiguation + :class: note + + This page describes the ``$count`` aggregation accumulator. For the + ``$count`` aggregation pipeline stage, see :pipeline:`$count + (aggregation pipeline) <$count>`. + +:group:`$count` is available in the :pipeline:`$group` and +:pipeline:`$setWindowFields` stages: + +- When used in the :pipeline:`$group` stage, :group:`$count` + returns the number of documents that share the same group + by key. For an example, see :ref:`count-accumulator-group-example`. + +- When used in the :pipeline:`$setWindowFields` stage, :group:`$count` + returns the number of documents that are in the same :ref:`window + `. For an example, see + :ref:`count-accumulator-window-example`. + +:group:`$count` syntax, which is the same for the stages: + +.. code-block:: none + :copyable: false + + { $count: { } } + +:group:`$count` does not accept any parameters. + +Examples +-------- + +.. include:: /includes/setWindowFields-example-collection.rst + +The ``cakeSales`` collection is used in the following examples. + +.. _count-accumulator-group-example: + +Use in ``$group`` Stage +~~~~~~~~~~~~~~~~~~~~~~~ + +This example uses :group:`$count` in the :pipeline:`$group` stage to +count the number of documents in the ``cakeSales`` collection for each +``state``: + +.. code-block:: javascript + + db.cakeSales.aggregate( [ + { + $group: { + _id: "$state", + countNumberOfDocumentsForState: { + $count: {} + } + } + } + ] ) + +In the example: + +- ``_id: "$state"`` groups the documents by the ``state`` field value. + There are groups for ``CA`` and ``WA``. + +- ``$count: {}`` sets the ``countNumberOfDocumentsForState`` field to + the number of documents that share the same ``state`` field value. + +In this output, the number of documents for ``CA`` and ``WA`` is shown +in the ``countNumberOfDocumentsForState`` field: + +.. code-block:: javascript + :copyable: false + + { "_id" : "CA", "countNumberOfDocumentsForState" : 3 } + { "_id" : "WA", "countNumberOfDocumentsForState" : 3 } + +.. _count-accumulator-window-example: + +Use in ``$setWindowFields`` Stage +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This example uses :group:`$count` in the :pipeline:`$setWindowFields` +stage to count the number of documents in the ``cakeSales`` collection +for each ``state`` defined in the :ref:`window +`: + +.. code-block:: javascript + + db.cakeSales.aggregate( [ + { + $setWindowFields: { + partitionBy: "$state", + sortBy: { orderDate: 1 }, + output: { + countNumberOfDocumentsForState: { + $count: {}, + window: { + documents: [ "unbounded", "current" ] + } + } + } + } + } + ] ) + +In the example: + +.. include:: /includes/setWindowFields-partition-sort-date.rst + +- ``output`` sets the ``countNumberOfDocumentsForState`` field to the + number of documents using :group:`$count` that is run in a + :ref:`documents ` window. + + The :ref:`window ` contains documents between + an ``unbounded`` lower limit and the ``current`` document in the + output. This means :group:`$count` returns the number of documents + between the beginning of the partition and the current document. + +In this output, the number of documents for ``CA`` and ``WA`` is shown +in the ``countNumberOfDocumentsForState`` field: + +.. code-block:: javascript + :copyable: false + + { "_id" : 4, "type" : "strawberry", "orderDate" : ISODate("2019-05-18T16:09:01Z"), + "state" : "CA", "price" : 41, "quantity" : 162, "countNumberOfDocumentsForState" : 1 } + { "_id" : 0, "type" : "chocolate", "orderDate" : ISODate("2020-05-18T14:10:30Z"), + "state" : "CA", "price" : 13, "quantity" : 120, "countNumberOfDocumentsForState" : 2 } + { "_id" : 2, "type" : "vanilla", "orderDate" : ISODate("2021-01-11T06:31:15Z"), + "state" : "CA", "price" : 12, "quantity" : 145, "countNumberOfDocumentsForState" : 3 } + { "_id" : 5, "type" : "strawberry", "orderDate" : ISODate("2019-01-08T06:12:03Z"), + "state" : "WA", "price" : 43, "quantity" : 134, "countNumberOfDocumentsForState" : 1 } + { "_id" : 3, "type" : "vanilla", "orderDate" : ISODate("2020-02-08T13:13:23Z"), + "state" : "WA", "price" : 13, "quantity" : 104, "countNumberOfDocumentsForState" : 2 } + { "_id" : 1, "type" : "chocolate", "orderDate" : ISODate("2021-03-20T11:30:05Z"), + "state" : "WA", "price" : 14, "quantity" : 140, "countNumberOfDocumentsForState" : 3 } diff --git a/source/reference/operator/aggregation/count.txt b/source/reference/operator/aggregation/count.txt index 27dc00457e0..391b0eae446 100644 --- a/source/reference/operator/aggregation/count.txt +++ b/source/reference/operator/aggregation/count.txt @@ -20,6 +20,13 @@ Definition Passes a document to the next stage that contains a count of the number of documents input to the stage. + .. admonition:: Disambiguation + :class: note + + This page describes the ``$count`` aggregation pipeline stage. + For the ``$count`` aggregation accumulator, see :group:`$count + (aggregation accumulator) <$count>`. + :pipeline:`$count` has the following prototype form: .. code-block:: javascript diff --git a/source/reference/operator/aggregation/covariancePop.txt b/source/reference/operator/aggregation/covariancePop.txt new file mode 100644 index 00000000000..c7188b12c02 --- /dev/null +++ b/source/reference/operator/aggregation/covariancePop.txt @@ -0,0 +1,113 @@ +============================ +$covariancePop (aggregation) +============================ + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +Definition +---------- + +.. versionadded:: 5.0 + +.. group:: $covariancePop + +Returns the population covariance of two numeric :ref:`expressions +` that are evaluated using documents in the +:pipeline:`$setWindowFields` stage :ref:`window +`. + +:group:`$covariancePop` is only available in the +:pipeline:`$setWindowFields` stage. + +:group:`$covariancePop` syntax: + +.. code-block:: none + :copyable: false + + { + $covariancePop: { + [ + , + + ] + } + } + +Behavior +-------- + +:group:`$covariancePop` behavior: + +.. |covariance-method| replace:: :group:`$covarianceSamp` +.. |covariance-value| replace:: ``null`` + +.. include:: /includes/covariance-behavior.rst + +Example +------- + +.. include:: /includes/setWindowFields-example-collection.rst + +This example uses :group:`$covariancePop` in the +:pipeline:`$setWindowFields` stage to output the population covariance +values for the cake sales ``orderDate`` year and ``quantity`` values: + +.. code-block:: javascript + + db.cakeSales.aggregate( [ + { + $setWindowFields: { + partitionBy: "$state", + sortBy: { orderDate: 1 }, + output: { + covariancePopForState: { + $covariancePop: [ { $year: "$orderDate" }, "$quantity" ], + window: { + documents: [ "unbounded", "current" ] + } + } + } + } + } + ] ) + +In the example: + +.. include:: /includes/setWindowFields-partition-sort-date.rst + +- ``output`` sets the population covariance values for the + ``orderDate`` year and ``quantity`` values using + :group:`$covariancePop` run in a :ref:`documents + ` window. + + The :ref:`window ` contains documents between + an ``unbounded`` lower limit and the ``current`` document in the + output. This means :group:`$covariancePop` sets the + ``covariancePopForState`` field to the population covariance values + for the documents between the beginning of the partition and the + current document. + +In this output, the population covariance is shown in the +``covariancePopForState`` field: + +.. code-block:: javascript + :copyable: false + + { "_id" : 4, "type" : "strawberry", "orderDate" : ISODate("2019-05-18T16:09:01Z"), + "state" : "CA", "price" : 41, "quantity" : 162, "covariancePopForState" : 0 } + { "_id" : 0, "type" : "chocolate", "orderDate" : ISODate("2020-05-18T14:10:30Z"), + "state" : "CA", "price" : 13, "quantity" : 120, "covariancePopForState" : -10.5 } + { "_id" : 2, "type" : "vanilla", "orderDate" : ISODate("2021-01-11T06:31:15Z"), + "state" : "CA", "price" : 12, "quantity" : 145, "covariancePopForState" : -5.666666666666671 } + { "_id" : 5, "type" : "strawberry", "orderDate" : ISODate("2019-01-08T06:12:03Z"), + "state" : "WA", "price" : 43, "quantity" : 134, "covariancePopForState" : 0 } + { "_id" : 3, "type" : "vanilla", "orderDate" : ISODate("2020-02-08T13:13:23Z"), + "state" : "WA", "price" : 13, "quantity" : 104, "covariancePopForState" : -7.5 } + { "_id" : 1, "type" : "chocolate", "orderDate" : ISODate("2021-03-20T11:30:05Z"), + "state" : "WA", "price" : 14, "quantity" : 140, "covariancePopForState" : 2 } diff --git a/source/reference/operator/aggregation/covarianceSamp.txt b/source/reference/operator/aggregation/covarianceSamp.txt new file mode 100644 index 00000000000..85068eb510f --- /dev/null +++ b/source/reference/operator/aggregation/covarianceSamp.txt @@ -0,0 +1,113 @@ +============================= +$covarianceSamp (aggregation) +============================= + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +Definition +---------- + +.. versionadded:: 5.0 + +.. group:: $covarianceSamp + +Returns the sample covariance of two numeric :ref:`expressions +` that are evaluated using documents in the +:pipeline:`$setWindowFields` stage :ref:`window +`. + +:group:`$covarianceSamp` is only available in the +:pipeline:`$setWindowFields` stage. + +:group:`$covarianceSamp` syntax: + +.. code-block:: none + :copyable: false + + { + $covarianceSamp: { + [ + , + + ] + } + } + +Behavior +-------- + +:group:`$covarianceSamp` behavior: + +.. |covariance-method| replace:: :group:`$covariancePop` +.. |covariance-value| replace:: ``0`` + +.. include:: /includes/covariance-behavior.rst + +Example +------- + +.. include:: /includes/setWindowFields-example-collection.rst + +This example uses :group:`$covarianceSamp` in the +:pipeline:`$setWindowFields` stage to output the sample covariance +values for the cake sales ``orderDate`` year and ``quantity`` values: + +.. code-block:: javascript + + db.cakeSales.aggregate( [ + { + $setWindowFields: { + partitionBy: "$state", + sortBy: { orderDate: 1 }, + output: { + covarianceSampForState: { + $covarianceSamp: [ { $year: "$orderDate" }, "$quantity" ], + window: { + documents: [ "unbounded", "current" ] + } + } + } + } + } + ] ) + +In the example: + +.. include:: /includes/setWindowFields-partition-sort-date.rst + +- ``output`` sets the sample covariance values for the + ``orderDate`` year and ``quantity`` values using + :group:`$covarianceSamp` run in a :ref:`documents + ` window. + + The :ref:`window ` contains documents between + an ``unbounded`` lower limit and the ``current`` document in the + output. This means :group:`$covarianceSamp` sets the + ``covarianceSampForState`` field to the sample covariance values + for the documents between the beginning of the partition and the + current document. + +In this output, the sample covariance is shown in the +``covarianceSampForState`` field: + +.. code-block:: javascript + :copyable: false + + { "_id" : 4, "type" : "strawberry", "orderDate" : ISODate("2019-05-18T16:09:01Z"), + "state" : "CA", "price" : 41, "quantity" : 162, "covarianceSampForState" : null } + { "_id" : 0, "type" : "chocolate", "orderDate" : ISODate("2020-05-18T14:10:30Z"), + "state" : "CA", "price" : 13, "quantity" : 120, "covarianceSampForState" : -21 } + { "_id" : 2, "type" : "vanilla", "orderDate" : ISODate("2021-01-11T06:31:15Z"), + "state" : "CA", "price" : 12, "quantity" : 145, "covarianceSampForState" : -8.500000000000007 } + { "_id" : 5, "type" : "strawberry", "orderDate" : ISODate("2019-01-08T06:12:03Z"), + "state" : "WA", "price" : 43, "quantity" : 134, "covarianceSampForState" : null } + { "_id" : 3, "type" : "vanilla", "orderDate" : ISODate("2020-02-08T13:13:23Z"), + "state" : "WA", "price" : 13, "quantity" : 104, "covarianceSampForState" : -15 } + { "_id" : 1, "type" : "chocolate", "orderDate" : ISODate("2021-03-20T11:30:05Z"), + "state" : "WA", "price" : 14, "quantity" : 140, "covarianceSampForState" : 3 } diff --git a/source/reference/operator/aggregation/denseRank.txt b/source/reference/operator/aggregation/denseRank.txt new file mode 100644 index 00000000000..b44836822b1 --- /dev/null +++ b/source/reference/operator/aggregation/denseRank.txt @@ -0,0 +1,231 @@ +======================== +$denseRank (aggregation) +======================== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +Definition +---------- + +.. versionadded:: 5.0 + +.. group:: $denseRank + +Returns the document position (known as the rank) relative to other +documents in the :pipeline:`$setWindowFields` stage :ref:`partition +`. + +The :pipeline:`$setWindowFields` stage :ref:`sortBy +` field value determines the document rank. For +more information on how MongoDB compares fields with different types, +see :ref:`BSON comparison order `. + +If multiple documents occupy the same rank, :group:`$denseRank` places +the document with the subsequent value at the next rank without any gaps +(see :ref:`denseRank-behavior`). + +:group:`$denseRank` is only available in the +:pipeline:`$setWindowFields` stage. + +:group:`$denseRank` syntax: + +.. code-block:: none + :copyable: false + + { $denseRank: { } } + +:group:`$denseRank` does not accept any parameters. + +.. seealso:: + + :group:`$rank` + +.. _denseRank-behavior: + +Behavior +-------- + +.. include:: /includes/rank-and-denseRank-behavior.rst + +See the example in +:ref:`denseRank-duplicate-null-missing-values-example`. + +Examples +-------- + +Dense Rank Partitions by an Integer Field +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/setWindowFields-example-collection.rst + +This example uses :group:`$denseRank` in the +:pipeline:`$setWindowFields` stage to output the ``quantity`` dense rank +of the cake sales for each ``state``: + +.. code-block:: javascript + + db.cakeSales.aggregate( [ + { + $setWindowFields: { + partitionBy: "$state", + sortBy: { quantity: -1 }, + output: { + denseRankQuantityForState: { + $denseRank: {} + } + } + } + } + ] ) + +In the example: + +.. include:: /includes/setWindowFields-partition-sort-quantity.rst + +- ``output`` sets the ``denseRankOrderDateForState`` field to the + ``orderDate`` dense rank using :group:`$denseRank`, as shown in the + following results. + +.. code-block:: javascript + :copyable: false + + { "_id" : 4, "type" : "strawberry", "orderDate" : ISODate("2019-05-18T16:09:01Z"), + "state" : "CA", "price" : 41, "quantity" : 162, "denseRankQuantityForState" : 1 } + { "_id" : 2, "type" : "vanilla", "orderDate" : ISODate("2021-01-11T06:31:15Z"), + "state" : "CA", "price" : 12, "quantity" : 145, "denseRankQuantityForState" : 2 } + { "_id" : 0, "type" : "chocolate", "orderDate" : ISODate("2020-05-18T14:10:30Z"), + "state" : "CA", "price" : 13, "quantity" : 120, "denseRankQuantityForState" : 3 } + { "_id" : 1, "type" : "chocolate", "orderDate" : ISODate("2021-03-20T11:30:05Z"), + "state" : "WA", "price" : 14, "quantity" : 140, "denseRankQuantityForState" : 1 } + { "_id" : 5, "type" : "strawberry", "orderDate" : ISODate("2019-01-08T06:12:03Z"), + "state" : "WA", "price" : 43, "quantity" : 134, "denseRankQuantityForState" : 2 } + { "_id" : 3, "type" : "vanilla", "orderDate" : ISODate("2020-02-08T13:13:23Z"), + "state" : "WA", "price" : 13, "quantity" : 104, "denseRankQuantityForState" : 3 } + +Dense Rank Partitions by a Date Field +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This example shows how to use dates with :group:`$denseRank` in the +:pipeline:`$setWindowFields` stage to output the ``orderDate`` dense +rank of the cake sales for each ``state``: + +.. code-block:: javascript + + db.cakeSales.aggregate( [ + { + $setWindowFields: { + partitionBy: "$state", + sortBy: { orderDate: 1 }, + output: { + denseRankOrderDateForState: { + $denseRank: {} + } + } + } + } + ] ) + +In the example: + +.. include:: /includes/setWindowFields-partition-sort-date.rst + +- ``output`` sets the ``denseRankOrderDateForState`` field to the + ``orderDate`` rank using :group:`$denseRank`, as shown in the + following results. + +.. code-block:: javascript + :copyable: false + + { "_id" : 4, "type" : "strawberry", "orderDate" : ISODate("2019-05-18T16:09:01Z"), + "state" : "CA", "price" : 41, "quantity" : 162, "denseRankOrderDateForState" : 1 } + { "_id" : 0, "type" : "chocolate", "orderDate" : ISODate("2020-05-18T14:10:30Z"), + "state" : "CA", "price" : 13, "quantity" : 120, "denseRankOrderDateForState" : 2 } + { "_id" : 2, "type" : "vanilla", "orderDate" : ISODate("2021-01-11T06:31:15Z"), + "state" : "CA", "price" : 12, "quantity" : 145, "denseRankOrderDateForState" : 3 } + { "_id" : 5, "type" : "strawberry", "orderDate" : ISODate("2019-01-08T06:12:03Z"), + "state" : "WA", "price" : 43, "quantity" : 134, "denseRankOrderDateForState" : 1 } + { "_id" : 3, "type" : "vanilla", "orderDate" : ISODate("2020-02-08T13:13:23Z"), + "state" : "WA", "price" : 13, "quantity" : 104, "denseRankOrderDateForState" : 2 } + { "_id" : 1, "type" : "chocolate", "orderDate" : ISODate("2021-03-20T11:30:05Z"), + "state" : "WA", "price" : 14, "quantity" : 140, "denseRankOrderDateForState" : 3 } + +.. _denseRank-duplicate-null-missing-values-example: + +Dense Rank for Duplicate, Null, and Missing Values +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/setWindowFields-duplicates-example-collection.rst + +This example uses :group:`$denseRank` in the +:pipeline:`$setWindowFields` stage to output the ``quantity`` dense rank +from the ``cakeSalesWithDuplicates`` collection for each ``state``: + +.. code-block:: javascript + + db.cakeSalesWithDuplicates.aggregate( [ + { + $setWindowFields: { + partitionBy: "$state", + sortBy: { quantity: -1 }, + output: { + denseRankQuantityForState: { + $denseRank: {} + } + } + } + } + ] ) + +In the example: + +.. include:: /includes/setWindowFields-partition-sort-quantity.rst + +- ``output`` sets the ``denseRankQuantityForState`` field to the + ``quantity`` dense rank using :group:`$denseRank`. + +In the following example output: + +- The documents with the same ``quantity`` and ``state`` have the same + rank and there is no gap between the ranks. This differs from + :group:`$rank` that has a gap between the ranks (for an example, see + :ref:`rank-duplicate-null-missing-values-example`). + +- The document with the ``null`` ``quantity`` and then the document with + the missing ``quantity`` are ranked the lowest in the output for the + ``CA`` partition. This sorting is the result of the :ref:`BSON + comparison order `, which sorts ``null`` + and missing values after number values in this example. + +.. code-block:: javascript + :copyable: false + + { "_id" : 4, "type" : "strawberry", "orderDate" : ISODate("2019-05-18T16:09:01Z"), + "state" : "CA", "price" : 41, "quantity" : 162, "denseRankQuantityForState" : 1 } + { "_id" : 9, "type" : "strawberry", "orderDate" : ISODate("2020-05-11T16:09:01Z"), + "state" : "CA", "price" : 39, "quantity" : 162, "denseRankQuantityForState" : 1 } + { "_id" : 2, "type" : "vanilla", "orderDate" : ISODate("2021-01-11T06:31:15Z"), + "state" : "CA", "price" : 12, "quantity" : 145, "denseRankQuantityForState" : 2 } + { "_id" : 0, "type" : "chocolate", "orderDate" : ISODate("2020-05-18T14:10:30Z"), + "state" : "CA", "price" : 13, "quantity" : 120, "denseRankQuantityForState" : 3 } + { "_id" : 10, "type" : "strawberry", "orderDate" : ISODate("2020-05-11T16:09:01Z"), + "state" : "CA", "price" : 39, "quantity" : null, "denseRankQuantityForState" : 4 } + { "_id" : 11, "type" : "strawberry", "orderDate" : ISODate("2020-05-11T16:09:01Z"), + "state" : "CA", "price" : 39, "denseRankQuantityForState" : 5 } + { "_id" : 1, "type" : "chocolate", "orderDate" : ISODate("2021-03-20T11:30:05Z"), + "state" : "WA", "price" : 14, "quantity" : 140, "denseRankQuantityForState" : 1 } + { "_id" : 5, "type" : "strawberry", "orderDate" : ISODate("2019-01-08T06:12:03Z"), + "state" : "WA", "price" : 43, "quantity" : 134, "denseRankQuantityForState" : 2 } + { "_id" : 6, "type" : "strawberry", "orderDate" : ISODate("2020-01-08T06:12:03Z"), + "state" : "WA", "price" : 41, "quantity" : 134, "denseRankQuantityForState" : 2 } + { "_id" : 7, "type" : "strawberry", "orderDate" : ISODate("2020-01-01T06:12:03Z"), + "state" : "WA", "price" : 34, "quantity" : 134, "denseRankQuantityForState" : 2 } + { "_id" : 8, "type" : "strawberry", "orderDate" : ISODate("2020-01-02T06:12:03Z"), + "state" : "WA", "price" : 40, "quantity" : 134, "denseRankQuantityForState" : 2 } + { "_id" : 3, "type" : "vanilla", "orderDate" : ISODate("2020-02-08T13:13:23Z"), + "state" : "WA", "price" : 13, "quantity" : 104, "denseRankQuantityForState" : 3 } diff --git a/source/reference/operator/aggregation/derivative.txt b/source/reference/operator/aggregation/derivative.txt new file mode 100644 index 00000000000..d2fa153360d --- /dev/null +++ b/source/reference/operator/aggregation/derivative.txt @@ -0,0 +1,188 @@ +========================= +$derivative (aggregation) +========================= + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +Definition +---------- + +.. versionadded:: 5.0 + +.. group:: $derivative + +Returns the mathematical derivative value, which is calculated using +the: + +- First and last documents in the :pipeline:`$setWindowFields` stage + :ref:`window `. + +- Numerator, which is set to the result of subtracting the numeric + :ref:`expression ` value for the first + document from the :ref:`expression ` value + for the last document. + +- Denominator, which is set to the result of subtracting the + :ref:`sortBy ` field value for the first + document from the :ref:`sortBy ` field value + for the last document. + +:group:`$derivative` is only available in the +:pipeline:`$setWindowFields` stage. You must specify a :ref:`window +` in the :pipeline:`$setWindowFields` stage when +using :group:`$derivative`. + +:group:`$derivative` syntax: + +.. code-block:: none + :copyable: false + + { + $derivative: { + input: , + unit: