Skip to content

Commit 7b9dc6a

Browse files
Initital stashed commit
1 parent d49af13 commit 7b9dc6a

File tree

3 files changed

+252
-155
lines changed

3 files changed

+252
-155
lines changed

source/core/timeseries/timeseries-best-practices.txt

Lines changed: 42 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -213,7 +213,45 @@ ratio.
213213
Optimize Query Performance
214214
--------------------------
215215

216-
To improve query performance,
217-
:ref:`create one or more secondary indexes <timeseries-add-secondary-index>`
218-
on your ``timeField`` and ``metaField`` to support common query
219-
patterns.
216+
Set Appropriate Bucket Granularity
217+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
218+
When you create a :ref:`time series collection
219+
<manual-timeseries-collection>`, MongoDB groups incoming time series
220+
data into buckets. By accurately setting granularity, you control how
221+
frequently data is bucketed based on the ingestion rate of your data.
222+
223+
Starting in MongoDB 6.3, you can use the custom bucketing parameters
224+
``bucketMaxSpanSeconds`` and ``bucketRoundingSeconds`` to specify bucket
225+
boundaries and more accurately control how time series data is bucketed.
226+
227+
You can improve performance by setting the ``granularity`` or custom
228+
bucketing parameters to the best match for the time span between
229+
incoming measurements from the same data source. For example, if you are
230+
recording weather data from thousands of sensors but only record data
231+
from each sensor once per 5 minutes, you can either set ``granularity``
232+
to ``"minutes"`` or set the custom bucketing parameters to ``300`` (seconds).
233+
234+
The following table shows the maximum time interval included in one
235+
bucket of data when using a given ``granularity`` value:
236+
237+
.. include:: /includes/table-timeseries-granularity-intervals.rst
238+
239+
In this case, setting the ``granularity`` to ``hours`` groups up to a
240+
month's worth of data ingest events into a single bucket, resulting in
241+
longer traversal times and slower queries. Setting it to ``seconds``
242+
leads to multiple buckets per polling interval, many of which
243+
might contain only a single document.
244+
245+
.. seealso::
246+
247+
:ref:`Timing of Automatic Removal
248+
<timeseries-collection-delete-operations-timing>`
249+
250+
Create Secondary Indexes
251+
~~~~~~~~~~~~~~~~~~~~~~~~
252+
253+
To improve query performance, :ref:`create one or more secondary indexes
254+
<timeseries-add-secondary-index>` on your ``timeField`` and
255+
``metaField`` to support common query patterns. In versions 6.3 and
256+
higher, MongoDB creates a secondary index on the ``timeField`` and
257+
``metaField`` automatically.

source/core/timeseries/timeseries-granularity.txt

Lines changed: 89 additions & 120 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,15 @@ Set Granularity for Time Series Data
1717
:keywords: Time series, granularity, IOT
1818

1919
When you create a :ref:`time series collection
20-
<manual-timeseries-collection>`, MongoDB automatically creates a ``system.buckets``
21-
:ref:`system collection <metadata-system-collections>` based on the ``granularity`` value you specify, and groups
22-
documents into those buckets.
20+
<manual-timeseries-collection>`, MongoDB automatically creates a
21+
``system.buckets`` :ref:`system collection
22+
<metadata-system-collections>` and groups incoming time series data
23+
into buckets. By setting granularity, you control how
24+
frequently data is bucketed based on the ingestion rate of your data.
25+
26+
Starting in MongoDB 6.3, you can use the custom bucketing parameters
27+
``bucketMaxSpanSeconds`` and ``bucketRoundingSeconds`` to specify bucket
28+
boundaries and more accurately control how time series data is bucketed.
2329

2430
.. note::
2531

@@ -28,83 +34,19 @@ documents into those buckets.
2834
created. See :ref:`MongoDB 5.0 known issues
2935
<5.0-known-issue-granularity>`.
3036

31-
When you create a :ref:`time series collection
32-
<manual-timeseries-collection>`, you can specify the
33-
``granularity`` parameter. The ``granularity`` determines how data
34-
in the time series collection is stored internally. For details about
35-
how MongoDB stores time series data, see :ref:`flexible-bucketing`.
36-
37-
By default, ``granularity`` is set to ``"seconds"``. To optimize
38-
data storage, set the ``granularity`` to a value that is closest
39-
to the ingestion rate for your data source as specified by
40-
the ``metaField`` field. The ingestion rate is the time span between
41-
consecutive incoming measurements that have the same unique value for
42-
the ``metaField``.
43-
44-
Consider the following example:
45-
46-
.. code-block:: javascript
47-
48-
db.createCollection(
49-
"weather24h",
50-
{
51-
timeseries: {
52-
timeField: "timestamp",
53-
metaField: "metadata",
54-
granularity: "minutes"
55-
},
56-
expireAfterSeconds: 86400
57-
}
58-
)
59-
60-
If your ``metaField`` data identifies weather sensors and
61-
you ingest data from each individual sensor once every 5 minutes, you
62-
should choose ``"minutes"``. Even if you have thousands of sensors and
63-
the data coming in from different sensors is only seconds apart, the
64-
``granularity`` should still be based on the ingestion rate for one
65-
sensor that is uniquely identified by its metadata.
66-
67-
In the following table, you can see the max time span of data that is
68-
stored together for each ``granularity`` value:
69-
70-
.. list-table::
71-
:header-rows: 1
72-
:widths: 40 60
73-
74-
* - ``granularity``
37+
Retrieve the Current bucketing parameters
38+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7539

76-
- Covered Time Span
77-
78-
* - ``"seconds"`` (default)
79-
80-
- one hour
81-
82-
* - ``"minutes"``
83-
84-
- 24 hours
85-
86-
* - ``"hours"``
87-
88-
- 30 days
89-
90-
91-
.. seealso::
92-
93-
:ref:`Timing of Automatic Removal
94-
<timeseries-collection-delete-operations-timing>`
95-
96-
Retrieve the ``granularity`` of a Time Series Collection
97-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
98-
99-
To retrieve the current value of ``granularity``, use the
40+
To retrieve current collection values, use the
10041
:dbcommand:`listCollections` command:
10142

10243
.. code-block:: javascript
10344

10445
db.runCommand( { listCollections: 1 } )
10546

10647
The result document contains a document for the time series collection
107-
which contains the ``options.timeseries.granularity`` field.
48+
which contains ``options.timeseries.granularity``,
49+
``options.timeseries.bucketMaxSpanSeconds``, and ``options.timeseries.bucketRoundingSeconds`` fields, if present.
10850

10951
.. code-block:: javascript
11052
:copyable: false
@@ -123,7 +65,8 @@ which contains the ``options.timeseries.granularity`` field.
12365
timeField: <string>,
12466
metaField: <string>,
12567
granularity: <string>,
126-
bucketMaxSpanSeconds: <number>
68+
bucketMaxSpanSeconds: <number>,
69+
bucketRoundingSeconds: <number>
12770
}
12871
},
12972
...
@@ -133,11 +76,9 @@ which contains the ``options.timeseries.granularity`` field.
13376
}
13477
}
13578

136-
Change the ``granularity`` of a Time Series Collection
137-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
13879

139-
Using the "granularity" Field
140-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
80+
Using a Single ``granularity`` Field
81+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
14182

14283
The following table shows the maximum time interval included in one
14384
bucket of data when using a given ``granularity`` value:
@@ -152,47 +93,48 @@ minutes, set ``granularity`` to ``"minutes"``.
15293

15394
.. code-block:: javascript
15495

155-
db.runCommand({
156-
collMod: "weather24h",
157-
timeseries: { granularity: "hours" }
158-
})
96+
db.createCollection(
97+
"weather24h",
98+
{
99+
timeseries: {
100+
timeField: "timestamp",
101+
metaField: "metadata",
102+
granularity: "minutes"
103+
},
104+
expireAfterSeconds: 86400
105+
}
106+
)
159107

160-
Once set, you cannot decrease granularity. For example, if granularity
161-
is set to ``minutes``, you can increase it to ``hours``, but you cannot
162-
decrease it to ``seconds``.
108+
Setting the ``granularity`` to ``hours`` groups up to a month's
109+
worth of data ingest events into a single bucket, resulting in longer
110+
traversal times and slower queries. Setting it to ``seconds``
111+
leads to multiple buckets per polling interval, many of which
112+
might contain only a single document.
163113

164-
.. note::
114+
.. seealso::
165115

166-
You cannot modify the ``granularity`` of a sharded time series
167-
collection.
116+
:ref:`Timing of Automatic Removal
117+
<timeseries-collection-delete-operations-timing>`
168118

169119
.. _flexible-bucketing:
170120

171-
Flexible Bucketing
172-
~~~~~~~~~~~~~~~~~~
173-
174-
MongoDB groups incoming time series data into buckets. By setting the
175-
``granularity`` parameter, you control how frequently data is bucketed
176-
based on the ingestion rate of your data.
177-
178-
Starting in MongoDB 6.3, you can specify the bucket boundaries to
179-
more accurately control how time series data is bucketed. Use the
180-
following parameters:
181-
182-
- ``bucketMaxSpanSeconds``, the maximum time span between measurements
183-
in a bucket.
184-
- ``bucketRoundingSeconds``, the time interval that determines the
185-
starting timestamp for a new bucket.
186-
187-
.. note::
121+
Using Custom Bucketing Parameters
122+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
188123

189-
If you want to specify the bucket boundaries:
124+
Instead of ``granularity``, you can set bucket boundaries manually using
125+
the two custom bucketing parameters. Set both parameters to the
126+
same value, and do not set ``granularity``:
190127

191-
- You must set both ``bucketMaxSpanSeconds`` and ``bucketRoundingSeconds``.
192-
- Both parameters must have the same value.
193-
- You can't additionally set the ``granularity`` parameter.
128+
- ``bucketMaxSpanSeconds``, sets the maximum time between timestamps
129+
in the same bucket. Possible values are 1-31536000.
130+
- ``bucketRoundingSeconds``, the time interval that determines the
131+
starting timestamp for a new bucket. When a document requires a new
132+
bucket, MongoDB rounds down the document's timestamp value by this
133+
interval to set the minimum time for the bucket.
194134

195-
Consider the following time series collection:
135+
For the weather station example with 5 minute sensor intervals, you
136+
could fine tune bucketing by setting the custom bucketing parameters to
137+
300 seconds, instead of using a ``granularity`` of ``"minutes"``:
196138

197139
.. code-block:: javascript
198140

@@ -202,20 +144,47 @@ Consider the following time series collection:
202144
timeseries: {
203145
timeField: "timestamp",
204146
metaField: "metadata",
205-
bucketMaxSpanSeconds: 60,
206-
bucketRoundingSeconds: 60
147+
bucketMaxSpanSeconds: 300,
148+
bucketRoundingSeconds: 300
207149
}
208150
}
209151
)
210152

211-
Each bucket has a maximum time span of 60 seconds. When MongoDB opens a
212-
new bucket, the starting timestamp is rounded down to the nearest 60-second
213-
interval.
153+
If a document with a time of ``2023-03-27T18:24:35Z`` does not fit an
154+
existing bucket, MongoDB creates a new bucket with a minimum time of
155+
``2023-03-27T18:20:00Z`` and a maximum time of ``2023-03-27T18:24:59Z``.
214156

215-
For example, if you insert a new measurement into the collection with a
216-
timestamp of ``11:59:59``, the measurement is added to the bucket
217-
with boundaries between ``11:59:00`` and ``12:00:00`` (non-inclusive).
218-
If the bucket doesn't exist, the starting timestamp of the new bucket
219-
is determined by rounding ``11:59:59`` down to ``11:59:00``.
157+
Change Time Series Granularity
158+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
159+
160+
You can increase ``timeseries.granularity`` from a shorter unit of time
161+
to a longer one using a :dbcommand:`collMod` command.
162+
163+
.. note::
164+
165+
To modify the granularity of a **sharded** time series collection,
166+
you must be running MongoDB 6.0 or later.
167+
168+
.. code-block:: javascript
169+
170+
db.runCommand({
171+
collMod: "weather24h",
172+
timeseries: { granularity: "seconds" || "minutes" || "hours" }
173+
})
174+
175+
If you are using the custom bucketing parameters
176+
``bucketRoundingSeconds`` and ``bucketMaxSpanSeconds`` instead of
177+
``granularity``, include both custom parameters in the ``collMod``
178+
command and set them to the same value:
179+
180+
.. code-block:: javascript
181+
182+
db.runCommand({
183+
collMod: "weather24h",
184+
timeseries: {
185+
bucketRoundingSeconds: "86400",
186+
bucketMaxSpanSeconds: "86400"
187+
}
188+
})
220189

221-
For more information on time series parameters, see :ref:`time-series-fields`.
190+
You cannot decrease the granularity interval or the custom bucketing values.

0 commit comments

Comments
 (0)