@@ -17,9 +17,15 @@ Set Granularity for Time Series Data
17
17
:keywords: Time series, granularity, IOT
18
18
19
19
When you create a :ref:`time series collection
20
- <manual-timeseries-collection>`, MongoDB automatically creates a ``system.buckets``
21
- :ref:`system collection <metadata-system-collections>` based on the ``granularity`` value you specify, and groups
22
- documents into those buckets.
20
+ <manual-timeseries-collection>`, MongoDB automatically creates a
21
+ ``system.buckets`` :ref:`system collection
22
+ <metadata-system-collections>` and groups incoming time series data
23
+ into buckets. By setting granularity, you control how
24
+ frequently data is bucketed based on the ingestion rate of your data.
25
+
26
+ Starting in MongoDB 6.3, you can use the custom bucketing parameters
27
+ ``bucketMaxSpanSeconds`` and ``bucketRoundingSeconds`` to specify bucket
28
+ boundaries and more accurately control how time series data is bucketed.
23
29
24
30
.. note::
25
31
@@ -28,83 +34,19 @@ documents into those buckets.
28
34
created. See :ref:`MongoDB 5.0 known issues
29
35
<5.0-known-issue-granularity>`.
30
36
31
- When you create a :ref:`time series collection
32
- <manual-timeseries-collection>`, you can specify the
33
- ``granularity`` parameter. The ``granularity`` determines how data
34
- in the time series collection is stored internally. For details about
35
- how MongoDB stores time series data, see :ref:`flexible-bucketing`.
36
-
37
- By default, ``granularity`` is set to ``"seconds"``. To optimize
38
- data storage, set the ``granularity`` to a value that is closest
39
- to the ingestion rate for your data source as specified by
40
- the ``metaField`` field. The ingestion rate is the time span between
41
- consecutive incoming measurements that have the same unique value for
42
- the ``metaField``.
43
-
44
- Consider the following example:
45
-
46
- .. code-block:: javascript
47
-
48
- db.createCollection(
49
- "weather24h",
50
- {
51
- timeseries: {
52
- timeField: "timestamp",
53
- metaField: "metadata",
54
- granularity: "minutes"
55
- },
56
- expireAfterSeconds: 86400
57
- }
58
- )
59
-
60
- If your ``metaField`` data identifies weather sensors and
61
- you ingest data from each individual sensor once every 5 minutes, you
62
- should choose ``"minutes"``. Even if you have thousands of sensors and
63
- the data coming in from different sensors is only seconds apart, the
64
- ``granularity`` should still be based on the ingestion rate for one
65
- sensor that is uniquely identified by its metadata.
66
-
67
- In the following table, you can see the max time span of data that is
68
- stored together for each ``granularity`` value:
69
-
70
- .. list-table::
71
- :header-rows: 1
72
- :widths: 40 60
73
-
74
- * - ``granularity``
37
+ Retrieve the Current bucketing parameters
38
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
75
39
76
- - Covered Time Span
77
-
78
- * - ``"seconds"`` (default)
79
-
80
- - one hour
81
-
82
- * - ``"minutes"``
83
-
84
- - 24 hours
85
-
86
- * - ``"hours"``
87
-
88
- - 30 days
89
-
90
-
91
- .. seealso::
92
-
93
- :ref:`Timing of Automatic Removal
94
- <timeseries-collection-delete-operations-timing>`
95
-
96
- Retrieve the ``granularity`` of a Time Series Collection
97
- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
98
-
99
- To retrieve the current value of ``granularity``, use the
40
+ To retrieve current collection values, use the
100
41
:dbcommand:`listCollections` command:
101
42
102
43
.. code-block:: javascript
103
44
104
45
db.runCommand( { listCollections: 1 } )
105
46
106
47
The result document contains a document for the time series collection
107
- which contains the ``options.timeseries.granularity`` field.
48
+ which contains ``options.timeseries.granularity``,
49
+ ``options.timeseries.bucketMaxSpanSeconds``, and ``options.timeseries.bucketRoundingSeconds`` fields, if present.
108
50
109
51
.. code-block:: javascript
110
52
:copyable: false
@@ -123,7 +65,8 @@ which contains the ``options.timeseries.granularity`` field.
123
65
timeField: <string>,
124
66
metaField: <string>,
125
67
granularity: <string>,
126
- bucketMaxSpanSeconds: <number>
68
+ bucketMaxSpanSeconds: <number>,
69
+ bucketRoundingSeconds: <number>
127
70
}
128
71
},
129
72
...
@@ -133,11 +76,9 @@ which contains the ``options.timeseries.granularity`` field.
133
76
}
134
77
}
135
78
136
- Change the ``granularity`` of a Time Series Collection
137
- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
138
79
139
- Using the " granularity" Field
140
- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
80
+ Using a Single `` granularity`` Field
81
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
141
82
142
83
The following table shows the maximum time interval included in one
143
84
bucket of data when using a given ``granularity`` value:
@@ -152,47 +93,48 @@ minutes, set ``granularity`` to ``"minutes"``.
152
93
153
94
.. code-block:: javascript
154
95
155
- db.runCommand({
156
- collMod: "weather24h",
157
- timeseries: { granularity: "hours" }
158
- })
96
+ db.createCollection(
97
+ "weather24h",
98
+ {
99
+ timeseries: {
100
+ timeField: "timestamp",
101
+ metaField: "metadata",
102
+ granularity: "minutes"
103
+ },
104
+ expireAfterSeconds: 86400
105
+ }
106
+ )
159
107
160
- Once set, you cannot decrease granularity. For example, if granularity
161
- is set to ``minutes``, you can increase it to ``hours``, but you cannot
162
- decrease it to ``seconds``.
108
+ Setting the ``granularity`` to ``hours`` groups up to a month's
109
+ worth of data ingest events into a single bucket, resulting in longer
110
+ traversal times and slower queries. Setting it to ``seconds``
111
+ leads to multiple buckets per polling interval, many of which
112
+ might contain only a single document.
163
113
164
- .. note ::
114
+ .. seealso ::
165
115
166
- You cannot modify the ``granularity`` of a sharded time series
167
- collection.
116
+ :ref:`Timing of Automatic Removal
117
+ <timeseries- collection-delete-operations-timing>`
168
118
169
119
.. _flexible-bucketing:
170
120
171
- Flexible Bucketing
172
- ~~~~~~~~~~~~~~~~~~
173
-
174
- MongoDB groups incoming time series data into buckets. By setting the
175
- ``granularity`` parameter, you control how frequently data is bucketed
176
- based on the ingestion rate of your data.
177
-
178
- Starting in MongoDB 6.3, you can specify the bucket boundaries to
179
- more accurately control how time series data is bucketed. Use the
180
- following parameters:
181
-
182
- - ``bucketMaxSpanSeconds``, the maximum time span between measurements
183
- in a bucket.
184
- - ``bucketRoundingSeconds``, the time interval that determines the
185
- starting timestamp for a new bucket.
186
-
187
- .. note::
121
+ Using Custom Bucketing Parameters
122
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
188
123
189
- If you want to specify the bucket boundaries:
124
+ Instead of ``granularity``, you can set bucket boundaries manually using
125
+ the two custom bucketing parameters. Set both parameters to the
126
+ same value, and do not set ``granularity``:
190
127
191
- - You must set both ``bucketMaxSpanSeconds`` and ``bucketRoundingSeconds``.
192
- - Both parameters must have the same value.
193
- - You can't additionally set the ``granularity`` parameter.
128
+ - ``bucketMaxSpanSeconds``, sets the maximum time between timestamps
129
+ in the same bucket. Possible values are 1-31536000.
130
+ - ``bucketRoundingSeconds``, the time interval that determines the
131
+ starting timestamp for a new bucket. When a document requires a new
132
+ bucket, MongoDB rounds down the document's timestamp value by this
133
+ interval to set the minimum time for the bucket.
194
134
195
- Consider the following time series collection:
135
+ For the weather station example with 5 minute sensor intervals, you
136
+ could fine tune bucketing by setting the custom bucketing parameters to
137
+ 300 seconds, instead of using a ``granularity`` of ``"minutes"``:
196
138
197
139
.. code-block:: javascript
198
140
@@ -202,20 +144,47 @@ Consider the following time series collection:
202
144
timeseries: {
203
145
timeField: "timestamp",
204
146
metaField: "metadata",
205
- bucketMaxSpanSeconds: 60 ,
206
- bucketRoundingSeconds: 60
147
+ bucketMaxSpanSeconds: 300 ,
148
+ bucketRoundingSeconds: 300
207
149
}
208
150
}
209
151
)
210
152
211
- Each bucket has a maximum time span of 60 seconds. When MongoDB opens a
212
- new bucket, the starting timestamp is rounded down to the nearest 60-second
213
- interval .
153
+ If a document with a time of ``2023-03-27T18:24:35Z`` does not fit an
154
+ existing bucket, MongoDB creates a new bucket with a minimum time of
155
+ ``2023-03-27T18:20:00Z`` and a maximum time of ``2023-03-27T18:24:59Z`` .
214
156
215
- For example, if you insert a new measurement into the collection with a
216
- timestamp of ``11:59:59``, the measurement is added to the bucket
217
- with boundaries between ``11:59:00`` and ``12:00:00`` (non-inclusive).
218
- If the bucket doesn't exist, the starting timestamp of the new bucket
219
- is determined by rounding ``11:59:59`` down to ``11:59:00``.
157
+ Change Time Series Granularity
158
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
159
+
160
+ You can increase ``timeseries.granularity`` from a shorter unit of time
161
+ to a longer one using a :dbcommand:`collMod` command.
162
+
163
+ .. note::
164
+
165
+ To modify the granularity of a **sharded** time series collection,
166
+ you must be running MongoDB 6.0 or later.
167
+
168
+ .. code-block:: javascript
169
+
170
+ db.runCommand({
171
+ collMod: "weather24h",
172
+ timeseries: { granularity: "seconds" || "minutes" || "hours" }
173
+ })
174
+
175
+ If you are using the custom bucketing parameters
176
+ ``bucketRoundingSeconds`` and ``bucketMaxSpanSeconds`` instead of
177
+ ``granularity``, include both custom parameters in the ``collMod``
178
+ command and set them to the same value:
179
+
180
+ .. code-block:: javascript
181
+
182
+ db.runCommand({
183
+ collMod: "weather24h",
184
+ timeseries: {
185
+ bucketRoundingSeconds: "86400",
186
+ bucketMaxSpanSeconds: "86400"
187
+ }
188
+ })
220
189
221
- For more information on time series parameters, see :ref:`time-series-fields` .
190
+ You cannot decrease the granularity interval or the custom bucketing values .
0 commit comments