From 6638bc24dd931087261ecc2453434f5b369dd3a7 Mon Sep 17 00:00:00 2001 From: Nick Larew Date: Mon, 9 Oct 2017 13:23:26 -0600 Subject: [PATCH] DOCS-10546: random sampling for profiler/logging --- .../analyzing-mongodb-performance.txt | 75 ++---- .../apiargs-dbcommand-profile-fields.yaml | 73 ++++++ ...rgs-method-db.setProfilingLevel-param.yaml | 45 +++- source/includes/database-profiler-levels.rst | 17 ++ .../includes/dbcommand-slowms-definition.rst | 4 + source/includes/fact-log-slow-queries.rst | 7 + source/includes/options-conf.yaml | 51 ++++- source/includes/options-mongod.yaml | 104 +++------ .../includes/warning-profiler-performance.rst | 9 + source/reference/command/profile.txt | 77 +++---- source/reference/configuration-options.txt | 12 +- .../method/db.getProfilingStatus.txt | 6 +- .../reference/method/db.setProfilingLevel.txt | 24 +- source/reference/program/mongod.txt | 15 +- .../tutorial/manage-the-database-profiler.txt | 216 ++++++++++-------- 15 files changed, 426 insertions(+), 309 deletions(-) create mode 100644 source/includes/apiargs-dbcommand-profile-fields.yaml create mode 100644 source/includes/database-profiler-levels.rst create mode 100644 source/includes/dbcommand-slowms-definition.rst create mode 100644 source/includes/fact-log-slow-queries.rst create mode 100644 source/includes/warning-profiler-performance.rst diff --git a/source/administration/analyzing-mongodb-performance.txt b/source/administration/analyzing-mongodb-performance.txt index 1a0c370998f..e95dc7cd744 100644 --- a/source/administration/analyzing-mongodb-performance.txt +++ b/source/administration/analyzing-mongodb-performance.txt @@ -179,74 +179,29 @@ information. Database Profiling ------------------ -MongoDB's "Profiler" is a database profiling system that can help identify -inefficient queries and operations. +The :doc:`/tutorial/manage-the-database-profiler` collects detailed +information about operations run against a mongod instance. The +profiler's output can help to identify inefficient queries and +operations. -The following profiling levels are available: - -.. list-table:: - :header-rows: 1 - - * - **Level** - - - **Setting** - - * - 0 - - - Off. No profiling - - * - 1 - - - On. Only includes *"slow"* operations - - * - 2 - - - On. Includes *all* operations +You can enable and configure profiling for individual databases or for +all databases on a :program:`mongod` instance. +Profiler settings affect only a single :program:`mongod` instance and +will not propagate across a :term:`replica set` or :term:`sharded +cluster`. -Enable the profiler by setting the -:dbcommand:`profile` value using the following command in the -:program:`mongo` shell: +See :doc:`/tutorial/manage-the-database-profiler` for information on +enabling and configuring the profiler. -.. code-block:: javascript - - db.setProfilingLevel(1) - -The :setting:`~operationProfiling.slowOpThresholdMs` setting defines what constitutes a "slow" -operation. To set the threshold above which the profiler considers -operations "slow" (and thus, included in the level ``1`` profiling -data), you can configure :setting:`~operationProfiling.slowOpThresholdMs` at runtime as an argument to -the :method:`db.setProfilingLevel()` operation. +The following profiling levels are available: -.. see:: The documentation of :method:`db.setProfilingLevel()` for more - information. +.. include:: /includes/database-profiler-levels.rst -By default, :program:`mongod` records all "slow" queries to its -:setting:`log `, as defined by :setting:`~operationProfiling.slowOpThresholdMs`. +.. include:: /includes/warning-profiler-performance.rst .. note:: + .. include:: /includes/fact-log-slow-queries.rst - Because the database profiler can negatively impact - performance, only enable profiling for strategic intervals and as - minimally as possible on production systems. - - You may enable profiling on a per-:program:`mongod` basis. This - setting will not propagate across a :term:`replica set` or - :term:`sharded cluster`. - -You can view the output of the profiler in the ``system.profile`` -collection of your database by issuing the ``show profile`` command in -the :program:`mongo` shell, or with the following operation: - -.. code-block:: javascript - - db.system.profile.find( { millis : { $gt : 100 } } ) - -This returns all operations that lasted longer than 100 milliseconds. -Ensure that the value specified here (``100``, in this example) is above the -:setting:`~operationProfiling.slowOpThresholdMs` threshold. - -You must use the :operator:`$query` operator to access the ``query`` -field of documents within ``system.profile``. .. class:: hidden diff --git a/source/includes/apiargs-dbcommand-profile-fields.yaml b/source/includes/apiargs-dbcommand-profile-fields.yaml new file mode 100644 index 00000000000..5a8c947e2d7 --- /dev/null +++ b/source/includes/apiargs-dbcommand-profile-fields.yaml @@ -0,0 +1,73 @@ +position: 1 +name: profile +type: int +optional: false +description: | + Default: 0 + + | Specifies which operations should be profiled. + | The following profiler levels are available: + + .. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Level + - Description + + * - ``-1`` + - Returns the current profiler level, but does not change it. + + * - ``0`` + - Turns the profiler off so that it does not + collect any data. + This is the default profiler level. + + * - ``1`` + - Sets the profiler to collect data for operations that take + longer than the value of ``slowms``. + + * - ``2`` + - Sets the profiler to collect data for all operations. + +interface: dbcommand +operation: profile +arg_name: field +--- +position: 2 +name: slowms +type: int +optional: true +description: | + Default: 100 + + .. include:: /includes/dbcommand-slowms-definition.rst + + .. note:: + This argument affects the same setting as the configuration option + :setting:`~operationProfiling.slowOpThresholdMs`. + + +interface: dbcommand +operation: profile +arg_name: field +--- +position: 3 +name: sampleRate +type: double +optional: true +description: | + Default: 1.0 + + | The fraction of *slow* operations that should be profiled. + | ``sampleRate`` accepts values between 0 and 1, inclusive. + + .. note:: + This argument affects the same setting as the configuration option + :setting:`~operationProfiling.slowOpSampleRate`. + + .. versionadded:: 3.6 +interface: dbcommand +operation: profile +arg_name: field +... \ No newline at end of file diff --git a/source/includes/apiargs-method-db.setProfilingLevel-param.yaml b/source/includes/apiargs-method-db.setProfilingLevel-param.yaml index 23ea81c0420..fcf90da4e3c 100644 --- a/source/includes/apiargs-method-db.setProfilingLevel-param.yaml +++ b/source/includes/apiargs-method-db.setProfilingLevel-param.yaml @@ -1,6 +1,10 @@ arg_name: param -description: |- - Specifies a profiling level, which is either ``0`` for no profiling, ``1`` for only slow operations, or ``2`` for all operations. +description: | + Configures the profiler level. + The following profiler levels are available: + + .. include:: /includes/database-profiler-levels.rst + interface: method name: level operation: db.setProfilingLevel @@ -9,12 +13,41 @@ position: 1 type: integer --- arg_name: param -description: |- - Sets the threshold in milliseconds for the profile to consider a query or operation to be slow. +description: | + Accepts an integer or an options document. If an integer value is + passed as the ``options`` argument instead of a document, the value is + assigned to ``slowms``. + The following options are available: + + .. list-table:: + :stub-columns: 1 + :widths: 15 85 + + * - slowms + - | Default: 100 + | Type: integer + + .. include:: /includes/dbcommand-slowms-definition.rst + + .. note:: + This argument affects the same setting as the configuration + file option :setting:`~operationProfiling.slowOpThresholdMs`. + + * - sampleRate + - | Default: 1.0 + | Type: double + + The fraction of *slow* operations that should be profiled. + ``sampleRate`` accepts values between 0 and 1, inclusive. + + .. note:: + This argument affects the same setting as the configuration option + :setting:`~operationProfiling.slowOpSampleRate`. + interface: method -name: slowms +name: options operation: db.setProfilingLevel optional: true position: 2 -type: integer +type: document or integer ... diff --git a/source/includes/database-profiler-levels.rst b/source/includes/database-profiler-levels.rst new file mode 100644 index 00000000000..0bde42e0b74 --- /dev/null +++ b/source/includes/database-profiler-levels.rst @@ -0,0 +1,17 @@ +.. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Level + - Description + + * - ``0`` + - The profiler is off and does not collect any data. + This is the default profiler level. + + * - ``1`` + - The profiler collects data for operations that take longer + than the value of ``slowms``. + + * - ``2`` + - The profiler collects data for all operations. \ No newline at end of file diff --git a/source/includes/dbcommand-slowms-definition.rst b/source/includes/dbcommand-slowms-definition.rst new file mode 100644 index 00000000000..71107dc1d24 --- /dev/null +++ b/source/includes/dbcommand-slowms-definition.rst @@ -0,0 +1,4 @@ +The *slow* operation time threshold, in milliseconds. Operations +that run for longer than this threshold are considered *slow*. + +.. include:: /includes/fact-log-slow-queries.rst \ No newline at end of file diff --git a/source/includes/fact-log-slow-queries.rst b/source/includes/fact-log-slow-queries.rst new file mode 100644 index 00000000000..d23c1d532c6 --- /dev/null +++ b/source/includes/fact-log-slow-queries.rst @@ -0,0 +1,7 @@ +The system log and profiler are configured separately but share the same +settings for ``slowms`` and ``sampleRate``. +When :setting:`~param.logLevel` is set to ``0``, :program:`mongod` +records *slow* queries to the system log at a rate determined by +:setting:`~operationProfiling.slowOpSampleRate`. At higher +:setting:`~param.logLevel` settings, all queries appear in the system +log regardless of their latency. diff --git a/source/includes/options-conf.yaml b/source/includes/options-conf.yaml index e7546787cf8..f6016ae5201 100644 --- a/source/includes/options-conf.yaml +++ b/source/includes/options-conf.yaml @@ -1010,6 +1010,37 @@ description: | method. --- program: conf +name: operationProfiling.mode +type: string +default: "``off``" +directive: setting +description: | + Specifies which operations should be :doc:`profiled + `. + The following profiler levels are available: + + .. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Level + - Description + + * - ``off`` + - The profiler is off and does not collect any data. + This is the default profiler level. + + * - ``slowOp`` + - The profiler collects data for operations that take longer + than the value of ``slowms``. + + * - ``all`` + - The profiler collects data for all operations. + + .. include:: /includes/warning-profiler-performance.rst + +--- +program: conf name: operationProfiling.slowOpThresholdMs type: integer default: 100 @@ -1024,18 +1055,32 @@ post: | The {{role}} setting is available only for :program:`mongod`. --- program: conf -name: operationProfiling.mode +name: operationProfiling.slowOpSampleRate +type: double +default: 1.0 +directive: setting +replacement: + program: ":program:`mongod`" +inherit: + name: slowOpSampleRate + program: mongod + file: options-mongod.yaml +--- +program: conf +name: storage.dbPath type: string directive: setting replacement: program: ":program:`mongod`" - intro: "The" + storage_engine: :setting:`~storage.engine` inherit: - name: profilingmode + name: dbpath program: mongod file: options-mongod.yaml post: | The {{role}} setting is available only for :program:`mongod`. + + .. include:: /includes/extracts/linux-config-expectations-storage-dbpath.rst --- program: conf name: storage.dbPath diff --git a/source/includes/options-mongod.yaml b/source/includes/options-mongod.yaml index 107f4698d16..da754982c5a 100644 --- a/source/includes/options-mongod.yaml +++ b/source/includes/options-mongod.yaml @@ -693,76 +693,6 @@ replacement: verb: "Specifies the" --- program: mongod -name: profile -args: -default: 0 -directive: option -description: | - Changes the level of database profiling, which inserts information about - operation performance into the ``system.profile`` collection. Specify one - of the following levels: - - .. list-table:: - :header-rows: 1 - :widths: 20 40 - - * - Level - - - Setting - - * - 0 - - - Off. No profiling. - - * - 1 - - - On. Only includes slow operations. - - * - 2 - - - On. Includes all operations. - - Database profiling can impact database - performance. Enable this option only after careful consideration. -optional: true ---- -program: mongod -name: profilingmode -args: -default: "off" -directive: option -description: | - {{intro}} level of database profiling, which inserts information about - operation performance into the ``system.profile`` collection. Specify one - of the following levels: - - .. list-table:: - :header-rows: 1 - :widths: 20 40 - - * - Level - - - Setting - - * - ``off`` - - - Off. No profiling. - - * - ``slowOp`` - - - On. Only includes slow operations. - - * - ``all`` - - - On. Includes all operations. - - Database profiling can impact database - performance. Enable this option only after careful consideration. -optional: true -replacement: - intro: "Changes the" ---- -program: mongod name: quota args: null directive: option @@ -852,16 +782,40 @@ description: | optional: true --- program: mongod +name: profile +args: +default: 0 +directive: option +description: | + Configures the :doc:`database profiler + ` level. + The following profiler levels are available: + + .. include:: /includes/database-profiler-levels.rst + + .. include:: /includes/warning-profiler-performance.rst + +optional: true +--- +program: mongod name: slowms args: default: 100 directive: option description: | - The threshold in milliseconds at which the database profiler considers a - query slow. MongoDB records all slow queries to the log, even when the - database profiler is off. When the profiler is on, it writes to the - ``system.profile`` collection. See the :dbcommand:`profile` command for - more information on the database profiler. + .. include:: /includes/dbcommand-slowms-definition.rst + + .. seealso:: :doc:`/tutorial/manage-the-database-profiler` +optional: true +--- +program: mongod +name: slowOpSampleRate +args: +default: 1.0 +directive: option +description: | + The fraction of *slow* operations that should be profiled. + ``sampleRate`` accepts values between 0 and 1, inclusive. optional: true --- program: mongod diff --git a/source/includes/warning-profiler-performance.rst b/source/includes/warning-profiler-performance.rst new file mode 100644 index 00000000000..ae0f8ff8e02 --- /dev/null +++ b/source/includes/warning-profiler-performance.rst @@ -0,0 +1,9 @@ +.. important:: + + Profiling can impact performance and shares settings with the system + log. Carefully consider any performance and security implications + before configuring and enabling the profiler on a production + deployment. + + See :ref:`database-profiling-overhead` for more information on + potential performance degradation. diff --git a/source/reference/command/profile.txt b/source/reference/command/profile.txt index 03ed198af7b..d5540c3cfc8 100644 --- a/source/reference/command/profile.txt +++ b/source/reference/command/profile.txt @@ -10,62 +10,53 @@ profile :depth: 1 :class: singlecol +Definition +---------- + .. dbcommand:: profile - Use the :dbcommand:`profile` command to enable, disable, or change - the query profiling level. This allows administrators to capture - data regarding performance. The database profiling system can - impact performance and can allow the server to write the contents - of queries to the log. Your deployment should carefully consider the - security implications of this. The profiler is off by default. - - .. |command| replace:: ``profile`` + Enables, disables, or configures the + :doc:`/tutorial/manage-the-database-profiler`. + The profiler captures and records data on the performance of write + operations, cursors, and database commands on a running + :program:`mongod` instance. + + The profiler is off by default. - .. include:: /includes/fact-dbcommand.rst + .. include:: /includes/warning-profiler-performance.rst The :dbcommand:`profile` command has the following syntax: .. code-block:: javascript - { profile: } - - The following profiling levels are available: - - ========= ============================================== - **Level** **Setting** - --------- ---------------------------------------------- - -1 No change. Returns the current profile level. - 0 Off. No profiling. The default profiler level. - 1 On. Only includes slow operations. - 2 On. Includes all operations. - ========= ============================================== + { + profile: , + slowms: , + sampleRate: + } .. _slowms-threshold-option: - You may optionally set a threshold in milliseconds for profiling using - the ``slowms`` option, as follows: - - .. code-block:: javascript - - { profile: 1, slowms: 200 } - - :program:`mongod` writes the output of the database profiler to the - ``system.profile`` collection. + .. include:: /includes/apiargs/dbcommand-profile-fields.rst - :program:`mongod` records queries that take longer than - the :setting:`~operationProfiling.slowOpThresholdMs` to the server log even when the database profiler is - not active. + .. versionchanged:: 3.6 + + The command can also take the following options: + + .. include:: /includes/extracts/sessions-option-table-options.rst - .. seealso:: Additional documentation regarding :ref:`Database Profiling `. +The :method:`db.getProfilingStatus()` and +:method:`db.setProfilingLevel()` :doc:`shell methods +` provide wrappers around the :dbcommand:`profile` +command. - .. seealso:: ":method:`db.getProfilingStatus()`" and - ":method:`db.setProfilingLevel()`" provide wrappers - around this functionality in the :program:`mongo` - shell. +Behavior +-------- - .. note:: +The :dbcommand:`profile` command obtains a write lock on the affected +database while enabling or disabling the profiler. This is typically a +short operation. The lock blocks other operations until the +:dbcommand:`profile` command has completed. - This command obtains a write lock on the affected database and - blocks other operations until it has completed. However, the write - lock is only held while enabling or disabling the - profiler. This is typically a short operation. +.. seealso:: + :ref:`Database Profiling `. diff --git a/source/reference/configuration-options.txt b/source/reference/configuration-options.txt index 8b589835f51..67c8c6cc5df 100644 --- a/source/reference/configuration-options.txt +++ b/source/reference/configuration-options.txt @@ -649,18 +649,26 @@ LDAP Parameters .. include:: /includes/option/setting-conf-storage.inMemory.engineConfig.inMemorySizeGB.rst +.. _operation-profiling-configuration-options: + ``operationProfiling`` Options ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: yaml operationProfiling: - slowOpThresholdMs: mode: + slowOpThresholdMs: + slowOpSampleRate: +.. _configuration-profiling-mode: +.. include:: /includes/option/setting-conf-operationProfiling.mode.rst + +.. _configuration-profiling-slowOpThresholdMs: .. include:: /includes/option/setting-conf-operationProfiling.slowOpThresholdMs.rst -.. include:: /includes/option/setting-conf-operationProfiling.mode.rst +.. _configuration-profiling-slowOpSampleRate: +.. include:: /includes/option/setting-conf-operationProfiling.slowOpSampleRate.rst ``replication`` Options ~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/source/reference/method/db.getProfilingStatus.txt b/source/reference/method/db.getProfilingStatus.txt index ad7cd399732..e1f1e1c9e22 100644 --- a/source/reference/method/db.getProfilingStatus.txt +++ b/source/reference/method/db.getProfilingStatus.txt @@ -12,5 +12,7 @@ db.getProfilingStatus() .. method:: db.getProfilingStatus() - :returns: The current :dbcommand:`profile` level and - :setting:`~operationProfiling.slowOpThresholdMs` setting. + :returns: The current :dbcommand:`profile` level, + :setting:`~operationProfiling.slowOpThresholdMs` setting, + and :setting:`~operationProfiling.slowOpSampleRate` + setting. diff --git a/source/reference/method/db.setProfilingLevel.txt b/source/reference/method/db.setProfilingLevel.txt index 2334c1d8da6..cedb00aef2d 100644 --- a/source/reference/method/db.setProfilingLevel.txt +++ b/source/reference/method/db.setProfilingLevel.txt @@ -13,27 +13,11 @@ db.setProfilingLevel() Definition ---------- -.. method:: db.setProfilingLevel(level, slowms) +.. method:: db.setProfilingLevel(level, options) - Modifies the current :term:`database profiler` level used by the - database profiling system to capture data about performance. The - method provides a wrapper around the :term:`database command` - :dbcommand:`profile`. + Configures the :term:`database profiler`. This method provides a + wrapper around the :dbcommand:`profile` command. .. include:: /includes/apiargs/method-db.setProfilingLevel-param.rst - The level chosen can affect performance. It also can allow the - server to write the contents of queries to the log, which might have - information security implications for your deployment. - - Configure the :setting:`~operationProfiling.slowOpThresholdMs` option to set the threshold - for the profiler to consider a query "slow." Specify this value in - milliseconds to override the default, 100 ms. - - :program:`mongod` writes the output of the database profiler to the - ``system.profile`` collection. - - :program:`mongod` prints information about queries that take longer than - the :setting:`~operationProfiling.slowOpThresholdMs` to the log even when the database profiler is - not active. - + .. include:: /includes/warning-profiler-performance.rst diff --git a/source/reference/program/mongod.txt b/source/reference/program/mongod.txt index 48ac22108af..0afbfc8d932 100644 --- a/source/reference/program/mongod.txt +++ b/source/reference/program/mongod.txt @@ -100,10 +100,6 @@ Core Options .. include:: /includes/option/option-mongod-transitionToAuth.rst -.. include:: /includes/option/option-mongod-slowms.rst - -.. include:: /includes/option/option-mongod-profile.rst - .. include:: /includes/option/option-mongod-cpu.rst .. include:: /includes/option/option-mongod-sysinfo.rst @@ -292,6 +288,17 @@ TLS/SSL Options .. include:: /includes/option/option-mongod-sslFIPSMode.rst + +Profiler Options +~~~~~~~~~~~~~~~~ + +.. include:: /includes/option/option-mongod-profile.rst + +.. include:: /includes/option/option-mongod-slowms.rst + +.. _cli-mongod-profiler-slowOpSampleRate: +.. include:: /includes/option/option-mongod-slowOpSampleRate.rst + Audit Options ~~~~~~~~~~~~~ diff --git a/source/tutorial/manage-the-database-profiler.txt b/source/tutorial/manage-the-database-profiler.txt index 6b7e5e89210..d58f1d59ac2 100644 --- a/source/tutorial/manage-the-database-profiler.txt +++ b/source/tutorial/manage-the-database-profiler.txt @@ -12,18 +12,20 @@ Database Profiler :depth: 1 :class: singlecol -The database profiler collects fine grained data about MongoDB write -operations, cursors, database commands on a running :program:`mongod` -instance. You can enable profiling on a per-database or per-instance -basis. The :ref:`profiling level ` is also -configurable when enabling profiling. The profiler is *off* by default. - -The database profiler writes all the data it collects to the -:data:`system.profile <.system.profile>` collection, which -is a :doc:`capped collection `. See -:doc:`/reference/database-profiler` for overview of the data in the -:data:`system.profile <.system.profile>` documents created -by the profiler. +The database profiler collects detailed information about +:ref:`database-commands` executed against a running :program:`mongod` +instance. This includes CRUD operations as well as configuration and +administration commands. +The profiler writes all the data it collects to the +:data:`system.profile <.system.profile>` collection, a +:doc:`capped collection ` in the ``admin`` +database. See :doc:`/reference/database-profiler` for an overview of the +:data:`system.profile <.system.profile>` documents created by +the profiler. + +The profiler is ``off`` by default. You can enable the profiler on a +per-database or per-instance basis at one of several :ref:`profiling +levels `. This document outlines a number of key administration options for the database profiler. For additional related information, consider the @@ -41,26 +43,12 @@ Profiling Levels The following profiling levels are available: -- ``0`` - the profiler is off, does not collect any data. :program:`mongod` - always writes operations longer than the :setting:`~operationProfiling.slowOpThresholdMs` threshold - to its log. This is the default profiler level. - -- ``1`` - collects profiling data for slow operations only. By default - slow operations are those slower than 100 milliseconds. - - You can modify the threshold for "slow" operations with the - :setting:`~operationProfiling.slowOpThresholdMs` runtime option or the :dbcommand:`setParameter` - command. See the :ref:`database-profiling-specify-slowms-threshold` - section for more information. - -- ``2`` - collects profiling data for all database operations. - -.. todo these objects should indexed. +.. include:: /includes/database-profiler-levels.rst .. _database-profiling-enable-profiling: -Enable Database Profiling and Set the Profiling Level ------------------------------------------------------ +Enable and Configure Database Profiling +--------------------------------------- You can enable database profiling from the :program:`mongo` shell or through a driver using the :dbcommand:`profile` command. This section @@ -77,9 +65,9 @@ enable profiling for that database. To enable profiling and set the profiling level, use the :method:`db.setProfilingLevel()` helper in the :program:`mongo` shell, -passing the profiling level as a parameter. For example, to enable profiling -for all database operations, consider the following operation in the -:program:`mongo` shell: +passing the profiling level as a parameter. +For example, to enable profiling for all database operations, consider +the following operation in the :program:`mongo` shell: .. code-block:: javascript @@ -90,7 +78,7 @@ The ``"ok" : 1`` key-value pair indicates the operation succeeded: .. code-block:: javascript - { "was" : 0, "slowms" : 100, "ok" : 1 } + { "was" : 0, "slowms" : 100, "sampleRate" : 1.0, "ok" : 1 } To verify the new setting, see the :ref:`database-profiling-view-status` section. @@ -100,33 +88,66 @@ To verify the new setting, see the Specify the Threshold for Slow Operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The threshold for slow operations applies to the entire -:program:`mongod` instance. When you change the threshold, you change it -for all databases on the instance. +To change the slow operation threshold, specify the desired threshold +value in one of the following ways: + +- Set the value of ``slowms`` using the :dbcommand:`profile` command or + :method:`db.setProfilingLevel()` shell helper method. -.. important:: Changing the slow operation threshold for the database - profiler also affects the profiling subsystem's slow operation - threshold for the entire :program:`mongod` instance. Always set the - threshold to the highest useful value. +- Set the value of :option:`--slowms` from the command line at startup. -By default the slow operation threshold is 100 milliseconds. Databases with a profiling level -of ``1`` will log operations slower than 100 milliseconds. +- Set the value of :setting:`~operationProfiling.slowOpThresholdMs` in a + :ref:`configuration file `. -To change the threshold, pass two parameters to the -:method:`db.setProfilingLevel()` helper in the :program:`mongo` shell. The first parameter sets the -profiling level for the current database, and the second sets the default -slow operation threshold *for the entire* :program:`mongod` -*instance*. +By default, the slow operation threshold is 100 milliseconds. Databases +with a profiling level of ``1`` will profile operations slower than the +threshold. -For example, the following command sets the profiling level for the current -database to ``0``, which disables profiling, and sets the -slow-operation threshold for the :program:`mongod` instance to 20 -milliseconds. Any database on the instance with a profiling level of ``1`` -will use this threshold: +For example, the following code sets the profiling level for the +current database to ``1`` and sets the slow operation threshold for the +:program:`mongod` instance to 20 milliseconds: .. code-block:: javascript - db.setProfilingLevel(0,20) + db.setProfilingLevel(1, { slowms: 20 }) + +.. important:: + The slow operation threshold applies to all databases in a + :program:`mongod` instance. It is used by both the database profiler + and the system log and should be set to the highest useful value to + avoid performance degradation. + +Profile a Random Sample of Slow Operations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. versionadded:: 3.6 + +To profile only a randomly sampled subset of all *slow* operations on a +specific database, specify the desired sample rate in one of the following ways: + +- Set the value of ``sampleRate`` using the :dbcommand:`profile` command + or :method:`db.setProfilingLevel()` shell helper method. + +- Set the value of :option:`--slowOpSampleRate` from the command line at + startup. + +- Set the value of :setting:`~operationProfiling.slowOpSampleRate` in a + :ref:`configuration file `. + +By default, ``sampleRate`` is set to ``1.0``, meaning all *slow* +operations are profiled. Databases with a ``sampleRate`` between 0 and 1 +will only profile a randomly sampled percentage of *slow* operations +according to ``sampleRate``. + +For example, the following method sets the profiling level for the +current database to ``1`` and sets the profiler to sample 42% of all *slow* operations: + +.. code-block:: javascript + + db.setProfilingLevel(1, { sampleRate: 0.42 }) + +.. important:: + .. include:: /includes/fact-log-slow-queries.rst .. _database-profiling-view-status: .. _database-profiling-check-level: @@ -145,19 +166,18 @@ The shell returns a document similar to the following: .. code-block:: javascript - { "was" : 0, "slowms" : 100 } + { "was" : 0, "slowms" : 100, "sampleRate" : 1.0, "ok" : 1 } -The ``was`` field indicates the current level of profiling. +The ``was`` field indicates the current profiling level. -The ``slowms`` field indicates how long an operation must exist in -milliseconds for an operation to pass the "slow" threshold. MongoDB -will log operations that take longer than the threshold if the -profiling level is ``1``. This document returns the profiling level in -the ``was`` field. For an explanation of profiling levels, see -:ref:`database-profiling-levels`. +The ``slowms`` field indicates operation time threshold, in +milliseconds, beyond which operations are considered *slow*. -To return only the profiling level, use the :method:`db.getProfilingLevel()` -helper in the :program:`mongo` as in the following: +The ``sampleRate`` field indicates the percentage of *slow* operations +that should be profiled. + +To return only the profiling level, use the +:method:`db.getProfilingLevel()` helper in the :program:`mongo` shell as in the following: .. code-block:: javascript @@ -182,18 +202,26 @@ profiling level applies to all databases provided by the :program:`mongod` instance. To enable profiling for a :program:`mongod` instance, pass the following -parameters to :program:`mongod` at startup or within the -:doc:`configuration file `: +parameters to :program:`mongod` at startup. .. code-block:: sh - mongod --profile 1 --slowms 15 + mongod --profile 1 --slowms 15 --slowOpSampleRate 0.5 + +Alternatively, you can specify :ref:`operationProfiling +` in the :doc:`configuration +file `. -This sets the profiling level to ``1``, which collects profiling data -for slow operations only, and defines slow operations as those that -last longer than ``15`` milliseconds. +This sets the profiling level to ``1``, defines slow operations as those +that last longer than ``15`` milliseconds, and specifies that only 50% +of *slow* operations should be profiled. These settings also affect +which operations are recorded to the system log when +:setting:`~param.logLevel` is set to ``0``. -.. seealso:: :setting:`~operationProfiling.mode` and :setting:`~operationProfiling.slowOpThresholdMs`. +.. seealso:: + :setting:`~operationProfiling.mode`, + :setting:`~operationProfiling.slowOpThresholdMs`, and + :setting:`~operationProfiling.slowOpSampleRate`. Database Profiling and Sharding ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -217,6 +245,8 @@ easier to analyze data from the profiler. To view example queries, see For an explanation of the output data, see :doc:`/reference/database-profiler`. +.. _database-profiling-example-queries: + Example Profiler Data Queries ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -253,34 +283,30 @@ similar to the following: db.system.profile.find( { millis : { $gt : 5 } } ).pretty() -To return information from a certain time range, run a query similar to the following: +To return information from a certain time range, run a query similar to +the following: .. code-block:: javascript - db.system.profile.find( - { - ts : { - $gt : new ISODate("2012-12-09T03:00:00Z") , - $lt : new ISODate("2012-12-09T03:40:00Z") - } - } - ).pretty() + db.system.profile.find({ + ts : { + $gt: new ISODate("2012-12-09T03:00:00Z"), + $lt: new ISODate("2012-12-09T03:40:00Z") + } + }).pretty() -The following example looks at the time range, suppresses the ``user`` field -from the output to make it easier to read, and sorts the results by how -long each operation took to run: +The following example looks at the time range, suppresses the ``user`` +field from the output to make it easier to read, and sorts the results +by how long each operation took to run: .. code-block:: javascript - db.system.profile.find( - { - ts : { - $gt : new ISODate("2011-07-12T03:00:00Z") , - $lt : new ISODate("2011-07-12T03:40:00Z") - } - }, - { user : 0 } - ).sort( { millis : -1 } ) + db.system.profile.find({ + ts : { + $gt: new ISODate("2011-07-12T03:00:00Z"), + $lt: new ISODate("2011-07-12T03:40:00Z") + } + }, { user: 0 }).sort( { millis: -1 } ) Show the Five Most Recent Events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -294,7 +320,7 @@ from the :program:`mongo` shell, as follows: show profile -.. _database-profiling-example-queries: +.. _database-profiling-overhead: Profiler Overhead ----------------- @@ -314,9 +340,11 @@ To change the size of the :data:`system.profile 1. Disable profiling. -#. Drop the :data:`system.profile <.system.profile>` collection. +#. Drop the :data:`system.profile <.system.profile>` + collection. -#. Create a new :data:`system.profile <.system.profile>` collection. +#. Create a new :data:`system.profile <.system.profile>` + collection. #. Re-enable profiling.