From 21488656f482b8dcd8f09db681215f1fdf36055b Mon Sep 17 00:00:00 2001 From: Andrew Aldridge Date: Tue, 21 Jul 2015 13:36:42 -0400 Subject: [PATCH] DOCS-5330: Tweak THP guide --- .../includes/steps-disable-thp-in-grub.yaml | 32 ------- .../includes/steps-disable-thp-in-initd.yaml | 83 +++++++++++++++++++ .../includes/steps-disable-thp-in-ktune.yaml | 51 ------------ .../steps-disable-thp-in-rc.local.yaml | 34 -------- .../steps-disable-thp-in-tuned-rhel-6.yaml | 35 ++++++++ .../steps-disable-thp-in-tuned-rhel-7.yaml | 30 +++++++ source/tutorial/transparent-huge-pages.txt | 66 ++++++++------- 7 files changed, 183 insertions(+), 148 deletions(-) delete mode 100644 source/includes/steps-disable-thp-in-grub.yaml create mode 100644 source/includes/steps-disable-thp-in-initd.yaml delete mode 100644 source/includes/steps-disable-thp-in-ktune.yaml delete mode 100644 source/includes/steps-disable-thp-in-rc.local.yaml create mode 100644 source/includes/steps-disable-thp-in-tuned-rhel-6.yaml create mode 100644 source/includes/steps-disable-thp-in-tuned-rhel-7.yaml diff --git a/source/includes/steps-disable-thp-in-grub.yaml b/source/includes/steps-disable-thp-in-grub.yaml deleted file mode 100644 index 74554ad1b29..00000000000 --- a/source/includes/steps-disable-thp-in-grub.yaml +++ /dev/null @@ -1,32 +0,0 @@ -title: Edit ``grub`` configuration file -stepnum: 1 -ref: edit-grub -pre: | - If your system uses ``grub-legacy``, you can edit the configuration - file directly. -action: - - pre: | - Add ``transparent_hugepage=never`` to the ``kernel`` line of your ``/etc/grub.conf`` or ``/boot/grub/grub.conf`` as ``root``: - language: cfg - code: | - kernel transparent_hugepage=never - - pre: | - If your system uses ``grub2``, you can edit the ``GRUB_CMDLINE_LINUX_DEFAULT`` value in ``/etc/default/grub`` as ``root`` to add ``transparent_hugepage=never`` to any existing kernel options: - language: cfg - code: | - GRUB_CMDLINE_LINUX_DEFAULT=" transparent_hugepage=never" - - pre: | - and subsequently run - language: cfg - code: | - grub2-mkconfig -o - - post: | - as ``root``. - - See your operating system's documentation for details on the - precise location of the ``grub-legacy`` or ``grub2`` configuration file. ---- -title: Reboot the system. -stepnum: 2 -ref: reboot -... diff --git a/source/includes/steps-disable-thp-in-initd.yaml b/source/includes/steps-disable-thp-in-initd.yaml new file mode 100644 index 00000000000..b19364f6a80 --- /dev/null +++ b/source/includes/steps-disable-thp-in-initd.yaml @@ -0,0 +1,83 @@ +title: Create the ``init.d`` script +stepnum: 1 +ref: create-initd +action: + - pre: | + Create the following file at ``/etc/init.d/disable-transparent-hugepages``: + + language: sh + code: | + #!/bin/sh + ### BEGIN INIT INFO + # Provides: disable-transparent-hugepages + # Required-Start: $local_fs + # Required-Stop: + # X-Start-Before: mongod mongodb-mms-automation-agent + # Default-Start: 2 3 4 5 + # Default-Stop: 0 1 6 + # Short-Description: Disable Linux transparent huge pages + # Description: Disable Linux transparent huge pages, to improve + # database performance. + ### END INIT INFO + + case $1 in + start) + if [ -d /sys/kernel/mm/transparent_hugepage ]; then + thp_path=/sys/kernel/mm/transparent_hugepage + elif [ -d /sys/kernel/mm/redhat_transparent_hugepage ]; then + thp_path=/sys/kernel/mm/redhat_transparent_hugepage + else + return 0 + fi + + echo 'never' > ${thp_path}/enabled + echo 'never' > ${thp_path}/defrag + + unset thp_path + ;; + esac +--- +title: Make it executable +stepnum: 2 +ref: make-executable +action: + - pre: | + Run the following command to ensure that the init script can be used: + + language: sh + code: sudo chmod 755 /etc/init.d/disable-transparent-hugepages +--- +title: Configure your operating system to run it on boot +stepnum: 3 +ref: configure-initd +action: + - pre: | + Use the appropriate command to configure the new init script on your Linux + distribution. + + .. list-table:: + :header-rows: 1 + :widths: 20 80 + + * - Distribution + + - Command + + * - Ubuntu and Debian + + - .. code:: sh + + sudo update-rc.d disable-transparent-hugepages defaults + + * - SUSE + + - .. code:: sh + + sudo insserv /etc/init.d/disable-transparent-hugepages + + * - Red Hat, CentOS, Amazon Linux, and derivatives + + - .. code:: sh + + sudo chkconfig --add disable-transparent-hugepages +... diff --git a/source/includes/steps-disable-thp-in-ktune.yaml b/source/includes/steps-disable-thp-in-ktune.yaml deleted file mode 100644 index cf36d5e2fe5..00000000000 --- a/source/includes/steps-disable-thp-in-ktune.yaml +++ /dev/null @@ -1,51 +0,0 @@ -title: Create a new profile -stepnum: 1 -ref: copy-profile -action: - - pre: | - If necessary, create a new profile from an existing default - profile by copying the relevant directory. In the example we use - the ``default`` profile as the base and call our new profile - ``no-thp``, but you may start with any profile and name the - new profile whatever you wish. - language: sh - code: | - cp -r /etc/tune-profiles/default /etc/tune-profiles/no-thp - post: | - You may also opt to edit an existing profile. ---- -title: Edit ``ktune.sh`` -stepnum: 2 -ref: edit-ktune -action: - - pre: | - Locate the ``ktune.sh`` file in the new profile directory - (``/etc/tune-profiles/no-thp/ktune.sh`` in this case) and add - the following: - language: cfg - code: | - set_transparent_hugepages never - post: | - to the ``start()`` block of the file, before the ``return 0`` - statement. ---- -title: Enable the new profile -stepnum: 3 -ref: enable-profile -action: - - pre: | - Finally, enable the new profile by issuing: - language: sh - code: | - tune-adm profile no-thp - post: | - where ``no-thp`` should be replaced with the name of the profile - you defined, if it is different. - - .. warning:: - - If you are using ``tuned`` and ``ktune`` and do not wish to - disable them, you **must** use this method, or ``tuned`` may - override THP settings configured using either of the other - two methods. -... diff --git a/source/includes/steps-disable-thp-in-rc.local.yaml b/source/includes/steps-disable-thp-in-rc.local.yaml deleted file mode 100644 index 81dcf827bf5..00000000000 --- a/source/includes/steps-disable-thp-in-rc.local.yaml +++ /dev/null @@ -1,34 +0,0 @@ -title: Edit ``/etc/rc.local`` -stepnum: 1 -ref: edit-rclocal -pre: | - ``/etc/rc.local`` is a user-configurable script that is run at the end of - post-boot system initialization. Add the following script lines to the file - as ``root`` to disable THP upon each boot. -action: - - language: sh - code: | - if test -f /sys/kernel/mm/transparent_hugepage/khugepaged/defrag; then - echo 0 > /sys/kernel/mm/transparent_hugepage/khugepaged/defrag - fi - if test -f /sys/kernel/mm/transparent_hugepage/defrag; then - echo never > /sys/kernel/mm/transparent_hugepage/defrag - fi - if test -f /sys/kernel/mm/transparent_hugepage/enabled; then - echo never > /sys/kernel/mm/transparent_hugepage/enabled - fi - - post: | - These lines should immediately precede ``exit 0``, which should - already be the last line in the file. Note that on Red Hat - Enterprise Linux, CentOS, and potentially other Red Hat-based - derivatives, ``transparent_hugepage`` in the paths in the script - should be replaced by ``redhat_transparent_hugepages``. ---- -title: Apply the changes to ``/etc/rc.local`` as ``root`` -stepnum: 2 -ref: source -action: - - language: sh - code: | - source /etc/rc.local -... diff --git a/source/includes/steps-disable-thp-in-tuned-rhel-6.yaml b/source/includes/steps-disable-thp-in-tuned-rhel-6.yaml new file mode 100644 index 00000000000..c9688ecb311 --- /dev/null +++ b/source/includes/steps-disable-thp-in-tuned-rhel-6.yaml @@ -0,0 +1,35 @@ +title: Create a new profile +stepnum: 1 +ref: copy-profile +action: + - pre: | + Create a new profile from an existing default + profile by copying the relevant directory. In the example we use + the ``default`` profile as the base and call our new profile ``no-thp``. + language: sh + code: | + sudo cp -r /etc/tune-profiles/default /etc/tune-profiles/no-thp +--- +title: Edit ``ktune.sh`` +stepnum: 2 +ref: edit-ktune +action: + - pre: | + Edit ``/etc/tune-profiles/no-thp/ktune.sh`` and add the following: + language: cfg + code: | + set_transparent_hugepages never + post: | + to the ``start()`` block of the file, before the ``return 0`` + statement. +--- +title: Enable the new profile +stepnum: 3 +ref: enable-profile +action: + - pre: | + Finally, enable the new profile by issuing: + language: sh + code: | + sudo tune-adm profile no-thp +... diff --git a/source/includes/steps-disable-thp-in-tuned-rhel-7.yaml b/source/includes/steps-disable-thp-in-tuned-rhel-7.yaml new file mode 100644 index 00000000000..dd0e1541e80 --- /dev/null +++ b/source/includes/steps-disable-thp-in-tuned-rhel-7.yaml @@ -0,0 +1,30 @@ +title: Create a new profile +stepnum: 1 +ref: copy-profile +action: + - pre: | + Create a new ``tuned`` profile directory: + language: sh + code: | + sudo mkdir /etc/tuned/no-thp +--- +title: Edit ``tuned.conf`` +stepnum: 2 +ref: edit-ktune +action: + - pre: | + Create and edit ``/etc/tuned/no-thp/tuned.conf`` so that it contains the + following: + language: ini + code: | + [main] + include=virtual-guest + + [vm] + transparent_hugepages=never +--- +stepnum: 3 +source: + file: steps-disable-thp-in-tuned-rhel-6.yaml + ref: enable-profile +... diff --git a/source/tutorial/transparent-huge-pages.txt b/source/tutorial/transparent-huge-pages.txt index 769fc53813b..316c957202f 100644 --- a/source/tutorial/transparent-huge-pages.txt +++ b/source/tutorial/transparent-huge-pages.txt @@ -4,62 +4,66 @@ Disable Transparent Huge Pages (THP) .. default-domain:: mongodb -Transparent Huge Pages (THP) is a Linux memory management kernel -module used to reduce the overhead of Translation Lookaside Buffer -(TLB) lookups on machines with large amounts of memory, and therefore -a large number of memory pages. In effect, it presents system memory -as a smaller number of much larger pages rather than as a larger -number of 4096-byte pages. +Transparent Huge Pages (THP) is a Linux memory management system +that reduces the overhead of Translation Lookaside Buffer (TLB) lookups on +machines with large amounts of memory by using larger memory pages. -However, THP is known to perform poorly under database workloads, -which tend to have sparse rather than contiguous memory access -patterns. You must disable THP on Linux machines used to run MongoDB -instances to ensure best performance. +However, database workloads often perform poorly with THP, +because they tend to have sparse rather than contiguous memory access +patterns. You should disable THP on Linux machines to ensure best performance +with MongoDB. -There are three ways to disable Transparent Huge Pages on Linux. +Init Script +----------- -In Boot-Time Configuration (Preferred) --------------------------------------- +.. include:: /includes/steps/disable-thp-in-initd.rst -.. include:: /includes/steps/disable-thp-in-grub.rst +Using ``tuned`` and ``ktune`` +----------------------------- -Using ``tuned`` and ``ktune`` (Necessary if you use ``tuned``) --------------------------------------------------------------- +.. important:: -``tuned`` and ``ktune`` are dynamic kernel tuning tools that act in -accordance with policies set by the system administrator. One of the -settings available to ``tuned`` and ``ktune`` is whether or not to use -THP. + If installed, you must configure ``tuned`` and ``ktune`` to prevent them from + re-enabling THP. -To disable transparent hugepages, you need to edit or create a new -profile that sets THP to ``never``. +``tuned`` and ``ktune`` are dynamic kernel tuning tools available on Red Hat +and CentOS that can disable transparent huge pages. -.. include:: /includes/steps/disable-thp-in-ktune.rst +To disable transparent huge pages in ``tuned`` or ``ktune``, you need to edit or +create a new profile that sets THP to ``never``. -In ``/etc/rc.local`` (Alternate) --------------------------------- +Red Hat/CentOS 6 +~~~~~~~~~~~~~~~~ -.. include:: /includes/steps/disable-thp-in-rc.local.rst +.. include:: /includes/steps/disable-thp-in-tuned-rhel-6.rst + +Red Hat/CentOS 7 +~~~~~~~~~~~~~~~~ + +.. include:: /includes/steps/disable-thp-in-tuned-rhel-7.rst + +.. _test-thp-changes: Test Your Changes ----------------- -Whichever of the three methods you use, you can check the status of THP support by issuing the command: +You can check the status of THP support by issuing the following commands: .. code:: sh cat /sys/kernel/mm/transparent_hugepage/enabled + cat /sys/kernel/mm/transparent_hugepage/defrag -or +On Red Hat Enterprise Linux, CentOS, and potentially other Red +Hat-based derivatives, you may instead need to use the following: .. code:: sh cat /sys/kernel/mm/redhat_transparent_hugepage/enabled + cat /sys/kernel/mm/redhat_transparent_hugepage/defrag -on Red Hat Enterprise Linux, CentOS, and potentially other Red -Hat-based derivatives. -Correct output resembles: +For both files, the correct output resembles: .. code:: sh