From 2d878cfa5ce6314b9a52283fdbeef8338876c0c1 Mon Sep 17 00:00:00 2001 From: Toby McLaughlin Date: Mon, 30 Oct 2017 16:41:20 +1100 Subject: [PATCH 1/6] Update Docker docs for 6.0.0-rc2 Update the docs to match the new Docker "image flavours" of "basic", "platinum", and "oss". --- docs/Versions.asciidoc | 3 +- docs/reference/setup/install.asciidoc | 4 +- docs/reference/setup/install/docker.asciidoc | 102 ++++++++++++------- 3 files changed, 70 insertions(+), 39 deletions(-) diff --git a/docs/Versions.asciidoc b/docs/Versions.asciidoc index b6a6f91d232c4..6127d599acfee 100644 --- a/docs/Versions.asciidoc +++ b/docs/Versions.asciidoc @@ -14,7 +14,8 @@ release-state can be: released | prerelease | unreleased :issue: https://github.com/elastic/elasticsearch/issues/ :pull: https://github.com/elastic/elasticsearch/pull/ -:docker-image: docker.elastic.co/elasticsearch/elasticsearch:{version} +:docker-repo: docker.elastic.co/elasticsearch/elasticsearch +:docker-image: {docker-repo}:{version} :plugin_url: https://artifacts.elastic.co/downloads/elasticsearch-plugins /////// diff --git a/docs/reference/setup/install.asciidoc b/docs/reference/setup/install.asciidoc index 484d9dea970e5..babdccc2d95fe 100644 --- a/docs/reference/setup/install.asciidoc +++ b/docs/reference/setup/install.asciidoc @@ -37,7 +37,9 @@ Elasticsearch on Windows. MSIs may be downloaded from the Elasticsearch website. `docker`:: -An image is available for running Elasticsearch as a Docker container. It ships with {xpack-ref}/index.html[X-Pack] pre-installed and may be downloaded from the Elastic Docker Registry. +Images are available for running Elasticsearch as Docker containers. They may be +downloaded from the Elastic Docker Registry. The default image ships with +{xpack-ref}/index.html[X-Pack] pre-installed. + <> diff --git a/docs/reference/setup/install/docker.asciidoc b/docs/reference/setup/install/docker.asciidoc index db5518694abd6..f0b7aee8ce261 100644 --- a/docs/reference/setup/install/docker.asciidoc +++ b/docs/reference/setup/install/docker.asciidoc @@ -1,32 +1,52 @@ [[docker]] === Install Elasticsearch with Docker -Elasticsearch is also available as a Docker image. -The image is built with {xpack-ref}/index.html[X-Pack] and uses https://hub.docker.com/_/centos/[centos:7] as the base image. -The source code can be found on https://github.com/elastic/elasticsearch-docker/tree/{branch}[GitHub]. +Elasticsearch is also available as Docker images. +The images use https://hub.docker.com/_/centos/[centos:7] as the base image and +are available with {xpack-ref}/index.html[X-Pack]. The source code can be found +on https://github.com/elastic/elasticsearch-docker/tree/{branch}[GitHub]. -==== Security note +==== Image types -NOTE: {xpack-ref}/index.html[X-Pack] is preinstalled in this image. -Please take a few minutes to familiarize yourself with {xpack-ref}/security-getting-started.html[X-Pack Security] and how to change default passwords. The default password for the `elastic` user is `changeme`. +The images are available in three different configurations or "flavours". The +`basic` flavour, which is the default, ships with X-Pack Basic features +pre-installed and automatically activated with a free licence. The `platinum` +flavour features all X-Pack functionally under a 30-day trial licence. The `oss` +flavour does not include X-Pack, and contains only open-source Elasticsearch. -NOTE: X-Pack includes a trial license for 30 days. After that, you can obtain one of the https://www.elastic.co/subscriptions[available subscriptions] or {ref}/security-settings.html[disable Security]. The Basic license is free and includes the https://www.elastic.co/products/x-pack/monitoring[Monitoring] extension. +NOTE: {xpack-ref}/xpack-security.html[X-Pack Security] is enabled in the `platinum` +image. To access your cluster, it's necessary to set an initial password for the +`elastic` user. The initial password can be set at start up time via the +`ELASTIC_PASSWORD` environment variable: + +["source","txt",subs="attributes"] +-------------------------------------------- +docker run -e ELASTIC_PASSWORD=MagicWord {docker-repo}-platinum:{version} +-------------------------------------------- + +NOTE: The `platinum` image includes a trial license for 30 days. After that, you +can obtain one of the https://www.elastic.co/subscriptions[available +subscriptions] or revert to a Basic licence. The Basic license is free and +includes a selection of X-Pack features. Obtaining Elasticsearch for Docker is as simple as issuing a +docker pull+ command against the Elastic Docker registry. ifeval::["{release-state}"=="unreleased"] -WARNING: Version {version} of Elasticsearch has not yet been released, so no Docker image is currently available for this version. +WARNING: Version {version} of Elasticsearch has not yet been released, so no +Docker image is currently available for this version. endif::[] ifeval::["{release-state}"!="unreleased"] -The Docker image can be retrieved with the following command: +Docker images can be retrieved with the following commands: ["source","sh",subs="attributes"] -------------------------------------------- -docker pull {docker-image} +docker pull {docker-repo}:{version} +docker pull {docker-repo}-platinum:{version} +docker pull {docker-repo}-oss:{version} -------------------------------------------- endif::[] @@ -49,7 +69,7 @@ Elasticsearch can be quickly started for development or testing use with the fol ["source","sh",subs="attributes"] -------------------------------------------- -docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" {docker-image} +docker run -p 9200:9200 -e "discovery.type=single-node" {docker-image} -------------------------------------------- endif::[] @@ -97,7 +117,7 @@ sysctl -w vm.max_map_count=262144 + The `vm.max_map_count` setting must be set via docker-machine: + -["source","sh"] +["source","txt"] -------------------------------------------- docker-machine ssh sudo sysctl -w vm.max_map_count=262144 @@ -109,7 +129,8 @@ To bring up the cluster, use the <> additionally require write access. +. By default, Elasticsearch runs inside the container as user `elasticsearch` +using uid:gid `1000:1000`. If you are bind-mounting a local directory or file, +ensure it is readable by this user, while the <> additionally require write access. + . It is important to ensure increased ulimits for <> and <> are available for the Elasticsearch containers. Verify the https://github.com/moby/moby/tree/ea4d1243953e6b652082305a9c3cda8656edab26/contrib/init[init system] for the Docker daemon is already setting those to acceptable values and, if needed, adjust them in the Daemon, or override them per container, for example using `docker run`: + @@ -273,13 +292,22 @@ NOTE: One way of checking the Docker daemon defaults for the aforementioned ulim + docker run --rm centos:7 /bin/bash -c 'ulimit -Hn && ulimit -Sn && ulimit -Hu && ulimit -Su' + -. Swapping needs to be disabled for performance and node stability. This can be achieved through any of the methods mentioned in the <>. If you opt for the `bootstrap.memory_lock: true` approach, apart from defining it through any of the <>, you will additionally need the `memlock: true` ulimit, either defined in the https://docs.docker.com/engine/reference/commandline/dockerd/#default-ulimits[Docker Daemon] or specifically set for the container. This has been demonstrated earlier in the <>, or using `docker run`: +. Swapping needs to be disabled for performance and node stability. This can be +achieved through any of the methods mentioned in the +<>. If you opt for the +`bootstrap.memory_lock: true` approach, apart from defining it through any of +the <>, you will +additionally need the `memlock: true` ulimit, either defined in the +https://docs.docker.com/engine/reference/commandline/dockerd/#default-ulimits[Docker +Daemon] or specifically set for the container. This is demonstrated above in the +<>. If using `docker run`: + -e "bootstrap.memory_lock=true" --ulimit memlock=-1:-1 + . The image https://docs.docker.com/engine/reference/builder/#/expose[exposes] TCP ports 9200 and 9300. For clusters it is recommended to randomize the published ports with `--publish-all`, unless you are pinning one container per host. + -. Use the `ES_JAVA_OPTS` environment variable to set heap size, e.g. to use 16GB use `-e ES_JAVA_OPTS="-Xms16g -Xmx16g"` with `docker run`. It is also recommended to set a https://docs.docker.com/engine/reference/run/#user-memory-constraints[memory limit] for the container. +. Use the `ES_JAVA_OPTS` environment variable to set heap size, e.g. to use 16GB +use `-e ES_JAVA_OPTS="-Xms16g -Xmx16g"` with `docker run`. + . Pin your deployments to a specific version of the Elasticsearch Docker image, e.g. +docker.elastic.co/elasticsearch/elasticsearch:{version}+. + From ee5d684ef92d2f43a1796d866e6512575eed081e Mon Sep 17 00:00:00 2001 From: Dimitrios Liappis Date: Mon, 30 Oct 2017 15:18:47 +0200 Subject: [PATCH 2/6] Docker: Various 6.0 updates * Clarifications for Openshift and bind-mounts * Bump docker-compose 2.x format to 2.2 * Combine Docker Toolbox instructions for setting vm.max_map_count for both macOS + Windows * devicemapper is not the default storage driver any more on RHEL --- docs/reference/setup/install/docker.asciidoc | 32 ++++++++++++-------- 1 file changed, 20 insertions(+), 12 deletions(-) diff --git a/docs/reference/setup/install/docker.asciidoc b/docs/reference/setup/install/docker.asciidoc index f0b7aee8ce261..8e0be76afe321 100644 --- a/docs/reference/setup/install/docker.asciidoc +++ b/docs/reference/setup/install/docker.asciidoc @@ -3,7 +3,9 @@ Elasticsearch is also available as Docker images. The images use https://hub.docker.com/_/centos/[centos:7] as the base image and -are available with {xpack-ref}/index.html[X-Pack]. The source code can be found +are available with {xpack-ref}/index.html[X-Pack]. + +A list of all published Docker images and tags can be found in https://www.docker.elastic.co[www.docker.elastic.co]. The source code can be found on https://github.com/elastic/elasticsearch-docker/tree/{branch}[GitHub]. ==== Image types @@ -96,7 +98,7 @@ vm.max_map_count=262144 + To apply the setting on a live system type: `sysctl -w vm.max_map_count=262144` + -* OSX with https://docs.docker.com/engine/installation/mac/#/docker-for-mac[Docker for Mac] +* macOS with https://docs.docker.com/engine/installation/mac/#/docker-for-mac[Docker for Mac] + The `vm.max_map_count` setting must be set within the xhyve virtual machine: + @@ -113,7 +115,7 @@ Then configure the `sysctl` setting as you would for Linux: sysctl -w vm.max_map_count=262144 -------------------------------------------- + -* OSX with https://docs.docker.com/engine/installation/mac/#docker-toolbox[Docker Toolbox] +* Windows and macOS with https://www.docker.com/products/docker-toolbox[Docker Toolbox] + The `vm.max_map_count` setting must be set via docker-machine: + @@ -165,7 +167,7 @@ endif::[] ifeval::["{release-state}"!="unreleased"] ["source","yaml",subs="attributes"] -------------------------------------------- -version: '2' +version: 2.2 services: elasticsearch: image: {docker-image} @@ -244,7 +246,7 @@ For example, bind-mounting a `custom_elasticsearch.yml` with `docker run` can be -------------------------------------------- -v full_path_to/custom_elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -------------------------------------------- -IMPORTANT: The container **runs Elasticsearch as user `elasticsearch` using uid:gid `1000:1000`**. Bind mounted host directories and files, such as `custom_elasticsearch.yml` above, **need to be accessible by this user**. For the https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#path-settings[data and log dirs], such as `/usr/share/elasticsearch/data`, write access is required as well. +IMPORTANT: The container **runs Elasticsearch as user `elasticsearch` using uid:gid `1000:1000`**. Bind mounted host directories and files, such as `custom_elasticsearch.yml` above, **need to be accessible by this user**. For the https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#path-settings[data and log dirs], such as `/usr/share/elasticsearch/data`, write access is required as well. Also see note 1 below. ===== C. Customized image In some environments, it may make more sense to prepare a custom image containing your configuration. A `Dockerfile` to achieve this may be as simple as: @@ -276,13 +278,19 @@ docker run bin/elasticsearch -Ecluster.name=mynewclusternam ==== Notes for production use and defaults We have collected a number of best practices for production use. +Any Docker parameters mentioned below assume the use of `docker run`. -NOTE: Any Docker parameters mentioned below assume the use of `docker run`. - -. By default, Elasticsearch runs inside the container as user `elasticsearch` -using uid:gid `1000:1000`. If you are bind-mounting a local directory or file, -ensure it is readable by this user, while the <> additionally require write access. +. By default, Elasticsearch runs inside the container as user `elasticsearch` using uid:gid `1000:1000`. ++ +CAUTION: One exception is https://docs.openshift.com/container-platform/3.6/creating_images/guidelines.html#openshift-specific-guidelines[Openshift] which runs containers using an arbitrarily assigned user ID. Openshift will present persistent volumes with the gid set to `0` which will work without any adjustments. ++ +If you are bind-mounting a local directory or file, ensure it is readable by this user, while the <> additionally require write access. A good strategy is to grant group access to gid `1000` or `0` for the local directory. As an example, to prepare a local directory for storing data through a bind-mount: ++ + mkdir esdatadir + chmod g+rwx esdatadir + chgrp 1000 esdatadir ++ +As a last resort, you can also force the container to mutate the ownership of any bind-mounts used for the <> through the environment variable `TAKE_FILE_OWNERSHIP`; in this case they will be owned by uid:gid `1000:0` providing read/write access to the elasticsearch process as required. + . It is important to ensure increased ulimits for <> and <> are available for the Elasticsearch containers. Verify the https://github.com/moby/moby/tree/ea4d1243953e6b652082305a9c3cda8656edab26/contrib/init[init system] for the Docker daemon is already setting those to acceptable values and, if needed, adjust them in the Daemon, or override them per container, for example using `docker run`: + @@ -317,7 +325,7 @@ use `-e ES_JAVA_OPTS="-Xms16g -Xmx16g"` with `docker run`. .. Elasticsearch is I/O sensitive and the Docker storage driver is not ideal for fast I/O .. It allows the use of advanced https://docs.docker.com/engine/extend/plugins/#volume-plugins[Docker volume plugins] + -. If you are using the devicemapper storage driver (default on at least RedHat (rpm) based distributions) make sure you are not using the default `loop-lvm` mode. Configure docker-engine to use https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/#configure-docker-with-devicemapper[direct-lvm] instead. +. If you are using the devicemapper storage driver make sure you are not using the default `loop-lvm` mode. Configure docker-engine to use https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/#configure-docker-with-devicemapper[direct-lvm] instead. + . Consider centralizing your logs by using a different https://docs.docker.com/engine/admin/logging/overview/[logging driver]. Also note that the default json-file logging driver is not ideally suited for production use. From 9f0565d5ab0332a8e6bd307f77b0eb330c321959 Mon Sep 17 00:00:00 2001 From: Toby McLaughlin Date: Tue, 31 Oct 2017 11:57:40 +1100 Subject: [PATCH 3/6] Restore transport client port to Docker example --- docs/reference/setup/install/docker.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/setup/install/docker.asciidoc b/docs/reference/setup/install/docker.asciidoc index 8e0be76afe321..7be77ed343f40 100644 --- a/docs/reference/setup/install/docker.asciidoc +++ b/docs/reference/setup/install/docker.asciidoc @@ -71,7 +71,7 @@ Elasticsearch can be quickly started for development or testing use with the fol ["source","sh",subs="attributes"] -------------------------------------------- -docker run -p 9200:9200 -e "discovery.type=single-node" {docker-image} +docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" {docker-image} -------------------------------------------- endif::[] From edb9407c6d5262208bc9521292c4120ee4a81563 Mon Sep 17 00:00:00 2001 From: Toby McLaughlin Date: Tue, 31 Oct 2017 12:02:57 +1100 Subject: [PATCH 4/6] Remove extra word in Docker docs --- docs/reference/setup/install/docker.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/setup/install/docker.asciidoc b/docs/reference/setup/install/docker.asciidoc index 7be77ed343f40..45164810a2aab 100644 --- a/docs/reference/setup/install/docker.asciidoc +++ b/docs/reference/setup/install/docker.asciidoc @@ -131,7 +131,7 @@ To bring up the cluster, use the < Date: Tue, 31 Oct 2017 12:10:37 +1100 Subject: [PATCH 5/6] Missing comma in Docker docs --- docs/reference/setup/install/docker.asciidoc | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/reference/setup/install/docker.asciidoc b/docs/reference/setup/install/docker.asciidoc index 45164810a2aab..95bb1e700031e 100644 --- a/docs/reference/setup/install/docker.asciidoc +++ b/docs/reference/setup/install/docker.asciidoc @@ -325,7 +325,10 @@ use `-e ES_JAVA_OPTS="-Xms16g -Xmx16g"` with `docker run`. .. Elasticsearch is I/O sensitive and the Docker storage driver is not ideal for fast I/O .. It allows the use of advanced https://docs.docker.com/engine/extend/plugins/#volume-plugins[Docker volume plugins] + -. If you are using the devicemapper storage driver make sure you are not using the default `loop-lvm` mode. Configure docker-engine to use https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/#configure-docker-with-devicemapper[direct-lvm] instead. +. If you are using the devicemapper storage driver, make sure you are not using +the default `loop-lvm` mode. Configure docker-engine to use +https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/#configure-docker-with-devicemapper[direct-lvm] +instead. + . Consider centralizing your logs by using a different https://docs.docker.com/engine/admin/logging/overview/[logging driver]. Also note that the default json-file logging driver is not ideally suited for production use. From d940a5d7d81def52244766cb2916be34bf318fdb Mon Sep 17 00:00:00 2001 From: Toby McLaughlin Date: Tue, 31 Oct 2017 15:09:41 +1100 Subject: [PATCH 6/6] Use US spelling in Docker docs --- docs/reference/setup/install/docker.asciidoc | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/reference/setup/install/docker.asciidoc b/docs/reference/setup/install/docker.asciidoc index 95bb1e700031e..1bcdefc5bc2b5 100644 --- a/docs/reference/setup/install/docker.asciidoc +++ b/docs/reference/setup/install/docker.asciidoc @@ -3,18 +3,18 @@ Elasticsearch is also available as Docker images. The images use https://hub.docker.com/_/centos/[centos:7] as the base image and -are available with {xpack-ref}/index.html[X-Pack]. +are available with {xpack-ref}/xpack-introduction.html[X-Pack]. A list of all published Docker images and tags can be found in https://www.docker.elastic.co[www.docker.elastic.co]. The source code can be found on https://github.com/elastic/elasticsearch-docker/tree/{branch}[GitHub]. ==== Image types -The images are available in three different configurations or "flavours". The -`basic` flavour, which is the default, ships with X-Pack Basic features +The images are available in three different configurations or "flavors". The +`basic` flavor, which is the default, ships with X-Pack Basic features pre-installed and automatically activated with a free licence. The `platinum` -flavour features all X-Pack functionally under a 30-day trial licence. The `oss` -flavour does not include X-Pack, and contains only open-source Elasticsearch. +flavor features all X-Pack functionally under a 30-day trial licence. The `oss` +flavor does not include X-Pack, and contains only open-source Elasticsearch. NOTE: {xpack-ref}/xpack-security.html[X-Pack Security] is enabled in the `platinum` image. To access your cluster, it's necessary to set an initial password for the