From 4bf0c290422306f6c1755434b89eac777cb40481 Mon Sep 17 00:00:00 2001 From: lcawl Date: Fri, 18 Dec 2020 09:40:17 -0800 Subject: [PATCH 1/6] [DOCS] Clarify use of CCS on ML nodes --- docs/reference/modules/node.asciidoc | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/reference/modules/node.asciidoc b/docs/reference/modules/node.asciidoc index 3646e9286fba4..564398f38ceef 100644 --- a/docs/reference/modules/node.asciidoc +++ b/docs/reference/modules/node.asciidoc @@ -340,6 +340,9 @@ xpack.ml.enabled: true <1> ---- <1> The `xpack.ml.enabled` setting is enabled by default. +If you want the node to use {ccs}, add `remote_cluster_client` to the list of +roles. See <>. + [[transform-node]] ==== [xpack]#{transform-cap} node# @@ -354,6 +357,9 @@ To create a dedicated {transform} node in the {default-dist}, set: node.roles: [ transform ] ---- +If you want the node to use {ccs}, add `remote_cluster_client` to the list of +roles. See <>. + [[change-node-role]] ==== Changing the role of a node From fceb692e140949105eab69f618ccbf9c94b91539 Mon Sep 17 00:00:00 2001 From: lcawl Date: Fri, 18 Dec 2020 12:55:02 -0800 Subject: [PATCH 2/6] [DOCS] Add recommendation to ML and transform node settings --- docs/reference/modules/node.asciidoc | 18 ++++++++---------- docs/reference/settings/ml-settings.asciidoc | 7 +++++-- .../settings/transform-settings.asciidoc | 7 ++++++- 3 files changed, 19 insertions(+), 13 deletions(-) diff --git a/docs/reference/modules/node.asciidoc b/docs/reference/modules/node.asciidoc index 564398f38ceef..cb2ec11a039ab 100644 --- a/docs/reference/modules/node.asciidoc +++ b/docs/reference/modules/node.asciidoc @@ -335,13 +335,12 @@ To create a dedicated {ml} node in the {default-dist}, set: [source,yaml] ---- -node.roles: [ ml ] -xpack.ml.enabled: true <1> +node.roles: [ ml, remote_cluster_client] <1> +xpack.ml.enabled: true <2> ---- -<1> The `xpack.ml.enabled` setting is enabled by default. - -If you want the node to use {ccs}, add `remote_cluster_client` to the list of -roles. See <>. +<1> The `remote_cluster_client` role is optional but strongly recommended. +Otherwise, {ccs} fails when used in {ml} jobs or {dfeeds}. See <>. +<2> The `xpack.ml.enabled` setting is enabled by default. [[transform-node]] ==== [xpack]#{transform-cap} node# @@ -354,11 +353,10 @@ To create a dedicated {transform} node in the {default-dist}, set: [source,yaml] ---- -node.roles: [ transform ] +node.roles: [ transform, remote_cluster_client ] <1> ---- - -If you want the node to use {ccs}, add `remote_cluster_client` to the list of -roles. See <>. +<1> The `remote_cluster_client` role is optional but strongly recommended. +Otherwise, {ccs} fails when used in {transforms}. See <>. [[change-node-role]] ==== Changing the role of a node diff --git a/docs/reference/settings/ml-settings.asciidoc b/docs/reference/settings/ml-settings.asciidoc index ba4e18047308c..f5be336b03812 100644 --- a/docs/reference/settings/ml-settings.asciidoc +++ b/docs/reference/settings/ml-settings.asciidoc @@ -28,9 +28,12 @@ node by default. If you use the `node.roles` setting, then all required roles must be explicitly set. Consult <> to learn more. + -IMPORTANT: On dedicated coordinating nodes or dedicated master nodes, do not set +[IMPORTANT] +==== +* On dedicated coordinating nodes or dedicated master nodes, do not set the `ml` role. - +* It is strongly recommended that dedicated {ml} nodes also have the `remote_cluster_client` role; otherwise, {ccs} fails when used in {ml} jobs or {dfeeds}. See <>. +==== `xpack.ml.enabled`:: (<>) Set to `true` (default) to enable {ml} APIs diff --git a/docs/reference/settings/transform-settings.asciidoc b/docs/reference/settings/transform-settings.asciidoc index fc9dc95f991ee..193e06dfa232a 100644 --- a/docs/reference/settings/transform-settings.asciidoc +++ b/docs/reference/settings/transform-settings.asciidoc @@ -17,13 +17,18 @@ default. `node.roles: [ transform ]`:: (<>) Set `node.roles` to contain `transform` to identify the node as a _transform node_. If you use the default settings for a -node, it does not have the `transform` role. +node, it does not have the `transform` role. However, by default all generic +data nodes are also {transform} nodes. + If a node doesn't have this role, the node cannot run transforms. If you want to run transforms, there must be at least one transform node in your cluster. + If you use the `node.roles` setting, all required roles must be explicitly set. Consult <> to learn more. ++ +IMPORTANT: It is strongly recommended that dedicated {transform} nodes also have +the `remote_cluster_client` role; otherwise, {ccs} fails when used in +{transforms}. See <>. `xpack.transform.enabled`:: deprecated:[7.8.0,Basic License features should always be enabled] From 073e64d9ee681c70cd7a2e076a40d8c7a44c590f Mon Sep 17 00:00:00 2001 From: lcawl Date: Mon, 21 Dec 2020 10:53:00 -0800 Subject: [PATCH 3/6] [DOCS] Clarifies default transform node behaviour --- docs/reference/modules/node.asciidoc | 2 +- docs/reference/settings/ml-settings.asciidoc | 9 +++++---- docs/reference/settings/transform-settings.asciidoc | 13 +++++-------- 3 files changed, 11 insertions(+), 13 deletions(-) diff --git a/docs/reference/modules/node.asciidoc b/docs/reference/modules/node.asciidoc index cb2ec11a039ab..315f3ef042676 100644 --- a/docs/reference/modules/node.asciidoc +++ b/docs/reference/modules/node.asciidoc @@ -15,7 +15,7 @@ All nodes know about all the other nodes in the cluster and can forward client requests to the appropriate node. By default, a node is all of the following types: master-eligible, data, ingest, -and (if available) machine learning. All data nodes are also transform nodes. +{transform} and (if available) {ml}. // end::modules-node-description-tag[] TIP: As the cluster grows and in particular if you have large {ml} jobs or diff --git a/docs/reference/settings/ml-settings.asciidoc b/docs/reference/settings/ml-settings.asciidoc index f5be336b03812..d295d65283409 100644 --- a/docs/reference/settings/ml-settings.asciidoc +++ b/docs/reference/settings/ml-settings.asciidoc @@ -22,11 +22,12 @@ hardware, you must disable {ml} (by setting `xpack.ml.enabled` to `false`). `node.roles: [ ml ]`:: (<>) Set `node.roles` to contain `ml` to identify -the node as a _{ml} node_ that is capable of running jobs. Every node is a {ml} -node by default. +the node as a _{ml} node_. If you want to run {ml} jobs, there must be at least +one {ml} node in your cluster. + -If you use the `node.roles` setting, then all required roles must be explicitly -set. Consult <> to learn more. +By default, every node is a {ml} node. If you set `node.roles`, however, +you must explicitly specify all the required roles for the node. To learn more, +refer to <>. + [IMPORTANT] ==== diff --git a/docs/reference/settings/transform-settings.asciidoc b/docs/reference/settings/transform-settings.asciidoc index 193e06dfa232a..eccd9bfba9a7c 100644 --- a/docs/reference/settings/transform-settings.asciidoc +++ b/docs/reference/settings/transform-settings.asciidoc @@ -16,15 +16,12 @@ default. `node.roles: [ transform ]`:: (<>) Set `node.roles` to contain `transform` to -identify the node as a _transform node_. If you use the default settings for a -node, it does not have the `transform` role. However, by default all generic -data nodes are also {transform} nodes. +identify the node as a _transform node_. If you want to run {transforms}, there +must be at least one {transform} node in your cluster. + -If a node doesn't have this role, the node cannot run transforms. If you want to -run transforms, there must be at least one transform node in your cluster. -+ -If you use the `node.roles` setting, all required roles must be explicitly set. -Consult <> to learn more. +By default, every node is a {tranform} node. If you set `node.roles`, however, +you must explicitly specify all the required roles for the node. To learn more, +refer to <>. + IMPORTANT: It is strongly recommended that dedicated {transform} nodes also have the `remote_cluster_client` role; otherwise, {ccs} fails when used in From c7ede462d87052759054041880ea08468e89705e Mon Sep 17 00:00:00 2001 From: Lisa Cawley Date: Tue, 22 Dec 2020 08:23:50 -0800 Subject: [PATCH 4/6] Update docs/reference/modules/node.asciidoc Co-authored-by: David Roberts --- docs/reference/modules/node.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/modules/node.asciidoc b/docs/reference/modules/node.asciidoc index 315f3ef042676..418f79485aebb 100644 --- a/docs/reference/modules/node.asciidoc +++ b/docs/reference/modules/node.asciidoc @@ -15,7 +15,7 @@ All nodes know about all the other nodes in the cluster and can forward client requests to the appropriate node. By default, a node is all of the following types: master-eligible, data, ingest, -{transform} and (if available) {ml}. +{transform}, remote_cluster_client and (if available) {ml}. // end::modules-node-description-tag[] TIP: As the cluster grows and in particular if you have large {ml} jobs or From 6ea1858e0f4d4a4ccac450d6096dde9d26206be5 Mon Sep 17 00:00:00 2001 From: lcawl Date: Tue, 22 Dec 2020 08:43:49 -0800 Subject: [PATCH 5/6] [DOCS] Removes duplicate list of default node roles --- docs/reference/modules/node.asciidoc | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/docs/reference/modules/node.asciidoc b/docs/reference/modules/node.asciidoc index 418f79485aebb..402e159cd3eb7 100644 --- a/docs/reference/modules/node.asciidoc +++ b/docs/reference/modules/node.asciidoc @@ -13,20 +13,13 @@ clients. // tag::modules-node-description-tag[] All nodes know about all the other nodes in the cluster and can forward client requests to the appropriate node. - -By default, a node is all of the following types: master-eligible, data, ingest, -{transform}, remote_cluster_client and (if available) {ml}. // end::modules-node-description-tag[] -TIP: As the cluster grows and in particular if you have large {ml} jobs or -{ctransforms}, consider separating dedicated master-eligible nodes from -dedicated data nodes, {ml} nodes, and {transform} nodes. - [[node-roles]] ==== Node roles You can define the roles of a node by setting `node.roles`. If you don't -configure this setting, then the node has the following roles by default: +configure this setting, the node has the following roles by default: * `master` * `data` @@ -37,9 +30,14 @@ configure this setting, then the node has the following roles by default: * `ingest` * `ml` * `remote_cluster_client` +* `transform` NOTE: If you set `node.roles`, the node is assigned only the roles you specify. +As the cluster grows and in particular if you have large {ml} jobs or +{ctransforms}, consider separating dedicated master-eligible nodes from +dedicated data nodes, {ml} nodes, and {transform} nodes. + <>:: A node that has the `master` role (default), which makes it eligible to be From 51204ea3c65d217429b955d9f7c191808bcf575d Mon Sep 17 00:00:00 2001 From: lcawl Date: Tue, 22 Dec 2020 09:52:44 -0800 Subject: [PATCH 6/6] [DOCS] Fixes typo --- docs/reference/settings/transform-settings.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/settings/transform-settings.asciidoc b/docs/reference/settings/transform-settings.asciidoc index eccd9bfba9a7c..1463c0e0cea34 100644 --- a/docs/reference/settings/transform-settings.asciidoc +++ b/docs/reference/settings/transform-settings.asciidoc @@ -19,7 +19,7 @@ default. identify the node as a _transform node_. If you want to run {transforms}, there must be at least one {transform} node in your cluster. + -By default, every node is a {tranform} node. If you set `node.roles`, however, +By default, every node is a {transform} node. If you set `node.roles`, however, you must explicitly specify all the required roles for the node. To learn more, refer to <>. +