Skip to content

[DOCS] Clarify use of CCS on ML nodes #66616

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Dec 22, 2020
Merged

[DOCS] Clarify use of CCS on ML nodes #66616

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4bf0c29
[DOCS] Clarify use of CCS on ML nodes
lcawl Dec 18, 2020
fceb692
[DOCS] Add recommendation to ML and transform node settings
lcawl Dec 18, 2020
073e64d
[DOCS] Clarifies default transform node behaviour
lcawl Dec 21, 2020
c7ede46
Update docs/reference/modules/node.asciidoc
lcawl Dec 22, 2020
6ea1858
[DOCS] Removes duplicate list of default node roles
lcawl Dec 22, 2020
51204ea
[DOCS] Fixes typo
lcawl Dec 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 14 additions & 12 deletions docs/reference/modules/node.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,20 +13,13 @@ clients.
// tag::modules-node-description-tag[]
All nodes know about all the other nodes in the cluster and can forward client
requests to the appropriate node.

By default, a node is all of the following types: master-eligible, data, ingest,
and (if available) machine learning. All data nodes are also transform nodes.
// end::modules-node-description-tag[]

TIP: As the cluster grows and in particular if you have large {ml} jobs or
{ctransforms}, consider separating dedicated master-eligible nodes from
dedicated data nodes, {ml} nodes, and {transform} nodes.

[[node-roles]]
==== Node roles

You can define the roles of a node by setting `node.roles`. If you don't
configure this setting, then the node has the following roles by default:
configure this setting, the node has the following roles by default:

* `master`
* `data`
Expand All @@ -37,9 +30,14 @@ configure this setting, then the node has the following roles by default:
* `ingest`
* `ml`
* `remote_cluster_client`
* `transform`

NOTE: If you set `node.roles`, the node is assigned only the roles you specify.

As the cluster grows and in particular if you have large {ml} jobs or
{ctransforms}, consider separating dedicated master-eligible nodes from
dedicated data nodes, {ml} nodes, and {transform} nodes.

<<master-node,Master-eligible node>>::

A node that has the `master` role (default), which makes it eligible to be
Expand Down Expand Up @@ -335,10 +333,12 @@ To create a dedicated {ml} node in the {default-dist}, set:

[source,yaml]
----
node.roles: [ ml ]
xpack.ml.enabled: true <1>
node.roles: [ ml, remote_cluster_client] <1>
xpack.ml.enabled: true <2>
----
<1> The `xpack.ml.enabled` setting is enabled by default.
<1> The `remote_cluster_client` role is optional but strongly recommended.
Otherwise, {ccs} fails when used in {ml} jobs or {dfeeds}. See <<remote-node>>.
<2> The `xpack.ml.enabled` setting is enabled by default.

Copy link

@droberts195 droberts195 Dec 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you are using {ccs} in any way it is strongly recommended to allow {ml} nodes to do remote searches unless you have a very good reason not to. It will likely lead to confusion and frustration if cross cluster searches usually work perfectly but fail when used in {ml} job or {dfeed} configs.

[[transform-node]]
==== [xpack]#{transform-cap} node#
Expand All @@ -351,8 +351,10 @@ To create a dedicated {transform} node in the {default-dist}, set:

[source,yaml]
----
node.roles: [ transform ]
node.roles: [ transform, remote_cluster_client ] <1>
----
<1> The `remote_cluster_client` role is optional but strongly recommended.
Otherwise, {ccs} fails when used in {transforms}. See <<remote-node>>.

Copy link

@droberts195 droberts195 Dec 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you are using {ccs} in any way it is strongly recommended to allow {transform} nodes to do remote searches unless you have a very good reason not to. It will likely lead to confusion and frustration if cross cluster searches usually work perfectly but fail when used in {transform} configs.

[[change-node-role]]
==== Changing the role of a node
Expand Down
16 changes: 10 additions & 6 deletions docs/reference/settings/ml-settings.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,15 +22,19 @@ hardware, you must disable {ml} (by setting `xpack.ml.enabled` to `false`).

`node.roles: [ ml ]`::
(<<static-cluster-setting,Static>>) Set `node.roles` to contain `ml` to identify
the node as a _{ml} node_ that is capable of running jobs. Every node is a {ml}
node by default.
the node as a _{ml} node_. If you want to run {ml} jobs, there must be at least
one {ml} node in your cluster.
+
If you use the `node.roles` setting, then all required roles must be explicitly
set. Consult <<modules-node>> to learn more.
By default, every node is a {ml} node. If you set `node.roles`, however,
you must explicitly specify all the required roles for the node. To learn more,
Copy link
Contributor

@robin13 robin13 Dec 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this sentence:

If you set node.roles, however, you must explicitly specify all the required roles for the node.

Would be better in docs/reference/modules/node.asciidoc

Copy link
Contributor Author

@lcawl lcawl Dec 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's already covered in the following sentence on that page: "If you set node.roles, the node is assigned only the roles you specify."

refer to <<modules-node>>.
+
IMPORTANT: On dedicated coordinating nodes or dedicated master nodes, do not set
[IMPORTANT]
====
* On dedicated coordinating nodes or dedicated master nodes, do not set
the `ml` role.

* It is strongly recommended that dedicated {ml} nodes also have the `remote_cluster_client` role; otherwise, {ccs} fails when used in {ml} jobs or {dfeeds}. See <<remote-node>>.
====

`xpack.ml.enabled`::
(<<static-cluster-setting,Static>>) Set to `true` (default) to enable {ml} APIs
Expand Down
16 changes: 9 additions & 7 deletions docs/reference/settings/transform-settings.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,16 @@ default.

`node.roles: [ transform ]`::
(<<static-cluster-setting,Static>>) Set `node.roles` to contain `transform` to
identify the node as a _transform node_. If you use the default settings for a
node, it does not have the `transform` role.
identify the node as a _transform node_. If you want to run {transforms}, there
must be at least one {transform} node in your cluster.
+
If a node doesn't have this role, the node cannot run transforms. If you want to
run transforms, there must be at least one transform node in your cluster.
+
If you use the `node.roles` setting, all required roles must be explicitly set.
Consult <<modules-node>> to learn more.
By default, every node is a {transform} node. If you set `node.roles`, however,
you must explicitly specify all the required roles for the node. To learn more,
refer to <<modules-node>>.
+
IMPORTANT: It is strongly recommended that dedicated {transform} nodes also have
the `remote_cluster_client` role; otherwise, {ccs} fails when used in
{transforms}. See <<remote-node>>.

`xpack.transform.enabled`::
deprecated:[7.8.0,Basic License features should always be enabled]
Expand Down