Introduce single-node discovery #23595

jasontedor · 2017-03-15T18:51:17Z

This commit adds a single node discovery type. With this discovery type, a node will elect itself as master and never form a cluster with another node.

jasontedor · 2017-03-15T18:51:42Z

A follow-up will reveal the true purpose of this. 😉

bleskes

Main change looks good to me. Left some minor comments.

bleskes · 2017-03-15T23:38:30Z

core/src/main/java/org/elasticsearch/discovery/single/SingleNodeDiscovery.java

+
+    @Override
+    public DiscoveryStats stats() {
+        return null;


I'm doubting if this should returns a new DiscoveryStats(null). wdyt?

I pushed e10a245.

bleskes · 2017-03-15T23:42:49Z

core/src/main/java/org/elasticsearch/discovery/single/SingleNodeDiscovery.java

+
+                };
+        final ClusterStateTaskConfig config = ClusterStateTaskConfig.build(Priority.URGENT);
+        clusterService.submitStateUpdateTasks(


out of curiosity - why didn't you use a simple ClusterStateUpdateTask ?

No reasoning other than this is what my fingers did.

No, it wasn't my fingers. I see why I did this. A ClusterStateUpdateTask can not override runOnlyOnMaster which we need to set to false here. 😄

Oh right. I forgot about that. ++

bleskes · 2017-03-15T23:43:00Z

core/src/main/java/org/elasticsearch/discovery/zen/UnicastZenPing.java

            Stream.concat(pingingRound.getSeedNodes().stream(), nodesFromResponses.stream())
                .collect(Collectors.toMap(DiscoveryNode::getAddress, Function.identity(), (n1, n2) -> n1));

-


nit: unneeded?

I pushed f235c8b.

bleskes · 2017-03-15T23:46:16Z

core/src/test/java/org/elasticsearch/discovery/single/SingleNodeDiscoveryIT.java

+                .build();
+    }
+
+    public void testDoesNotPing() throws IOException, InterruptedException, ExecutionException {


I'm a bit confused by this test - as far as I can tell it doesn't do anything with single node discovery? what are we testing here?

A single node with discovery type set to single-node is started, and then we attempt to ping as zen discovery would. The purpose is to test that the node will not join a cluster with zen discovery configured. Does that make sense? Do you have an idea for a better test? Do you think that we need to test that two nodes with single-node configured as the discovery type will not join each other?

Oh, I see. So it's more like a "node has a good cloaking device" test. Reading the name I assumed you check that a single-node discovery doesn't ping other nodes (i.e., doesn't join). I think this test is nice (with a new name). Maybe another good test is to start two nodes with the same cluster name and port ranges that would normally cause them to find each other and see that they don't (it's a bit tricky w.r.t fixing ports but I think we can figure something out)

@bleskes I pushed 4c91310 to start a discussion. Let me know what you think.

bleskes · 2017-03-15T23:48:51Z

core/src/test/java/org/elasticsearch/discovery/single/SingleNodeDiscoveryTests.java

+            stack.push(clusterService);
+            final SingleNodeDiscovery discovery =
+                    new SingleNodeDiscovery(Settings.EMPTY, clusterService);
+            discovery.startInitialJoin();


doesn't this go async? should we wait via an observer? maybe take the code from node.java ?

Good catch. I pushed 5caab7d.

jasontedor · 2017-03-16T00:18:16Z

Thanks @bleskes; I have responded to your feedback.

bleskes

LGTM. See if you want to bite your teeth on that test suggestion

bleskes · 2017-03-16T00:17:13Z

core/src/test/java/org/elasticsearch/discovery/single/SingleNodeDiscoveryIT.java

+                .build();
+    }
+
+    public void testDoesNotPing() throws IOException, InterruptedException, ExecutionException {


Oh, I see. So it's more like a "node has a good cloaking device" test. Reading the name I assumed you check that a single-node discovery doesn't ping other nodes (i.e., doesn't join). I think this test is nice (with a new name). Maybe another good test is to start two nodes with the same cluster name and port ranges that would normally cause them to find each other and see that they don't (it's a bit tricky w.r.t fixing ports but I think we can figure something out)

bleskes · 2017-03-16T00:19:29Z

core/src/main/java/org/elasticsearch/discovery/single/SingleNodeDiscovery.java

+
+                };
+        final ClusterStateTaskConfig config = ClusterStateTaskConfig.build(Priority.URGENT);
+        clusterService.submitStateUpdateTasks(


Oh right. I forgot about that. ++

We need to negate this condition as other discovery types follow the zen logic too, only single-node needs to be excluded.

* master: Clear the interrupt flag before joining Migrate to max line length of 100 Docs: Corrected path to elasticsearch-plugin (elastic#23622) Docs: Add comma to reverse nested agg snippet Fix third-party audit task for Gradle 3.4 (elastic#23612) Adapter action future should restore interrupts Update scripting.asciidoc Unmark reindex as experimental CompletionSuggestionContext#toQuery() should also consider text if prefix/regex missing (elastic#23451) Docs: Specify that byte units use powers of 1024 (elastic#23574) Remove Settings.settingsBuilder (elastic#23575) Change params._source to params['_source'] in example. Fix example in documentation for Painless using _source. (elastic#21322) Remove extra line from license header Fix num docs to be positive in bucket deferring collector test Mapping: Fix NPE with scaled floats stats when field is not indexed (elastic#23528)

* master: Eclipse: move print margin to 100 columns Add support for fragment_length in the unified highlighter (elastic#23431)

* master: (137 commits) CONSOLEify the "using scripts" documentation Adds tests for cardinality and filter aggregations Add Backoff policy to azure repository Revert "Adds tests for cardinality and filter aggregations (elastic#23826)" Adds tests for cardinality and filter aggregations (elastic#23826) mute testDifferentRolesMaintainPathOnRestart Replace custom sort field with SortedSetSortField and SortedNumericSortField when possible (elastic#23827) Prevent nodes from joining if newer indices exist in the cluster (elastic#23843) Synchronized CollapseTopFieldDocs with lucenes relatives (elastic#23854) CONSOLEify analysis docs Adapted search_shards rest test to work with Perl To examine an exception in rest tests, the exception should be caught, not ignored Fixed bad YAML in rest tests Fix BootstrapForTesting blowup Ban Boolean#getBoolean Fix language in some docs CONSOLEify lang-analyzer docs Stricter parsing of remote node attribute Fix cross-cluster remote node gateway attributes FieldCapabilitiesRequest should implements Replaceable since it accepts index patterns ...

* master: Await termination after shutting down executors Fix initialization issue in ElasticsearchException Fix bulk queue size in thread pool docs [DOCS] Remove line about eager loading global ordinals GceDiscoverTests - remove intitial_state_timeout SpecificMasterNodesIT shouldn't use autoMinMasterNodes testRestorePersistentSettings doesn't to mess with discovery settings testDifferentRolesMaintainPathOnRestart shouldn't use auto managing of min master nodes

This commit adds a single node discovery type. With this discovery type, a node will elect itself as master and never form a cluster with another node. Relates #23595

jasontedor · 2017-04-04T07:16:53Z

Thanks @bleskes.

For the single node, dev example, the `discovery.type=single-node`[1] is a perfect fit and makes the example shorter and more self explanatory. [1] elastic#23595 [2] elastic#23598

For the single node, dev example, the `discovery.type=single-node`[1],[2] is a perfect fit and makes the example shorter and more self explanatory. Also expose the transport port, to help with dev use-cases using the transport client. [1] #23595 [2] #23598 Relates #26289

chitraj8 · 2018-02-05T21:15:09Z

I am trying to use elasticsearch version 5.6 and will need test it in a single node type. please suggest the configuration files that will require the change and get started.

jasontedor · 2018-02-05T21:26:36Z

@chitraj8 Please check the docs, and please use the forum if you need more help. We reserve GitHub issues for verified bug reports and feature requests, and merged pull requests are not the place to ask general questions.

Introduce single-node discovery

4d1f5e2

This commit adds a single node discovery type. With this discovery type, a node will elect itself as master and never form a cluster with another node.

jasontedor added :Distributed Coordination/Discovery-Plugins Anything related to our integration plugins with EC2, GCP and Azure >enhancement review v5.4.0 v6.0.0-alpha1 labels Mar 15, 2017

jasontedor requested a review from bleskes March 15, 2017 18:51

jasontedor added 3 commits March 15, 2017 11:53

Revert accidental change

27659dc

Remove unnecessary imports

fccd985

Cleanup

548a0de

jasontedor mentioned this pull request Mar 15, 2017

Disable bootstrap checks for single-node discovery #23598

Merged

jasontedor added 4 commits March 15, 2017 14:11

Fix NPE getting discovery type

3f9aa74

Fix variable name

5c6990a

Log the discovery type

0cd52f9

Fix license headers

93cefc2

bleskes suggested changes Mar 15, 2017

View reviewed changes

jasontedor added 3 commits March 15, 2017 17:04

Revert inadvertent change

f235c8b

Avoid test race when starting initial join

5caab7d

Return empty discovery stats

e10a245

bleskes approved these changes Mar 16, 2017

View reviewed changes

jasontedor added 7 commits March 16, 2017 10:59

Add test single nodes do not discover

4c91310

Change discovery condition

bfc3a44

We need to negate this condition as other discovery types follow the zen logic too, only single-node needs to be excluded.

Wait for pinging to finish

bad79b1

Remove repeat

94a79d4

Merge branch 'master' into single-node-discovery

f58cd38

* master: Eclipse: move print margin to 100 columns Add support for fragment_length in the unified highlighter (elastic#23431)

jasontedor mentioned this pull request Apr 4, 2017

Await termination after shutting down executors #23889

Merged

jasontedor merged commit 71293a8 into elastic:master Apr 4, 2017

jasontedor added a commit that referenced this pull request Apr 4, 2017

Introduce single-node discovery

e328e21

This commit adds a single node discovery type. With this discovery type, a node will elect itself as master and never form a cluster with another node. Relates #23595

jasontedor deleted the single-node-discovery branch April 4, 2017 07:16

This was referenced Apr 4, 2017

Allow disabling bootstrap checks explicitly #23580

Closed

can't use elasticsearch with circleci because of bootstrap checks #23866

Closed

Production/development mode should be a configurable option #21655

Closed

tvernum mentioned this pull request Jul 4, 2017

Unable to run Elasticsearch with single-node discovery setting. #25525

Closed

dliappis mentioned this pull request Aug 18, 2017

Docs: Use single-node discovery.type for dev example #26289

Merged

		Stream.concat(pingingRound.getSeedNodes().stream(), nodesFromResponses.stream())
		.collect(Collectors.toMap(DiscoveryNode::getAddress, Function.identity(), (n1, n2) -> n1));

Introduce single-node discovery #23595

Introduce single-node discovery #23595

Uh oh!

Conversation

jasontedor commented Mar 15, 2017

Uh oh!

jasontedor commented Mar 15, 2017

Uh oh!

bleskes left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jasontedor Mar 16, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jasontedor commented Mar 16, 2017

Uh oh!

bleskes left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jasontedor commented Apr 4, 2017

Uh oh!

chitraj8 commented Feb 5, 2018

Uh oh!

jasontedor commented Feb 5, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jasontedor Mar 16, 2017 •

edited

Loading