Threadpool defaults assume single node per machine

[drewr]

The threadpool defaults are too aggressive if you run more than one ES node on a single machine. Would like to add a threadpool.nodes_per_machine setting (default 1) which is used to calculate the defaults. Could be that influences the value of availableProcessors that is then used in the settings.

[mattweber]

+1, I was going to ask about this on the webinar earlier today!

[mattweber]

I think using 2767c08 and setting the value to processors/nodes_on_machine would do the same thing correct?

[kimchy]

yea, we already have support for that in the processors setting, which we will base all computations based on that.

[drewr]

I had actually seen that but I didn't realize it was intended to serve this purpose. Makes total sense now though. Perhaps we could also support a nodes_per_machine to save the hassle of having to manually set it. I can envision having a heterogeneous environment where you have differing numbers of cores on machines but want to provision two nodes across the board. This could prevent people from having to do the work in chef or puppet to handle all cases. Too clever?

[kimchy]

@drewr good Q..., feels like this should be left to the user..., but would love to get feedback from others

[nik9000]

We won't run more than one node per machine without better support from the deb packages for any use cases I can cook up are just academic (for us) but even if we did we'd probably end up wanting more fine grained control then "divide the detected cores by n". I'm thinking of use cases like:

1. Run a master node in a separate vm from a data node on the same machine. processors would be set to 1 and -1 respectively.
2. Run two data nodes on a machine with a ton of ram or two disks or something. processors would both be set to 50%.
   And a use cases we'd never attempt but I imagine others might:
3. Run Elasticsearch on a node also containing a db/intensive app server as some kind of demonstration node. processors as is would work for this, maybe with the additional support of parsing percentages. For most demonstration use cases this doesn't matter but I can imagine some where is might.

[clintongormley]

No further interest since last year. Closing