Improve upgrade experience of node level index settings

[s1monw]

In 5.0 we don't allow index settings to be specified on the node level ie.
in yaml files or via commandline argument. This can cause problems during
upgrade if this was used extensively. For instance if analyzers where
specified on a node level this might cause the index to be closed when
imported (see #17187). In such a case all indices relying on this
must be updated via PUT /${index}/_settings. Yet, this API has slightly
different semantics since it overrides existing settings. To make this less
painful this change adds a preserve_existing parameter on that API to ensure
we have the same semantics as if the setting was applied on the node level.

This change also adds a better error message and a change to the migration guide
to ensure upgrades are smooth if index settings are specified on the node level.

If a index setting is detected this change fails the node startup and prints a message
like this:

*************************************************************************************
Found index level settings on node level configuration.

Since elasticsearch 5.x index level settings can NOT be set on the nodes
configuration like the elasticsearch.yaml, in system properties or command line
arguments.In order to upgrade all indices the settings must be updated via the
/${index}/_settings API. Unless all settings are dynamic all indices must be closed
in order to apply the upgradeIndices created in the future should use index templates
to set default values.

Please ensure all required values are updated on all indices by executing:

curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
  "index.number_of_shards" : "1",
  "index.query.default_field" : "main_field",
  "index.translog.durability" : "async",
  "index.ttl.disable_purge" : "true"
}'
*************************************************************************************

[rjernst]

typo: two "if"s

[rjernst]

LGTM. Very helpful error message!

[bleskes]

+1 on going the extra mile. Is it possible/realistic I wonder if we should backport it to 2.x, so people can do that before they upgrade, when the migration plugin tells them they're in potential trouble?

[s1monw]

Is it possible/realistic I wonder if we should backport it to 2.x, so people can do that before they upgrade, when the migration plugin tells them they're in potential trouble?

I can try looking into it