Storing a non-secure setting in the keystore should prevent node start

[tvernum]

AbstractScopedSettings validates against unknown settings, and that validation reflects where we read the setting's value from (so the error message says "unknown setting" or "unknown secure setting" depending on context).

However, we only validate whether a setting is "known", we don't validate whether it has been added to the correct file.

This means, for example the following does not produce any errors (but really ought to)

echo maybe | bin/elasticsearch-keystore add node.master
bin/elasticsearch

[elasticmachine]

Pinging @elastic/es-core-infra

[jasontedor]

This is quite, shall we say, tricky because knowing what secure settings are possible and what are valid values requires loading plugins. Loading plugins allows them to execute code. The keystore tool is not currently locked down by the security manager. Plugins are untrusted, so we can't let them execute code without a security manager. Ergo, we can't know what are valid and invalid settings in this context.

[tvernum]

The issue isn't with adding it to the keystore (which we would like to prevent, but aren't in a position to do so) but that the node will start without complaint and ignore the non-secure setting you added to the keystore.

In my example, I would expect bin/elasticsearch to fail with something like "node.master is not a secure setting"