Better extension point for custom cluster state parts

[martijnvg]

Today custom cluster state, metadata and index metadata can be registered via adding static codeblock that invokes ClusterState|MetaData|IndexMetaData#registerProtoType(...) in the plugin class. Apart from the fact that requiring static code blocks for extension points is far from ideal, there is no mechanism that prevents code from registering custom cluster state parts after a node has started (i'm not sure what the behaviour is when this happens).

Instead we should have a ClusterStatePlugin interface that plugins can implements that allows registering custom cluster state, metadata and or index metadata. The interface should have 3 default methods that allows to register the 3 different custom cluster state parts that exist today.

Also as far as I know there is nothing that registers custom index metadata, so I wonder if we can remove that.

[nik9000]

Sounds about right.

[bleskes]

Discussed this in Fix it friday and we all feel it's a good idea. The only gotcha, which already exist now, is how to deal with recovered state which has custom metadata with no corresponding client. That is tricky because it can mean it some install went broken (and we don't want to just destroy the custom metadata) but it can also mean that someone uninstalled a plugin and the custom metadata needs to be removed.

[clintongormley]

@martijnvg are you taking this or is it adoptme?

[martijnvg]

@clintongormley I'll label it with an adoptme for now. When I start to work on this issue then I'll assign it to myself.

[martijnvg]

This has been implemented via #22336