[Zen2] PersistedState interface implementation

[andrershov]

Today GatewayMetaState is capable of atomically storing MetaData to
disk. We've also moved fields that are needed to be persisted in Zen2
from ClusterState to ClusterState.MetaData.CoordinationMetaData.

This PR implements PersistedState interface.

1. version and currentTerm are persisted as a part of Mainfest.
2. GatewayMetaState now have two descendants Zen1GatewayMetaState
   and Zen2GatewayMetaState (which implements PersistedState).
3. GatewayMetaState now constructs previousClusterState (including
   MetaData) and previousManifest inside the constructor, so that all
   PersistedState methods are usable as soon as GatewayMetaState
   instance is constructed. Also loadMetaData is renamed to
   getMetaData, because it just returns
   previousClusterState.metaData().
4. Currently when deciding whether to write IndexMetaData to disk,
   we're comparing current IndexMetaData version and received
   IndexMetaData version. This is not safe in Zen2 if term has changed.
   So updateClusterState now accepts writeWithoutComparingVersion
   method parameter. When it's set to true, we always write
   IndexMetaData to disk.
5. Things that are not covered by GatewayMetaStateTests are covered
   by Zen2GatewayMetaStateTests.

This PR does not inject Zen2GatewayMetaState in place of
Zen1GatewayMetaState and in place of InMemoryPersistedState, this
is a work to be done in a follow-up PR.

[elasticmachine]

Pinging @elastic/es-distributed

[ywelsch]

I've left some comments, the main one being about initialization where the localNode object is not available yet. I'm also not sure about the class names Zen1GatewayMetaState and Zen2GatewayMetaState, but we can address that in a follow-up.

[DaveCTurner]

I would really like to see an ESIntegTestCase that forms a Zen2 cluster and performs a full cluster restart here. Is that possible, or is the lack of changes to state recovery an obstruction? That kind of test would catch things like this
https://github.com/elastic/elasticsearch/pull/35819/files#r235791731 for instance.

[andrershov]

@ywelsch @DaveCTurner, I've made further changes, could you please take a look?

1. I've merged Zen1GatewayMetaState and Zen2GatewayMetaState into GatewayMetaState to make sure we can use GatewayMetaState for both Zen1 and Zen2 discovery.
2. GatewayService is no longer responsible for registering GatewayMetaState as ClusterStateApplier, now ZenDiscovery is responsible for that.
3. ZenDiscovery accepts ClusterApplier, not ClusterApplierService. And it turned out to be not easy to make it accept ClusterApplierService and make tests working. That's why I've moved addLowPriorityApplier to ClusterApplier interface, which is not elegant.
4. My first attempt was to change TestZenDiscovery (in zen2 mode) to use GatewayMetaState instead of InMemoryPersistedState, after this change 10 integration tests were failing, because we need to implement proper state recovery, which is a subject of separate PR. That's why I've implemented another option to use GatewayMetaState in place of InMemoryPersistedState with InMemoryPersistedState being default. IfInMemoryPeristedState is used, we're currently registering addLowPriorityApplier with GatewayMetaState, to make sure it receives all cluster state updates (otherwise 13 integration tests and one unit tests are failing).
5. There is one test that uses Zen2 with GatewayMetaState, see PersistedStateIT. It tests that cluster persistent settings work properly, it does not pass with InMemoryPersistentState. I was trying to make a test with IndexMetaData, but it requires proper recovery implementation.
6. Also, I've merged VotingTombstone class from zen2 branch and enhanced Zen2GatewayMetaStateTests.

[ywelsch]

I've left some more smaller comments. Looking good already.

[andrershov]

@ywelsch I'm done with the changes, ready for the next pass.

[ywelsch]

LGTM