[5pt] VShard router master discovery

[TarantoolBot]

Epic: Document master discovery
Related dev issue: tarantool/vshard#75
See also tarantool/vshard@551b106

Product: Tarantool
Root document: https://www.tarantool.io/en/doc/latest/reference/reference_rock/vshard/
SME: @ Gerold103

Details

Router used not to be able to find master nodes in the configured
replicasets on its own. It relied only on how they were specified
in the config.

This becomes a problem when master changes and the change is not
delivered to the router's config somewhy. For instance, the router
does not rely on a central config provider. Or it does rely, but
the provider can't deliver a new config due to any reason.

This is getting especially tricky with built-in automatic master
elections which are not supported by vshard yet, but they will be,
they won't depend on any config. Master role won't be pinned to
one node then.

Now there is a new feature to overcome the master search problem -
configurable automatic master discovery on the router.

Router goes to the replicasets, marked as having an auto master,
finds a master in them, and periodically checks if the master is
still a master.

When a master in a replicaset stops being a master, the router
walks all nodes of the replicaset and finds who is the new master.

To turn the feature on there is a new option in router's config:
master = 'auto'. It should be specified per-replicaset, and is
not compatible with specifying a master manually.

This is how a good config looks:

config = {
    sharding = {
        <replicaset uuid> = {
            master = 'auto',
            replicas = {...},
        },
        ...
    },
    ...
}

This is how a bad config looks:

config = {
    sharding = {
        <replicaset uuid> = {
            master = 'auto',
            replicas = {
                <replica uuid1> = {
                    master = true,
                    ...
                },
                <replica uuid2> = {
                    master = false,
                    ...
                },
            },
        },
        ...
    },
    ...
}

It will not work, because either master = 'auto' can be
specified, or the master is assigned manually. Not both at the
same time.

Master discovery works in its own fiber on the router, which is
activated only if at least one replicaset is configured to look
for the master. It wakes up with a certain period. But it is
possible to wake it up on demand using
vshard.router.master_search_wakeup() function. It does not do
anything if master search is not configured for any replicaset.

Requested by @Gerold103 in tarantool/vshard@27b0f3b.