Sharding reference

Author: andreyaksenov

doc/reference/configuration/configuration_reference.rst

@@ -2389,3 +2389,270 @@ The ``security`` section defines configuration parameters related to various sec
     | Type: boolean
     | Default: false
     | Environment variable: TT_SECURITY_SECURE_ERASING
+
+
+
+
+..  _configuration_reference_sharding:
+
+sharding
+--------
+
+The ``sharding`` section defines configuration parameters related to sharding using :ref:`vshard <vshard>`.
+
+-   :ref:`sharding.bucket_count <configuration_reference_sharding_bucket_count>`
+-   :ref:`sharding.connection_outdate_delay <configuration_reference_sharding_connection_outdate_delay>`
+-   :ref:`sharding.discovery_mode <configuration_reference_sharding_discovery_mode>`
+-   :ref:`sharding.failover_ping_timeout <configuration_reference_sharding_failover_ping_timeout>`
+-   :ref:`sharding.lock <configuration_reference_sharding_lock>`
+-   :ref:`sharding.rebalancer_disbalance_threshold <configuration_reference_sharding_rebalancer_disbalance_threshold>`
+-   :ref:`sharding.rebalancer_max_receiving <configuration_reference_sharding_rebalancer_max_receiving>`
+-   :ref:`sharding.rebalancer_max_sending <configuration_reference_sharding_rebalancer_max_sending>`
+-   :ref:`sharding.rebalancer_mode <configuration_reference_sharding_rebalancer_mode>`
+-   :ref:`sharding.roles <configuration_reference_sharding_roles>`
+-   :ref:`sharding.sched_move_quota <configuration_reference_sharding_sched_move_quota>`
+-   :ref:`sharding.sched_ref_quota <configuration_reference_sharding_sched_ref_quota>`
+-   :ref:`sharding.shard_index <configuration_reference_sharding_shard_index>`
+-   :ref:`sharding.sync_timeout <configuration_reference_sharding_sync_timeout>`
+-   :ref:`sharding.zone <configuration_reference_sharding_zone>`
+
+
+
+..  _configuration_reference_sharding_bucket_count:
+
+..  confval:: sharding.bucket_count
+
+    The total number of buckets in a cluster.
+
+    This number should be several orders of magnitude larger than the potential number
+    of cluster nodes, considering potential scaling out in the foreseeable future.
+
+    **Example:**
+
+    If the estimated number of nodes is M, then the data set should be divided into
+    100M or even 1000M buckets, depending on the planned scaling out. This number is
+    greater than the potential number of cluster nodes in the system being designed.
+
+    Keep in mind that too many buckets can cause a need to allocate more memory to store
+    routing information. On the other hand, an insufficient number of buckets can lead to
+    decreased granularity when rebalancing.
+
+    |
+    | Type: integer
+    | Default: 3000
+    | Environment variable: TT_SHARDING_BUCKET_COUNT
+
+
+..  _configuration_reference_sharding_connection_outdate_delay:
+
+..  confval:: sharding.connection_outdate_delay
+
+    [TODO] Router: Time to outdate old objects on reload.
+
+    |
+    | Type: number
+    | Default: nil
+    | Environment variable: TT_SHARDING_CONNECTION_OUTDATE_DELAY
+
+
+..  _configuration_reference_sharding_discovery_mode:
+
+..  confval:: sharding.discovery_mode
+
+    [TODO] A mode of a bucket discovery fiber.
+    See details in :ref:`vshard.router.discovery_set() <router_api-discovery_set>`.
+
+    |
+    | Type: string
+    | Default: 'on'
+    | Possible values: 'on', 'off', 'once'
+    | Environment variable: TT_SHARDING_DISCOVERY_MODE
+
+
+..  _configuration_reference_sharding_failover_ping_timeout:
+
+..  confval:: sharding.failover_ping_timeout
+
+    [TODO] Timeout after which a ping is considered to be unacknowledged.
+    Used by failover fiber to detect if a node is down.
+
+    |
+    | Type: number
+    | Default: 5
+    | Environment variable: TT_SHARDING_FAILOVER_PING_TIMEOUT
+
+
+..  _configuration_reference_sharding_lock:
+
+..  confval:: sharding.lock
+
+    [TODO] Flag whether the replicaset is locked.
+    If set to true, the replicaset won't send nor receive any buckets.
+    Rebalancer will simply ignore it.
+    To work this option must be set at least on the affected replicaset itself in its vshard.storage.cfg.
+    Default is false.
+
+    Replica set scope.
+
+    |
+    | Type: boolean
+    | Default: nil
+    | Environment variable: TT_SHARDING_LOCK
+
+
+..  _configuration_reference_sharding_rebalancer_disbalance_threshold:
+
+..  confval:: sharding.rebalancer_disbalance_threshold
+
+    The maximum bucket disbalance threshold (in percent).
+    The disbalance is calculated for each replica set using the following formula:
+
+    .. code-block:: none
+
+        |etalon_bucket_count - real_bucket_count| / etalon_bucket_count * 100
+
+    |
+    | Type: number
+    | Default: 1
+    | Environment variable: TT_SHARDING_REBALANCER_DISBALANCE_THRESHOLD
+
+
+..  _configuration_reference_sharding_rebalancer_max_receiving:
+
+..  confval:: sharding.rebalancer_max_receiving
+
+    The maximum number of buckets that can be received in parallel by a single
+    replica set. This number must be limited, because when a new replica set is added to
+    a cluster, the rebalancer sends a very large amount of buckets from the existing
+    replica sets to the new one. This produces a heavy load on the new replica set.
+
+    **Example:**
+
+    Suppose ``rebalancer_max_receiving`` is equal to 100, ``bucket_count`` is equal to 1000.
+    There are 3 replica sets with 333, 333 and 334 buckets on each respectively.
+    When a new replica set is added, each replica set’s ``etalon_bucket_count`` becomes
+    equal to 250. Rather than receiving all 250 buckets at once, the new replica set
+    receives 100, 100 and 50 buckets sequentially.
+
+    |
+    | Type: integer
+    | Default: 100
+    | Environment variable: TT_SHARDING_REBALANCER_MAX_RECEIVING
+
+
+..  _configuration_reference_sharding_rebalancer_max_sending:
+
+..  confval:: sharding.rebalancer_max_sending
+
+    The degree of parallelism for :ref:`parallel rebalancing <vshard-parallel-rebalancing>`.
+
+    Works for storages only, ignored for routers.
+
+    |
+    | Type: integer
+    | Default: 1
+    | Maximum: 15
+    | Environment variable: TT_SHARDING_REBALANCER_MAX_SENDING
+
+
+..  _configuration_reference_sharding_rebalancer_mode:
+
+..  confval:: sharding.rebalancer_mode
+
+    [TODO] A rebalancer mode:
+
+    *   ``manual``
+    *   ``auto``
+    *   ``off``
+
+    |
+    | Type: string
+    | Default: 'auto'
+    | Environment variable: TT_SHARDING_REBALANCER_MODE
+
+
+..  _configuration_reference_sharding_roles:
+
+..  confval:: sharding.roles
+
+    [TODO] A sharding role:
+
+    *   ``router``
+    *   ``storage``
+    *   ``rebalancer``
+
+    Replica set scope.
+
+    |
+    | Type: array
+    | Default: nil
+    | Environment variable: TT_SHARDING_ROLES
+
+
+..  _configuration_reference_sharding_sched_move_quota:
+
+..  confval:: sharding.sched_move_quota
+
+    [TODO] Scheduler bucket move quota.
+
+    |
+    | Type: number
+    | Default: 1
+    | Environment variable: TT_SHARDING_SCHED_MOVE_QUOTA
+
+
+..  _configuration_reference_sharding_sched_ref_quota:
+
+..  confval:: sharding.sched_ref_quota
+
+    [TODO] Scheduler storage ref quota.
+
+    The scheduler shares storage time between bucket moves and storage refs fairly. The distribution depends on how long and frequent the moves and refs are. It can be configured using the storage options ``sched_move_quota`` and ``sched_ref_quota``. Keep in mind that the scheduler configuration may affect map-reduce requests if used during rebalancing.
+
+    |
+    | Type: number
+    | Default: 300
+    | Environment variable: TT_SHARDING_SCHED_REF_QUOTA
+
+
+..  _configuration_reference_sharding_shard_index:
+
+..  confval:: sharding.shard_index
+
+    The name or ID of a TREE index over the :ref:`bucket id <vshard-vbuckets>`.
+    Spaces without this index do not participate in a sharded Tarantool
+    cluster and can be used as regular spaces if needed. It is necessary to
+    specify the first part of the index, other parts are optional.
+
+    |
+    | Type: string
+    | Default: 'bucket_id'
+    | Environment variable: TT_SHARDING_SHARD_INDEX
+
+
+..  _configuration_reference_sharding_sync_timeout:
+
+..  confval:: sharding.sync_timeout
+
+    The timeout to wait for synchronization of the old master with replicas before demotion.
+    Used when switching a master or when manually calling the :ref:`sync() <storage_api-sync>` function.
+
+    |
+    | Type: number
+    | Default: 1
+    | Environment variable: TT_SHARDING_SYNC_TIMEOUT
+
+
+..  _configuration_reference_sharding_zone:
+
+..  confval:: sharding.zone
+
+    [TODO] Replica zone (see weighted routing in the section 'Replicas weight configuration');
+    Zone where the replica is located. String or number. See weights root config option for more info about what zones do. Default is nil. If a replica has no zone, in the weights matrix it gets the worst score.
+
+    Instance scope.
+
+    |
+    | Type: integer
+    | Default: nil
+    | Environment variable: TT_SHARDING_ZONE