Skip to content

Data center awareness #602

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 2 additions & 155 deletions draft/multi-data-center-awareness.txt
Original file line number Diff line number Diff line change
@@ -1,159 +1,6 @@
==========================================================================
Location and Operational Segregation in MongoDB Operations and Deployments
==========================================================================

.. rename this file /administration/operational-segregation.txt

.. default-domain:: mongodb

Operational Overview
--------------------

MongoDB includes a cluster features that allow database administrators
and developers to segregate application operations to MongoDB
deployments by functional or geographical groupings. This capability
provides "data center awareness," where application operations target
he MongoDB deployment with consideration of the location of
:program:`mongod` instances in multi-data center deployments. In
single data center deployments, users can use these features to
provide "rack awareness," that allows users to control operations
within a single data center.

Beyond operational segregation based on location, MongoDB also
supports segregation based on functional parameters, to ensure that
certain :program:`mongod` instances are only used for reporting
workloads or that certain high-frequency portions of a sharded
collection only exist on specific shards.

Specifically, with MongoDB, you can:

- ensure write operations propagate to specific members of a replica
set, or specific members of replica sets.

- ensure that specific members of a replica set respond to queries.

- ensure that specific ranges of your :term:`shard key` balance onto and
reside on specific :term:`shards`.

- combine the above features in a single distributed deployment, on a
per-operation (for read and write operations) and collection (for
chunk distribution in sharded clusters distribution.) basis.

For full documentation of these features see the following
documentation in the MongoDB Manual:

- :ref:`Read Preferences <read-preference>`, which controls how drivers
help applications target read operations to members of a replica
set.

- :ref:`Write Concerns <replica-set-write-concern>`, which controls
how MongoDB ensures that write operations propagate to members of a
replica set.

- :ref:`Replica Set Tags <replica-set-configuration-tag-sets>`, which
controls how applications create and interact with custom groupings
of replica set members to create custom application specific a read
preference and write concerns.

- :ref:`Tag Aware Sharding <tag-aware-sharding>`, which allows MongoDB
administrators to define an application specific balancing policy,
to control how documents belonging to specific ranges of a shard key
distribute to shards in the :term:`sharded cluster`.

.. seealso::

Before adding operational segregation features to your application
and MongoDB deployment, become familiar with all documentation of
:doc:`replication </replication>` and :doc:`sharding </sharding>`,
particularly :doc:`/core/replication` and :doc:`/core/sharding`.

Examples of Operational Segregation
-----------------------------------

Increase Data Locality in Geographically Distributed Cluster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Functional Segregation for Reporting and Backups
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Increase Read Locality for Distributed Applications
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Ensure Geographical Redundancy for Write Operations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


................................................................................

.. this section will become a separate document.
perhaps /adminsitration/tag-aware-sharding

.. _tag-aware-sharding:

==================
Tag Aware Sharding
==================

For sharded clusters, MongoDB makes it possible to associate specific
ranges of a :term:`shard key` with a specific :term:`shard` or subset
of shards. This association dictates the policy of the cluster
balancer process as it balances the :term:`chunks <chunk>` around the
cluster. This capability enables the following deployment patterns:

- isolating a specific subset of data on specific set of shards.

- controlling the balancing policy so that in a geographically
distributed cluster the most relevant portions of the data set
reside on the shards with greatest proximity to the application
servers.

This document describes the behavior, operation, and use of tag aware
sharding in MongoDB deployments.

Behavior and Operations
-----------------------

- tags are cluster metadata. (implications.) Shell helpers to set the
metadata. The balancer does the rest.

- tags are arbitrary. tags are independent of replica-set member
tags.

- tags only dictate balancing policy, and depending on chunk splits,
and current distribution of data it may take time for the cluster to
achieve desired distribution

- (more?)

Administer Shard Tags
---------------------

Tag a Shard Key Range
~~~~~~~~~~~~~~~~~~~~~

Remove a Tag From a Shard Key Range
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

View Existing Shard Tags
~~~~~~~~~~~~~~~~~~~~~~~~

................................................................................

.. in addition to the above documents, this section requires:

- improved documentation on the
/applications/replica-set/#write-concern section about tag sets
(DOCS-710)

- a landing page called data-center-awareness.txt This will be
orphaned and unlinked from the main hierarchy, but will provide a
general overview into the relevant documentation.

................................................................................

.. below this fold may be dropped or completly rewritten as needed.


.. the remainder of be dropped or completly rewritten as needed, or
used in filling out the blanks in existing documentation.

Data Center Awareness Through Read Preferences and Write Concerns
-----------------------------------------------------------------
Expand Down
2 changes: 2 additions & 0 deletions source/administration.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ documentation in other sections including: :doc:`/sharding`,
:maxdepth: 1

administration/configuration
administration/operational-segregation
administration/journaling
administration/ssl
administration/monitoring
Expand All @@ -26,6 +27,7 @@ documentation in other sections including: :doc:`/sharding`,
- :doc:`/administration/replica-set-architectures`
- :doc:`/administration/sharded-clusters`
- :doc:`/administration/sharded-cluster-architectures`
- :doc:`/administration/tag-aware-sharding`
- :doc:`/core/indexes`
- :doc:`/administration/indexes`

Expand Down
83 changes: 83 additions & 0 deletions source/administration/operational-segregation.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
==========================================================================
Location and Operational Segregation in MongoDB Operations and Deployments
==========================================================================

.. rename this file /administration/operational-segregation.txt

.. default-domain:: mongodb

Operational Overview
--------------------

MongoDB includes a cluster features that allow database administrators
and developers to segregate application operations to MongoDB
deployments by functional or geographical groupings. This capability
provides "data center awareness," where application operations target
he MongoDB deployment with consideration of the location of
:program:`mongod` instances in multi-data center deployments. In
single data center deployments, users can use these features to
provide "rack awareness," that allows users to control operations
within a single data center.

Beyond operational segregation based on location, MongoDB also
supports segregation based on functional parameters, to ensure that
certain :program:`mongod` instances are only used for reporting
workloads or that certain high-frequency portions of a sharded
collection only exist on specific shards.

Specifically, with MongoDB, you can:

- ensure write operations propagate to specific members of a replica
set, or specific members of replica sets.

- ensure that specific members of a replica set respond to queries.

- ensure that specific ranges of your :term:`shard key` balance onto and
reside on specific :term:`shards`.

- combine the above features in a single distributed deployment, on a
per-operation (for read and write operations) and collection (for
chunk distribution in sharded clusters distribution.) basis.

For full documentation of these features see the following
documentation in the MongoDB Manual:

- :ref:`Read Preferences <read-preference>`, which controls how drivers
help applications target read operations to members of a replica
set.

- :ref:`Write Concerns <replica-set-write-concern>`, which controls
how MongoDB ensures that write operations propagate to members of a
replica set.

- :ref:`Replica Set Tags <replica-set-configuration-tag-sets>`, which
controls how applications create and interact with custom groupings
of replica set members to create custom application specific a read
preference and write concerns.

- :ref:`Tag Aware Sharding <tag-aware-sharding>`, which allows MongoDB
administrators to define an application specific balancing policy,
to control how documents belonging to specific ranges of a shard key
distribute to shards in the :term:`sharded cluster`.

.. seealso::

Before adding operational segregation features to your application
and MongoDB deployment, become familiar with all documentation of
:doc:`replication </replication>` and :doc:`sharding </sharding>`,
particularly :doc:`/core/replication` and :doc:`/core/sharding`.

Examples of Operational Segregation
-----------------------------------

Increase Data Locality in Geographically Distributed Cluster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Functional Segregation for Reporting and Backups
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Increase Read Locality for Distributed Applications
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Ensure Geographical Redundancy for Write Operations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
50 changes: 50 additions & 0 deletions source/administration/tag-aware-sharding.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
.. _tag-aware-sharding:

==================
Tag Aware Sharding
==================

.. default-domain:: mongodb

For sharded clusters, MongoDB makes it possible to associate specific
ranges of a :term:`shard key` with a specific :term:`shard` or subset
of shards. This association dictates the policy of the cluster
balancer process as it balances the :term:`chunks <chunk>` around the
cluster. This capability enables the following deployment patterns:

- isolating a specific subset of data on specific set of shards.

- controlling the balancing policy so that in a geographically
distributed cluster the most relevant portions of the data set
reside on the shards with greatest proximity to the application
servers.

This document describes the behavior, operation, and use of tag aware
sharding in MongoDB deployments.

Behavior and Operations
-----------------------

- tags are cluster metadata. (implications.) Shell helpers to set the
metadata. The balancer does the rest.

- tags are arbitrary. tags are independent of replica-set member
tags.

- tags only dictate balancing policy, and depending on chunk splits,
and current distribution of data it may take time for the cluster to
achieve desired distribution

- (more?)

Administer Shard Tags
---------------------

Tag a Shard Key Range
~~~~~~~~~~~~~~~~~~~~~

Remove a Tag From a Shard Key Range
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

View Existing Shard Tags
~~~~~~~~~~~~~~~~~~~~~~~~
24 changes: 24 additions & 0 deletions source/data-center-awareness.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
:orphan:

=====================
Data Center Awareness
=====================

MongoDB provides a number of features that allow application
developers and database administrators to customize the behavior of a
:term:`sharded cluster` or :term:`replica set` deployment so that it
MongoDB may be "data center aware," or allow operations

Consider the following documents:

.. toctree::
:maxdepth: 1

/administration/operational-segregation
/administration/tag-aware-sharding
/tutorial/deploy-geographically-distributed-replica-set

Additionally, consider the :ref:`replica-set-write-concern` and
:ref:`replica-set-read-preference` sections of the
:doc:`/applications/replication` document, which addresses
capabilities related to data center awareness.
8 changes: 5 additions & 3 deletions source/replication.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@ see :doc:`/administration/replica-sets`. To configure deployment
architecture, see :doc:`/administration/replica-set-architectures`. To
modify read and write operations, see :doc:`/applications/replication`.
For procedures for performing certain replication tasks, see the
:ref:`list of replication tutorials <replica-set-tutorials-list>`.
:ref:`list of replication tutorials <replica-set-tutorials-list>`. For
documentation of MongoDB's operational segregation capabilities for
replica set deployments see the :doc:`/data-center-awareness`

This section contains full documentation, tutorials, and pragmatic
guides, as well as links to the reference material that describes all
Expand Down Expand Up @@ -49,9 +51,9 @@ The following tutorials describe a number of common replica set
maintenance and operational practices in greater detail.

.. Updates to this tutorial list should also be made in
source/administration/replica-sets.txt
/source/administration/replica-sets.txt
and if appropriate in
source/includes/list-administration-tutorials.rest
/source/includes/list-administration-tutorials.rst

Getting Started with Replica Sets
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down
9 changes: 7 additions & 2 deletions source/sharding.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,10 @@ of machines. Sharding uses range-based portioning to distribute
:term:`documents <document>` based on a specific :term:`shard key`.

For a general introduction to sharding, cluster operations, and
relevant implications and administration see: :doc:`/faq/sharding`.
relevant implications and administration see:
:doc:`/faq/sharding`.



Sharded Cluster Use and Operation
---------------------------------
Expand Down Expand Up @@ -72,11 +75,13 @@ Application Development Patterns for Sharded Clusters

The following documents describe processes that application developers
may find useful when developing applications that use data stored in a
MongoDB sharded cluster.
MongoDB sharded cluster. For some cases you will also want to consider
the documentation of :doc:`/data-center-awareness`.

.. toctree::
:maxdepth: 1

administration/tag-aware-sharding
tutorial/enforce-unique-keys-for-sharded-collections
tutorial/convert-replica-set-to-replicated-shard-cluster

Expand Down
2 changes: 1 addition & 1 deletion source/tutorial/manage-sharded-cluster-balancer.txt
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ of operations:
.. code-block:: javascript

use config
db.settings.update({ _id : "balancer" }, { $unset : { activeWindow : true })
db.settings.update({ _id : "balancer" }, { $unset : { activeWindow : true } })

.. _sharding-balancing-disable-temporally:

Expand Down