Distinguish single-machine and distributed model parallel

Author: mrshenli

intermediate_source/model_parallel_tutorial.py

@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 """
-Model Parallel Best Practices
+Single-Machine Model Parallel Best Practices
 ================================
 **Author**: `Shen Li <https://mrshenli.github.io/>`_
 
@@ -27,6 +27,13 @@
 of model parallel. It is up to the readers to apply the ideas to real-world
 applications.
 
+.. note::
+
+    For distributed model parallel training where a model spans multiple
+    servers, please refer to
+    `Getting Started With Distributed RPC Framework <rpc_tutorial.html>__
+    for examples and details.
+
 Basic Usage
 -----------
 """

intermediate_source/rpc_tutorial.rst

@@ -12,10 +12,10 @@ This tutorial uses two simple examples to demonstrate how to build distributed
 training with the `torch.distributed.rpc <https://pytorch.org/docs/master/rpc.html>`__
 package which is first introduced as an experimental feature in PyTorch v1.4.
 Source code of the two examples can be found in
-`PyTorch examples <https://github.com/pytorch/examples>`__
+`PyTorch examples <https://github.com/pytorch/examples>`__.
 
 Previous tutorials,
-`Getting Started With Distributed Data Parallel <https://pytorch.org/tutorials/intermediate/ddp_tutorial.html>`__
+`Getting Started With Distributed Data Parallel <ddp_tutorial.html>`__
  and `Writing Distributed Applications With PyTorch <https://pytorch.org/tutorials/intermediate/dist_tuto.html>`__,
 described `DistributedDataParallel <https://pytorch.org/docs/stable/_modules/torch/nn/parallel/distributed.html>`__
 which supports a specific training paradigm where the model is replicated across