diff --git a/README.rst b/README.rst index cf1f3ce650..76e220c033 100644 --- a/README.rst +++ b/README.rst @@ -84,7 +84,7 @@ As a managed service, Amazon SageMaker performs operations on your behalf on the Amazon SageMaker can perform only operations that the user permits. You can read more about which permissions are necessary in the `AWS Documentation `__. -The SageMaker Python SDK should not require any additional permissions. +The SageMaker Python SDK should not require any additional permissions aside from what is required for using SageMaker. However, if you are using an IAM role with a path in it, you should grant permission for ``iam:GetRole``. Licensing @@ -117,27 +117,28 @@ To run the unit tests with tox, run: To run the integration tests, the following prerequisites must be met 1. AWS account credentials are available in the environment for the boto3 client to use. -2. The AWS account has an IAM role named :code:`SageMakerRole` with the AmazonSageMakerFullAccess policy attached. +2. The AWS account has an IAM role named :code:`SageMakerRole`. + It should have the AmazonSageMakerFullAccess policy attached as well as a policy with `the necessary permissions to use Elastic Inference `__. We recommend selectively running just those integration tests you'd like to run. You can filter by individual test function names with: :: - pytest -k 'test_i_care_about' + tox -- -k 'test_i_care_about' You can also run all of the integration tests by running the following command, which runs them in sequence, which may take a while: :: - pytest tests/integ + tox -- tests/integ You can also run them in parallel: :: - pytest -n auto tests/integ + tox -- -n auto tests/integ Building Sphinx docs @@ -149,16 +150,16 @@ Building Sphinx docs make html -You can edit the templates for any of the pages in the docs by editing the .rst files in the "doc" directory and then running "``make html``" again. +You can edit the templates for any of the pages in the docs by editing the .rst files in the ``doc`` directory and then running ``make html`` again. MXNet SageMaker Estimators -------------------------- -By using MXNet SageMaker ``Estimators``, you can train and host MXNet models on Amazon SageMaker. +By using MXNet SageMaker Estimators, you can train and host MXNet models on Amazon SageMaker. -Supported versions of MXNet: ``1.3.0``, ``1.2.1``, ``1.1.0``, ``1.0.0``, ``0.12.1``. +Supported versions of MXNet: ``0.12.1``. ``1.0.0``, ``1.1.0``, ``1.2.1``, ``1.3.0``, ``1.4.0`` -Supported versions of MXNet for Elastic Inference: ``1.3.0`` +Supported versions of MXNet for Elastic Inference: ``1.3.0``, ``1.4.0`` We recommend that you use the latest supported version, because that's where we focus most of our development efforts. @@ -170,7 +171,7 @@ For more information, see `Using MXNet with the SageMaker Python SDK`_. TensorFlow SageMaker Estimators ------------------------------- -By using TensorFlow SageMaker ``Estimators``, you can train and host TensorFlow models on Amazon SageMaker. +By using TensorFlow SageMaker Estimators, you can train and host TensorFlow models on Amazon SageMaker. Supported versions of TensorFlow: ``1.4.1``, ``1.5.0``, ``1.6.0``, ``1.7.0``, ``1.8.0``, ``1.9.0``, ``1.10.0``, ``1.11.0``, ``1.12.0``. @@ -186,7 +187,7 @@ For more information, see `Using TensorFlow with the SageMaker Python SDK`_. Chainer SageMaker Estimators ---------------------------- -By using Chainer SageMaker ``Estimators``, you can train and host Chainer models on Amazon SageMaker. +By using Chainer SageMaker Estimators, you can train and host Chainer models on Amazon SageMaker. Supported versions of Chainer: ``4.0.0``, ``4.1.0``, ``5.0.0``. @@ -194,7 +195,7 @@ We recommend that you use the latest supported version, because that's where we For more information about Chainer, see https://github.com/chainer/chainer. -For more information about Chainer SageMaker ``Estimators``, see `Using Chainer with the SageMaker Python SDK`_. +For more information about Chainer SageMaker Estimators, see `Using Chainer with the SageMaker Python SDK`_. .. _Using Chainer with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_chainer.html @@ -202,7 +203,7 @@ For more information about Chainer SageMaker ``Estimators``, see `Using Chainer PyTorch SageMaker Estimators ---------------------------- -With PyTorch SageMaker ``Estimators``, you can train and host PyTorch models on Amazon SageMaker. +With PyTorch SageMaker Estimators, you can train and host PyTorch models on Amazon SageMaker. Supported versions of PyTorch: ``0.4.0``, ``1.0.0``. @@ -210,7 +211,7 @@ We recommend that you use the latest supported version, because that's where we For more information about PyTorch, see https://github.com/pytorch/pytorch. -For more information about PyTorch SageMaker ``Estimators``, see `Using PyTorch with the SageMaker Python SDK`_. +For more information about PyTorch SageMaker Estimators, see `Using PyTorch with the SageMaker Python SDK`_. .. _Using PyTorch with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_pytorch.html @@ -218,7 +219,7 @@ For more information about PyTorch SageMaker ``Estimators``, see `Using PyTorch Scikit-learn SageMaker Estimators --------------------------------- -With Scikit-learn SageMaker ``Estimators``, you can train and host Scikit-learn models on Amazon SageMaker. +With Scikit-learn SageMaker Estimators, you can train and host Scikit-learn models on Amazon SageMaker. Supported versions of Scikit-learn: ``0.20.0``. @@ -226,7 +227,7 @@ We recommend that you use the latest supported version, because that's where we For more information about Scikit-learn, see https://scikit-learn.org/stable/ -For more information about Scikit-learn SageMaker ``Estimators``, see `Using Scikit-learn with the SageMaker Python SDK`_. +For more information about Scikit-learn SageMaker Estimators, see `Using Scikit-learn with the SageMaker Python SDK`_. .. _Using Scikit-learn with the SageMaker Python SDK: https://sagemaker.readthedocs.io/en/stable/using_sklearn.html @@ -236,13 +237,13 @@ SageMaker Reinforcement Learning Estimators With Reinforcement Learning (RL) Estimators, you can use reinforcement learning to train models on Amazon SageMaker. -Supported versions of Coach: ``0.10.1`` with TensorFlow, ``0.11.0`` with TensorFlow or MXNet. +Supported versions of Coach: ``0.10.1``, ``0.11.1`` with TensorFlow, ``0.11.0`` with TensorFlow or MXNet. For more information about Coach, see https://github.com/NervanaSystems/coach -Supported versions of Ray: ``0.5.3`` with TensorFlow. +Supported versions of Ray: ``0.5.3``, ``0.6.5`` with TensorFlow. For more information about Ray, see https://github.com/ray-project/ray -For more information about SageMaker RL ``Estimators``, see `SageMaker Reinforcement Learning Estimators`_. +For more information about SageMaker RL Estimators, see `SageMaker Reinforcement Learning Estimators`_. .. _SageMaker Reinforcement Learning Estimators: src/sagemaker/rl/README.rst diff --git a/doc/using_mxnet.rst b/doc/using_mxnet.rst index e2d1f4c73c..c4fb3c5ec0 100644 --- a/doc/using_mxnet.rst +++ b/doc/using_mxnet.rst @@ -35,13 +35,10 @@ In the following sections, we'll discuss how to prepare a training script for ex Preparing the MXNet training script ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| WARNING | -+==========================================================================================================================================================+ -| The structure for training scripts changed starting at MXNet version 1.3. | -| Make sure you refer to the correct section of this README when you prepare your script. | -| For information on how to upgrade an old script to the new format, see `"Updating your MXNet training script" <#updating-your-mxnet-training-script>`__. | -+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +.. warning:: + The structure for training scripts changed starting at MXNet version 1.3. + Make sure you refer to the correct section of this README when you prepare your script. + For information on how to upgrade an old script to the new format, see `"Updating your MXNet training script" <#updating-your-mxnet-training-script>`__. For versions 1.3 and higher ^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -142,7 +139,7 @@ You don't have to use all the arguments, arguments you don't care about can be i def train(hyperparameters, num_gpus, **kwargs): pass -**Note: Writing a training script that imports correctly** +**Note: Writing a training script that imports correctly:** When SageMaker runs your training script, it imports it as a Python module and then invokes ``train`` on the imported module. Consequently, you should not include any statements that won't execute successfully in SageMaker when your module is imported. For example, don't attempt to open any local files in top-level statements in your training script. If you want to run your training script locally via the Python interpreter, look at using a ``___name__ == '__main__'`` guard, discussed in more detail here: https://stackoverflow.com/questions/419163/what-does-if-name-main-do . @@ -701,12 +698,12 @@ This one function should handle processing the input, performing a prediction, a The return object should be one of the following: For versions 1.4 and higher: ----------------------------- + - a tuple with two items: the response data and ``accept_type`` (the content type of the response data), or -- the response data: (the content type of the response will be set to either the accept header in the initial request or default to application/json) +- the response data: (the content type of the response will be set to either the accept header in the initial request or default to "application/json") For versions 1.3 and lower: ---------------------------- + - a tuple with two items: the response data and ``accept_type`` (the content type of the response data), or - a Flask response object: http://flask.pocoo.org/docs/1.0/api/#response-objects @@ -828,5 +825,7 @@ The Docker images extend Ubuntu 16.04. You can select version of MXNet by passing a ``framework_version`` keyword arg to the MXNet Estimator constructor. Currently supported versions are listed in the above table. You can also set ``framework_version`` to only specify major and minor version, e.g ``1.2``, which will cause your training script to be run on the latest supported patch version of that minor version, which in this example would be 1.2.1. Alternatively, you can build your own image by following the instructions in the SageMaker MXNet containers repository, and passing ``image_name`` to the MXNet Estimator constructor. -You can visit the SageMaker MXNet training containers repository here: https://github.com/aws/sagemaker-mxnet-container -You can visit the SageMaker MXNet serving containers repository here: https://github.com/aws/sagemaker-mxnet-serving-container +You can visit the SageMaker MXNet container repositories here: + +- training: https://github.com/aws/sagemaker-mxnet-container +- serving: https://github.com/aws/sagemaker-mxnet-serving-container diff --git a/doc/using_tf.rst b/doc/using_tf.rst index c984f952a7..d8463942ce 100644 --- a/doc/using_tf.rst +++ b/doc/using_tf.rst @@ -8,6 +8,7 @@ models on SageMaker Hosting. **Note:** This topic describes how to use script mode for TensorFlow versions 1.11 and later. For Documentation of the previous Legacy Mode versions, see: + * `1.4.1 `_ * `1.5.0 `_ * `1.6.0 `_ @@ -16,18 +17,15 @@ For Documentation of the previous Legacy Mode versions, see: * `1.9.0 `_ * `1.10.0 `_ -+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| WARNING | -+=============================================================================================================================================================================+ -| We have added a new format of your TensorFlow training script with TensorFlow version 1.11. | -| This new way gives the user script more flexibility. | -| This new format is called Script Mode, as opposed to Legacy Mode, which is what we support with TensorFlow 1.11 and older versions. | -| In addition we are adding Python 3 support with Script Mode. | -| Last supported version of Legacy Mode will be TensorFlow 1.12. | -| Script Mode is available with TensorFlow version 1.11 and newer. | -| Make sure you refer to the correct version of this README when you prepare your script. | -| You can find the Legacy Mode README `here `_. | -+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +.. warning:: + We have added a new format of your TensorFlow training script with TensorFlow version 1.11. + This new way gives the user script more flexibility. + This new format is called Script Mode, as opposed to Legacy Mode, which is what we support with TensorFlow 1.11 and older versions. + In addition we are adding Python 3 support with Script Mode. + Last supported version of Legacy Mode will be TensorFlow 1.12. + Script Mode is available with TensorFlow version 1.11 and newer. + Make sure you refer to the correct version of this README when you prepare your script. + You can find the Legacy Mode README `here `_. .. contents:: @@ -507,4 +505,7 @@ You can select version of TensorFlow by passing a ``framework_version`` keyword Alternatively, you can build your own image by following the instructions in the SageMaker TensorFlow containers repository, and passing ``image_name`` to the TensorFlow Estimator constructor. -For more information on the contents of the images, see the SageMaker TensorFlow containers repository here: https://github.com/aws/sagemaker-tensorflow-containers/ \ No newline at end of file +For more information on the contents of the images, see the SageMaker TensorFlow containers repositories here: + +- training: https://github.com/aws/sagemaker-tensorflow-container +- serving: https://github.com/aws/sagemaker-tensorflow-serving-container diff --git a/src/sagemaker/tensorflow/README.rst b/src/sagemaker/tensorflow/README.rst index bfcde6cb84..cae54f7185 100644 --- a/src/sagemaker/tensorflow/README.rst +++ b/src/sagemaker/tensorflow/README.rst @@ -70,4 +70,7 @@ You can select version of TensorFlow by passing a ``framework_version`` keyword Alternatively, you can build your own image by following the instructions in the SageMaker TensorFlow containers repository, and passing ``image_name`` to the TensorFlow Estimator constructor. -For more information on the contents of the images, see the SageMaker TensorFlow containers repository here: https://github.com/aws/sagemaker-tensorflow-containers/ +For more information on the contents of the images, see the SageMaker TensorFlow containers repositories here: + +- training: https://github.com/aws/sagemaker-tensorflow-container +- serving: https://github.com/aws/sagemaker-tensorflow-serving-container diff --git a/src/sagemaker/tensorflow/deploying_tensorflow_serving.rst b/src/sagemaker/tensorflow/deploying_tensorflow_serving.rst index 415073b24e..1d194843d1 100644 --- a/src/sagemaker/tensorflow/deploying_tensorflow_serving.rst +++ b/src/sagemaker/tensorflow/deploying_tensorflow_serving.rst @@ -425,7 +425,7 @@ You can also bring in external dependencies to help with your data processing. There are 2 ways to do this: 1. If you included ``requirements.txt`` in your ``source_dir`` or in - your dependencies, the container installs the Python dependencies at runtime using ``pip install -r``: + your dependencies, the container installs the Python dependencies at runtime using ``pip install -r``: .. code::