singularityware · vsoch · Aug 22, 2017 · Aug 2, 2017 · Aug 2, 2017 · Aug 2, 2017
diff --git a/_posts/recipes/2017-05-09-hostlibs-gpus-and-mpi.md b/_posts/recipes/2017-05-09-hostlibs-gpus-and-mpi.md
@@ -4,6 +4,11 @@ category: recipes
 permalink: tutorial-gpu-drivers-open-mpi-mtls
 ---
 
+**Note: _Much of the GPU portion of this tutorial is deprecated by the `--nv`
+option that automatically binds host system driver libraries into your 
+container at runtime.  See the [`exec`](/docs-exec#a-gpu-example) command for 
+an example_**
+
 Singularity does a fantastic job  of isolating you from the host so you don't
 have to muck about with `LD_LIBRARY_PATH`, you just get exactly the library
 versions you want. However, in some situations you need to use library

diff --git a/pages/docs/contributing/contributing-docs.md b/pages/docs/contributing/contributing-docs.md
@@ -3,12 +3,14 @@ title: Contributing to Documentation
 sidebar: main_sidebar
 permalink: contributing-docs
 folder: releases
+toc: false
 ---
 
 We (like almost all open source software providers) have a documentation dillemma... We tend to focus on the code features and functionality before working on documentation. And there is very good reason for this, we want to share the love so nobody feels left out!
 
 The following documentation page assumes one is running on OS X, but if you are not, you should be able to easily transpose the necessary commands to your operating system of choice.
 
+{% include toc.html %}
 
 ## Setting Up Your Development Environment
 
@@ -113,8 +115,16 @@ asciinema rec -w 1 demo-asciicast.js
 # To play
 asciinema play demo-asciicast.js
 ```
+Make sure to resize your terminal to 25 rows x 115 columns or less before you 
+begin your recording.  Otherwise it will not display properly on the webpage. To get the size of your current terminal (in lines and columns) you can use `tput`:
 
-Once you've generated an asciicast, you should drop the file (e.g., some `demo-asciicast.js`) into the `assets/asciicast` folder, and then include the following in the page or post:
+```
+echo -e "lines\ncols"|tput -S
+34
+80
+```
+
+Once you've generated an asciicast, you should drop the file (e.g., `demo-asciicast.js`) into the `assets/asciicast` folder. Since we will have many asciicasts here, please name it meaningfully. Then include the following in the page or post:
 
 ```bash
 {{ "{% include asciicast.html source='demo-asciicast.js' title='How to make demos' author='[email protected]'" }}%}

diff --git a/pages/docs/overview/faq.md b/pages/docs/overview/faq.md
@@ -213,7 +213,10 @@ See the Singularity on HPC page for more details.
 
 ### Does Singularity support containers that require GPUs?
 
-Yes, Singularity has been tested to run some test and diagnostic code from within a container without modification. There are however potential issues that can come into play when using GPUs, for instance there are version API compatibilities between kernel and user land which will have to be considered.
+Yes. Many users run GPU dependant code within Singularity containers.  The
+experimental `--nv` option allows you to leverage host GPUs without installing 
+system level drivers into your container. See the [`exec`](/docs-exec#a-gpu-example) command for
+an example.
 
 ## Container portability
 

diff --git a/pages/docs/user-docs/docs-bootstrap-image.md b/pages/docs/user-docs/docs-bootstrap-image.md
@@ -3,9 +3,9 @@ title: Getting Started with Bootstrap
 sidebar: user_docs
 permalink: bootstrap-image
 folder: docs
+toc: false
 ---
 
-## Bootstrapping a Container
 Bootstrapping is the process where we install an operating system and then configure it appropriately for a specified need. To do this we use a bootstrap definition file (a text file called `Singularity`) which is a recipe of how to specifically build the container. Here we will overview the sections, best practices, and a quick example.
 
 {% include toc.html %}
@@ -142,7 +142,17 @@ Version 2.0
 ```
 
 #### %environment
-Akin to labels, you can add pairs of `VARIABLE` and `VALUE` under environment to be sourced when the container is used as variables in the environment. The entire section is written to a file that gets sourced, so you should generally use the same conventions that you might use in a `bashrc` or `profile`. See <a href="/docs-environment-metadata">Environment and Metadata</a> for more information about these two sections.
+You can add environment variables to be sourced when the container is used in the `%environment` section. The entire section is written to a file that gets sourced, so you should generally use the same conventions that you might use in a `bashrc` or `profile`. 
+
+```
+%environment
+    VADER=badguy
+    LUKE=goodguy
+    SOLO=someguy
+    export VADER LUKE SOLO
+```
+
+See <a href="/docs-environment-metadata">Environment and Metadata</a> for more information about the `%labels` and `%environment` sections.
 
 
 #### %post

diff --git a/pages/docs/user-docs/docs-bootstrap.md b/pages/docs/user-docs/docs-bootstrap.md
@@ -6,10 +6,10 @@ toc: false
 folder: docs
 ---
 
-{% include toc.html %}
-
 The process of *bootstrapping* a Singularity container is equivalent to describing a recipe for the container creation. There are several recipe formats that Singularity supports, but only the primary format of version 2.3 will be documented here. If you want a general overview with examples, see <a href="/bootstrap-image">Bootstrapping an Image</a>. The detailed options for each of the header and sections are provided here.
 
+{% include toc.html %}
+
 ## The header fields:
 
 ### Bootstrap:
@@ -48,7 +48,7 @@ The Docker bootstrap module will create a core operating system image based on a
 Once the `Bootstrap` module has completed, the sections are identified and utilized if present. The following sections are supported in the bootstrap definition, and integrated during the bootstrap process:
 
 
-#### %setup
+### %setup
 This section blob is a Bourne shell scriptlet which will be executed on the host outside the container during bootstrap. The path to the container is accessible from within the running scriptlet environment via the variable `$SINGULARITY_ROOTFS`. For example, consider the following scriptlet:
 
 ```
@@ -65,8 +65,7 @@ As we investigate this example scriptlet, you will first see this is the outside
 
 *note: Any uncaught command errors that occur within the scriptlet will cause the entire build process to halt!*
 
-
-#### %post
+### %post
 Similar to the `%setup` section, this section will be executed once during bootstrapping, but this scriptlet will be run from inside the container. This is where you should put additional installation commands, downloads, and configuration into your containers. Here is an example to consider:
 
 ```
@@ -94,8 +93,15 @@ The above example runs inside the container, so in this case we will first insta
 
 *another note: This is not a good example of a reproducible definition because it is pulling Open MPI from a moving target. A better example, would be to pull a static released version, but this serves as a good example of building a `%post` scriptlet.*
 
+### %environment 
+Beginning with Singularity v2.3 you can set up your environment using the `%environment` section of the definition file.  For example, if you wanted to add a directory to your search path, you could do so like this.
+
+```
+%environment
+    export PATH=/opt/good/stuff:$PATH
+```
 
-#### %runscript
+### %runscript
 The `%runscript` is another scriptlet, but it does not get executed during bootstrapping. Instead it gets persisted within the container to a file called `/singularity` which is the execution driver when the container image is ***run*** (either via the `singularity run` command or via executing the container directly).
 
 When the `%runscript` is executed, all options are passed along to the executing script at runtime, this means that you can (and should) manage argument processing from within your runscript. Here is an example of how to do that:
@@ -108,7 +114,7 @@ When the `%runscript` is executed, all options are passed along to the executing
 
 In this particular runscript, the arguments are printed as a single string (`$*`) and then they are passed to `/usr/bin/python` via a quoted array (`$@`) which ensures that all of the arguments are properly parsed by the executed command. The `exec` command causes the given command to replace the current entry in the process table with the one that is to be called. This makes it so the runscript shell process ceases to exist, and the only process running inside this container is the called Python command.
 
-#### %test
+### %test
 You may choose to add a `%test` section to your definition file. This section will be run at the very end of the boostrapping process and will give you a chance to validate the container during the bootstrap process. If you are building on Singularity Hub, [it is a good practice](https://github.com/singularityhub/singularityhub.github.io/wiki/Generate-Images#add-tests) to have this test section so you can be sure that your container works as expected. A non-zero status code indicates that one or more of your tests did not pass. You can also execute this scriptlet through the container itself, such that you can always test the validity of the container itself as you transport it to different hosts. Extending on the above Open MPI `%post`, consider this example:
 
 ```

diff --git a/pages/docs/user-docs/docs-changing-containers.md b/pages/docs/user-docs/docs-changing-containers.md
@@ -7,7 +7,7 @@ folder: docs
 
 
 # Making Changes to an Existing Container
-It is possible that you may need to make changes to a container after it has been bootstrapped. For that, let's repeat the Singularity mantra "*A user inside a Singularity container is the same user as outside the container*". If you want to make changes to your container, you must mount the container as `--writable` so you can change the contents. Note that standard Linux ownership and permission rules pertain to files within the container, so the `--writable` option does not guarantee you can do things like install new software. This might be a bit confusing if you copy a container from one computer to another. If your pids are different on the two computers you will lose the ability to edit files you previous had write access to. In these instances, it might be best to modify your container as root and so you would first need to become root outside of the container. Let's examine the following example:
+It is possible that you may need to make changes to a container after it has been bootstrapped. For that, let's repeat the Singularity mantra "*A user inside a Singularity container is the same user as outside the container*". If you want to make changes to your container, you must mount the container as `--writable` so you can change the contents. Note that standard Linux ownership and permission rules pertain to files within the container, so the `--writable` option does not guarantee you can do things like install new software. This might be a bit confusing if you copy a container from one computer to another. If your numeric userid are different on the two computers you will lose the ability to edit files you previous had write access to. In these instances, it might be best to modify your container as root and so you would first need to become root outside of the container. Let's examine the following example:
 
 ## Installing Additional Software
 We strongly recommend that you add additional software installation to your bootstrap, and re-create the image. However if you must, you can use `shell` and `exec` with `--writable` to issue additional commands.

diff --git a/pages/docs/user-docs/docs-create-an-image.md b/pages/docs/user-docs/docs-create-an-image.md
@@ -7,7 +7,7 @@ folder: docs
 
 A Singularity image, which can be referred to as a "container," is a single file that contains a virtual file system. After creating an image you can install an operating system, applications, and save meta-data with it.
 
-Whereas Docker assembles images from layers that are stored on your computer (viewed with the docker -ps command), a Singularity image is just one file that can sit on your Desktop, in a folder on your cluster, or anywhere.
+Whereas Docker assembles images from layers that are stored on your computer (viewed with the `docker history` command), a Singularity image is just one file that can sit on your Desktop, in a folder on your cluster, or anywhere.
 
 Having Singularity containers housed within a single image file greatly simplifies management tasks such as sharing, copying, and branching your containers. It also means that standard Linux file system concepts like permissions, ownership, and ACLs apply to the container (e.g. I can give read only access to a colleague, or block access completely with a simple chmod command).
 

diff --git a/pages/docs/user-docs/docs-docker.md b/pages/docs/user-docs/docs-docker.md
@@ -125,9 +125,9 @@ From: tensorflow/tensorflow:latest
 relative_path.py /tmp/analysis2.py
 
 %environment
-
-TOPSECRET pancakes
-HELLO WORLD
+TOPSECRET=pancakes
+HELLO=WORLD
+export HELLO TOPSECRET
 
 %labels
 

diff --git a/pages/docs/user-docs/docs-environment-metadata.md b/pages/docs/user-docs/docs-environment-metadata.md
@@ -1,16 +1,16 @@
 ---
-title: Changing Existing Containers
+title: Environment and Metadata
 sidebar: user_docs
 permalink: docs-environment-metadata
 folder: docs
+toc: false
 ---
 
-## Container Metadata
+Singularity containers support environment variables and labels that can be added by user during the bootstrap process.
 
-Singularity containers have two level of metadata - environment variables, and labels from the user and bootstrap process.
+{% include toc.html %}
 
-
-### Environment
+## Environment
 
 If you are importing a Docker container, the environment will be imported as well. If you want to define custom environment variables in your bootstrap recipe file `Singularity` you can do that like this
 
@@ -19,7 +19,8 @@ Bootstrap:docker
 From: ubuntu:latest
 
 %environment
-VARIABLE_NAME=VARIABLE_VALUE
+    VARIABLE_NAME=VARIABLE_VALUE
+    export VARIABLE_NAME
 ```
 
 Forget something, or need a variable defined at runtime? You can set any variable you want inside the container by prefixing it with "SINGULARITYENV_". It will be transposed automatically and the prefix will be stripped. For example, let's say we want to set the variable `HELLO` to have value `WORLD`. We can do that as follows:
@@ -58,7 +59,7 @@ singularity exec centos7.img cat /.singularity.d/env/10-docker.sh
 export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
 ```
 
-### Labels
+## Labels
 Your container stores metadata about it's build, along with Docker labels, and your custom labels that you define in a bootstrap `%labels` section. For containers that are generated with Singularity version 2.4 and later, labels are represented using the <a href="http://label-schema.org/rc1/">rc1 Label Schema</a>. For example:
 
 ```