singularityware · gmkurtzer · Jun 1, 2017 · Apr 10, 2017 · Apr 10, 2017 · Apr 10, 2017
diff --git a/Gemfile b/Gemfile
@@ -1,5 +1,5 @@
 source "https://rubygems.org"
 
-# gem "rails"
+#gem "rails"
 gem 'github-pages'
-gem 'jekyll'
+gem 'jekyll'
diff --git a/_data/sidebars/user_docs.yml b/_data/sidebars/user_docs.yml
@@ -19,18 +19,26 @@ entries:
       url: /docs-quick-start-installation
       output: web, pdf
 
+    - title: Create an Image
+      url: /create-image
+      output: web, pdf
+
     - title: Bootstrap an Image
       url: /bootstrap-image
       output: web, pdf
 
-    - title: Create an Image
-      url: /create-image
+    - title: Adding Content
+      url: /docs-content
       output: web, pdf
 
     - title: Bind Paths and Files
       url: /docs-mount
       output: web, pdf
 
+    - title: Environment and Metadata
+      url: /docs-environment-metadata
+      output: web, pdf
+
     - title: Change an Existing Container
       url: /docs-changing-containers
       output: web, pdf
@@ -67,10 +75,18 @@ entries:
       url: /docs-import
       output: web, pdf
 
-    - title: shell
-      url: /docs-shell
+    - title: inspect
+      url: /docs-inspect
+      output: web, pdf
+
+    - title: pull
+      url: /docs-pull
       output: web, pdf
 
     - title: run
       url: /docs-run
       output: web, pdf
+
+    - title: shell
+      url: /docs-shell
+      output: web, pdf
diff --git a/_data/topnav.yml b/_data/topnav.yml
@@ -26,10 +26,17 @@ topnav_dropdowns:
           url: /faq#troubleshooting
     - title: People
       folderitems:
-        - title: LBNL
-          external_url: http://www.lbl.gov/ 
-        - title: Stanford
-          external_url: https://srcc.stanford.edu/
+        - title: Gregory M. Kurtzer
+          external_url: https://github.com/gmkurtzer
+        - title: Vanessa Sochat
+          external_url: https://github.com/vsoch
+        - title: Michael Bauer
+          external_url: https://github.com/bauerm97
+        - title: Brian Bockelman
+          external_url: https://github.com/bbockelm
+        - title: Complete Authors List
+          external_url: https://github.com/singularityware/singularity/blob/master/AUTHORS
+
 
 ## Topnav single links
 ## if you want to list an external url, use external_url instead of url. the theme will apply a different link base.

diff --git a/_includes/gist.html b/_includes/gist.html
@@ -0,0 +1,9 @@
+<style>
+/* Better styles for embedding GitHub Gists */
+.gist{font-size:13px;line-height:20px;margin-bottom:20px;width:100%}
+.gist pre{font-family:Menlo,Monaco,'Bitstream Vera Sans Mono','Courier New',monospace !important}
+.gist-meta{font-family:Helvetica,Arial,sans-serif;font-size:13px !important}
+.gist-meta a{color:#26a !important;text-decoration:none}
+.gist-meta a:hover{color:#0e4071 !important}
+</style>
+<script src="https://gist.github.com/{{ include.username }}/{{ include.id  }}.js?file={{ include.file }}"></script>
diff --git a/_includes/head.html b/_includes/head.html
@@ -9,6 +9,7 @@
 
 
 <link rel="stylesheet" type="text/css" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css">
+<link href="https://fonts.googleapis.com/css?family=Open+Sans" rel="stylesheet">
 <!--<link rel="stylesheet" type="text/css" href="css/bootstrap.min.css">-->
 <link rel="stylesheet" href="{{ "assets/css/modern-business.css" }}">
 <link rel="stylesheet" href="{{ "assets/css/lavish-bootstrap.css" }}">

diff --git a/_layouts/page.html b/_layouts/page.html
@@ -36,8 +36,8 @@ <h1 class="post-title-main">{{ page.title }}</h1>
 
     <!-- Previous and next buttons-->
     <div class="row" style="padding-top:30px; margin-bottom:10px"><div class="col-md-12">
-        <a href="#"><button style="width:20%; height: 70px; float:left" class="hidden previous-button btn btn-lg btn-default">Previous</button></a>
-        <a href="#"><button style="width:20%; height: 70px; float:right" class="hidden next-button btn btn-lg btn-default">Next</button></a>
+        <a href="#"><button style="float:left" class="hidden previous-button btn btn-circle btn-default"><i class="fa-2x fa fa-angle-double-left"></i></button></a>
+        <a href="#"><button style="float:right" class="hidden next-button btn btn-circle btn-default"><i class="fa fa-angle-double-right fa-2x"></i></button></a>
     </div></div>
 
     <script>

diff --git a/_posts/news/2017-05-11-singularity-manuscript.md b/_posts/news/2017-05-11-singularity-manuscript.md
@@ -4,14 +4,13 @@ category: news
 permalink: 2017-singularity-manuscript
 ---
 
-We are proud to announce that you can now <a href="http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459" target="_blank">cite Singularity proper</a>! Our recent publication in PLOS One can be cited as follows:
+We are proud to announce that we have <a href="http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459" target="_blank">published Singularity proper!</a> Please cite the Singularity software as:
 
 ```
 Kurtzer GM, Sochat V, Bauer MW (2017) 
 Singularity: Scientific containers for mobility of compute. 
 PLoS ONE 12(5): e0177459. https://doi.org/10.1371/journal.pone.0177459
 ```
-
-Thanks to all of our contributors and community for helping us clunky software folk properly put together this paper. Looking forward to more great things to come, woohoo!
+Woohoo!
 
 {% include links.html %}
diff --git a/_posts/recipes/2017-05-09-hostlibs-gpus-and-mpi.md b/_posts/recipes/2017-05-09-hostlibs-gpus-and-mpi.md
@@ -0,0 +1,68 @@
+---
+title:  "Using Host libraries: GPU drivers and OpenMPI BTLs"
+category: recipes
+permalink: tutorial-gpu-drivers-open-mpi-mtls
+---
+
+Singularity does a fantastic job  of isolating you from the host so you don't
+have to muck about with `LD_LIBRARY_PATH`, you just get exactly the library
+versions you want. However, in some situations you need to use library
+versions that match host exactly. Two common ones are NVIDIA gpu
+driver user-space libraries, and OpenMPI transport drivers for high performance
+networking. There are many ways to solve these problems. Some people build a container and
+copy the version of the libs (installed on the host) into the container. 
+
+{% include toc.html %}
+
+## What We will learn today
+This document describes how to use a bind mount, symlinks and ldconfig so that when the host
+libraries are updated the container does not need to be rebuilt.
+
+**Note** this tutorial is tested with Singularity <a href="https://github.com/singularityware/singularity/commit/945c6ee343a1e6101e22396a90dfdb5944f442b6" target="_blank">commit 945c6ee343a1e6101e22396a90dfdb5944f442b6</a>,
+ which is part of the (current) development branch, and thus it should work with version 2.3 
+when that is released. The version of OpenMPI used is 2.1.0 (versions above 2.1 should work).
+
+## Environment
+
+In our environment we run CentOS 7 hosts with:
+
+  1. slurm located on `/opt/slurm-<version>` and the slurm user `slurm`
+  2. Mellanox network cards with drivers installed to `/opt/mellanox` (
+    Specifically we run a RoCEv1 network for Lustre and MPI communications)
+  3. NVIDIA GPUs with drivers installed to `/lib64`
+  4. OpenMPI (by default) for MPI processes
+
+## Creating your image
+Since we are building an ubuntu image, it may be easier to create an ubuntu VM
+to create the image. Alternatively you can follow the recipe
+<a href="/building-ubuntu-rhel-host" target="_blank"> here</a>.
+
+Use the following def file to create the image.
+
+{% include gist.html username='l1ll1' id='89b3f067d5b790ace6e6767be5ea2851' file='hostlibs.def' %}
+
+The mysterious `wget` line gets a list of all the libraries that the CentOS host
+has in `/lib64` that *we* think its safe to use in the container. Specifically
+these are things like nvidia drivers.
+
+{% include gist.html username='l1ll1' id='89b3f067d5b790ace6e6767be5ea2851' file='desired_hostlibs.txt' %}
+
+Also note:
+
+1.  in `hostlibs.def` we create a slurm user. Obviously if your `SlurmUser` is different you should change this name.
+2.  We make directories for `/opt` and `/usr/local/openmpi`. We're going to bindmount these from the host so we get all the bits of OpenMPI and Mellanox and Slurm that we need.
+
+
+## Executing your image
+On our system we do:
+
+```
+SINGULARITYENV_LD_LIBRARY_PATH=/usr/local/openmpi/2.1.0-gcc4/lib:/opt/munge-0.5.11/lib:/opt/slurm-16.05.4/lib:/opt/slurm-16.05.4/lib/slurm:/desired_hostlibs:/opt/mellanox/mxm/lib/
+export SINGULARITYENV_LD_LIBRARY_PATH
+```
+
+then
+
+```
+srun  singularity exec -B /usr/local/openmpi:/usr/local/openmpi -B /opt:/opt -B /lib64:/all_hostlibs hostlibs.img <path to binary>
+```