Gpus

_config.yml

@@ -106,3 +106,5 @@ asciicast:
 # solarized-dark
 # solarized-light
 # monokai
+gems:
+- jekyll-gist

_data/sidebars/user_docs.yml

@@ -19,18 +19,26 @@ entries:
       url: /docs-quick-start-installation
       output: web, pdf
 
+    - title: Create an Image
+      url: /create-image
+      output: web, pdf
+
     - title: Bootstrap an Image
       url: /bootstrap-image
       output: web, pdf
 
-    - title: Create an Image
-      url: /create-image
+    - title: Adding Content
+      url: /docs-content
       output: web, pdf
 
     - title: Bind Paths and Files
       url: /docs-mount
       output: web, pdf
 
+    - title: Environment and Metadata
+      url: /docs-environment-metadata
+      output: web, pdf
+
     - title: Change an Existing Container
       url: /docs-changing-containers
       output: web, pdf

_data/topnav.yml

@@ -26,10 +26,17 @@ topnav_dropdowns:
           url: /faq#troubleshooting
     - title: People
       folderitems:
-        - title: LBNL
-          external_url: http://www.lbl.gov/ 
-        - title: Stanford
-          external_url: https://srcc.stanford.edu/
+        - title: Gregory M. Kurtzer
+          external_url: https://github.com/gmkurtzer
+        - title: Vanessa Sochat
+          external_url: https://github.com/vsoch
+        - title: Michael Bauer
+          external_url: https://github.com/bauerm97
+        - title: Brian Bockelman
+          external_url: https://github.com/bbockelm
+        - title: Complete Authors List
+          external_url: https://github.com/singularityware/singularity/blob/master/AUTHORS
+
 
 ## Topnav single links
 ## if you want to list an external url, use external_url instead of url. the theme will apply a different link base.

_includes/head.html

@@ -9,6 +9,7 @@
 
 
 <link rel="stylesheet" type="text/css" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css">
+<link href="https://fonts.googleapis.com/css?family=Open+Sans" rel="stylesheet">
 <!--<link rel="stylesheet" type="text/css" href="css/bootstrap.min.css">-->
 <link rel="stylesheet" href="{{ "assets/css/modern-business.css" }}">
 <link rel="stylesheet" href="{{ "assets/css/lavish-bootstrap.css" }}">

_layouts/page.html

@@ -36,8 +36,8 @@ <h1 class="post-title-main">{{ page.title }}</h1>
 
     <!-- Previous and next buttons-->
     <div class="row" style="padding-top:30px; margin-bottom:10px"><div class="col-md-12">
-        <a href="#"><button style="width:20%; height: 70px; float:left" class="hidden previous-button btn btn-lg btn-default">Previous</button></a>
-        <a href="#"><button style="width:20%; height: 70px; float:right" class="hidden next-button btn btn-lg btn-default">Next</button></a>
+        <a href="#"><button style="float:left" class="hidden previous-button btn btn-circle btn-default"><i class="fa-2x fa fa-angle-double-left"></i></button></a>
+        <a href="#"><button style="float:right" class="hidden next-button btn btn-circle btn-default"><i class="fa fa-angle-double-right fa-2x"></i></button></a>
     </div></div>
 
     <script>

_posts/news/2017-04-28-singularity-llc.md

@@ -0,0 +1,25 @@
+---
+title:  "Singularity Big Announcement"
+category: news
+permalink: 2017-singularity-llc
+---
+
+### Notice to all Singularity users, contributors, and collaborators
+
+This is an important announcement from Greg Kurtzer, founder and lead developer of Singularity.
+
+Singularity continues to gain popularity and traction very quickly. This growth has been extremely demanding on my time and is outpacing my ability to sustain the broad amount of use cases for the rapidly expanding community with me working on it only part time. In order to meet this rapidly expanding growth, I have created a new organization, SingularityWare, LLC, which is dedicated to further developing Singularity, supporting the associated open source community and growing the project. To facilitate this, I have partnered with a company, RStor, Inc.
+
+RStor is not only interested in the success of Singularity, as an open source, scientific enabling platform, but they are bringing their own very compelling and much needed cloud based storage/RDM platform to market. A partnership like this (containerized computing coupled with a cloud based storage platform) offers a very exciting and synergistic value to research computing.
+
+In addition to continuing my leadership of Singularity (and the new LLC), I will be maintaining my association with Lawrence Berkeley National Laboratory, as a scientific advisor as well as continuing other efforts I am associated with (e.g. Warewulf and OpenHPC).
+
+This means that I now have the resources and ability to both offer support for Singularity and hire key developers, and I would like to invite interested people to send me your resume! I am looking to hire both part time and full time staff engineers as well as University collaborations to fund interns, graduates and postdocs for Singularity development and relevant involvement.
+
+Aside from additional resources, dedication, attention, and growth, the overall direction of the project will remain the same.
+
+If you have any questions, comments or want to be part of this exciting new endeavor, please feel free to contact me directly.
+
+Thank you!
+
+{% include links.html %}

_posts/recipes/2017-05-09-hostlibs-gpus-and-mpi.md

@@ -0,0 +1,64 @@
+---
+title:  "Using Host libraries: GPU drivers and OpenMPI BTLs"
+category: recipes
+permalink: singularity-tutorial
+---
+
+Singularity does a fantastic job  of isolating you from the host so you don't
+have to muck about with 'LD_LIBRARY_PATH', you just get exactly the library
+versions you want. However in some situations you actually need to use library
+versions that match exactly host libraries. Two common ones are NVIDIA gpu
+drivers and user space libraries and OpenMPI transport drivers for high performance
+networking.  
+
+There are many ways to solve these problems. Some people build a container and
+copy the version of the libs installed on the host into the container. This document
+describes how to use a bind mount, symlinks and ldconfig so that when the host
+libraries are updated the container does not need to be rebuilt.
+{% include toc.html %}
+
+**Note this tutorial is tested with Singularity commit 945c6ee343a1e6101e22396a90dfdb5944f442b6
+(version 2.3 should work once released)
+and OpenMPI version 2.1.0 (versions above 2.1 should work)**
+
+## Environment
+
+In our environment we run CentOS 7 hosts with:
+  1. slurm located on '/opt/slurm-<version>' and the slurm user 'slurm'
+  2. Mellanox network cards with drivers installed to '/opt/mellanox' (
+    Specifically we run a RoCEv1 network for Lustre and MPI communications)
+  3. NVIDIA GPUs with drivers installed to '/lib64'
+  4. OpenMPI (by default) for MPI processes
+
+## Creating your image
+
+Since we are building an ubuntu image, it may be easier to create an ubuntu VM
+to create the image. Alternatively you can follow the recipe
+<a href="/building-ubuntu-rhel-host"> here </a>.
+
+Use the following def file to create the image.
+
+{% gist l1ll1/89b3f067d5b790ace6e6767be5ea2851 hostlibs.def %}
+
+The mysterious wget line gets a list of all the libraries that the CentOS host
+has in '/lib64' that *we* think its safe to use in the container. Specifically
+this is things like nvidia drivers.
+
+{% gist l1ll1/89b3f067d5b790ace6e6767be5ea2851 desired_hostlibs.txt %}
+
+Also note:
+
+1.  in 'hostlibs.def' we create a slurm user. Obviously if your 'SlurmUser'
+is different you should change this name.
+2.  We make directories for '/opt' and '/usr/local/openmpi'. We're going to
+bindmount these from the host so we get all the bits of OpenMPI and Mellanox
+and Slurm that we need.
+
+## Executing your image
+On our system we do
+```export SINGULARITYENV_LD_LIBRARY_PATH=/usr/local/openmpi/2.1.0-gcc4/lib:/opt/munge-0.5.11/lib:/opt/slurm-16.05.4/lib:/opt/slurm-16.05.4/lib/slurm:/desired_hostlibs:/opt/mellanox/mxm/lib/
+```
+then
+```
+srun  singularity exec -B /usr/local/openmpi:/usr/local/openmpi -B /opt:/opt -B /lib64:/all_hostlibs hostlibs.img <path to binary>
+```