Docker via Singularity
Kamiak has the ability to run Docker containers by utilizing a tool called Singularity.
What is a Docker container? According to Docker’s documentation, “a container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings.” To add to this, containers are not virtual machines.
What is Singularity? Singularity is a utility which enables the use of containers in HPC environments such as Kamiak. You don’t use docker’s commands but instead use singularity to create, manage, and run containers.
When running a compute job on Kamiak not using containers, you are running in what we’ll refer to as the host environment. This includes the operating system, software, libraries, devices (Infiniband, GPU), etc. that is a Kamiak compute node. When running a compute job on Kamiak within a container, you are running in the container’s environment and cannot access anything in the host environment. However, some aspects of the host environment are copied into every container. This includes:
- All mount points to Kamiak storage (/data, /home, /scratch, etc.)
- /dev (including Infiniband and GPU devices)
To start with, let’s look around the host environment for a moment. We can see Kamiak compute nodes (as of this writing) run CentOS 7 on a 3.10.0-514 kernel.
$ uname -r 3.10.0-514.10.2.el7.x86_64 $ cat /etc/os-release NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"
We’ll also create a scratch workspace to work within, but you can use any storage you have access to.
$ mkworkspace Successfully created workspace. Details: Workspace: /path/to/workspace User: john.doe Group: its_p_sys_ur_kam-mygroup Expiration: 2017-08-07 14:55:28.384943 $ cd /path/to/workspace
Now we can to load singularity’s module, pull down the latest Ubuntu container from Docker Hub, and run it.
$ module load singularity/2.3.1 $ singularity pull docker://ubuntu:latest Initializing Singularity image subsystem Opening image file: ubuntu-latest.img Creating 224MiB image Binding image to loop Creating file system within image Image is done: ubuntu-latest.img Docker image path: index.docker.io/library/ubuntu:latest Cache folder set to /home/my.nid/.singularity/docker Importing: base Singularity environment Done. Container is at: ubuntu-latest.img $ singularity shell ubuntu-latest.img Singularity: Invoking an interactive shell within container... Singularity ubuntu-latest.img:/path/to/workspace>
You are now running within the container. Let’s compare it to the host environment and verify we’re really in an Ubuntu container.
Singularity ubuntu-latest.img:/path/to/workspace> uname -r 3.10.0-514.10.2.el7.x86_64 Singularity ubuntu-latest.img:/path/to/workspace> cat /etc/os-release NAME="Ubuntu" VERSION="16.04.2 LTS (Xenial Xerus)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 16.04.2 LTS" VERSION_ID="16.04" HOME_URL="http://www.ubuntu.com/" SUPPORT_URL="http://help.ubuntu.com/" BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/" VERSION_CODENAME=xenial UBUNTU_CODENAME=xenial
Exit the shell with
exit to return to the host environment. Notice that while in the container we appear to be running Ubuntu – because we are – but we are still running on the host’s kernel since a container is not a virtual machine. Let’s see how we can use this Ubuntu image in a compute job. Create a file with the following submission script and submit it with sbatch.
#!/bin/bash #SBATCH -n 1 # Number of cores #SBATCH -t 0-00:05 # Runtime in D-HH:MM #SBATCH --job-name=singularity echo "Starting singularity on host $HOSTNAME" image=/path/to/workspace/ubuntu-latest.img module load singularity/2.3.1 singularity exec $image ls $HOME echo "Completed singularity on host $HOSTNAME"
The job’s output file should contain a listing of the contents of your home directory. At this point you will likely want to know how to install your application into a container or otherwise change it suit your needs. For that you need to read Singularity’s documentation, linked below. That will provide information on how to create and use your own container images or modify existing ones. Note that Kamiak does not support the use of sudo with Singularity. If you need sudo rights you can create a container that suits your needs using your own workstation or in a virtual machine running Linux where you have sudo/root access. After you create the container or Singularity image you can copy it to Kamiak and run it.