This is an old revision of the document!
Singularity on Caviness
Build a Singularity container
There are several ways to get a Singularity container onto Caviness.
- You can build a Singularity container from a Docker container.
- You can copy a container from a container registry to Caviness.
- You can build a container on your local system and copy it to Caviness
Build Singularity container from a Docker container
Caviness does not support Docker, but you can build a Singularity container from an existing Docker container. Starting on the head node, load the Singularity software into your environment using VALET, and build your container.
$ vpkg_require singularity $ singularity build tensorflow.simg docker://tensorflow/tensorflow
Pull an image from Docker or Singularity Hub
Another option is to copy an existing Singularity container from a registry. Use the singularity pull
command on Caviness to copy a pre-existing container from a container registry. If you use the singularity pull
command to copy a Docker container, it will be converted to Singularity during the pull process.
To copy a container from a registry:
- Log in to Caviness
- Load singularity via VALET
vpkg_require singularity
- Copy the container you want with the
singularity pull
command.
More details on this can be found in the Singularity User Guide.
One example of a container registry is https://biocontainers.pro/registry/#/.
Build a container on your local system
You can build your Singularity container on your local system, installing the singularity program if you need to. For more information on how to do this, see the Singularity web site. Copy your container to Caviness using the usual file transfer methods. See the Transferring files to/from Caviness for more information.
Execute your Singularity container through SLURM
Once your container is on Caviness you can run it. Containers must run on Caviness' compute nodes, not on the head node. Remember you must specify a workgroup before running any jobs on Caviness, then use either the salloc
or sbatch
command to get access to a compute node. See the Running Jobs on Caviness for more information on SLURM and using Caviness' compute nodes.
Inside your interactive session or your batch job you must first issue the command
vpkg_require singularity
Then you can use the singularity commands to execute your container, like the example below.
[(it_css:traine)@login01 ~]$ salloc --ntasks=1 --cpus-per-task=4 salloc: Granted job allocation 844 salloc: Waiting for resource configuration salloc: Nodes r00n45 are ready for job [traine@r00n45 ~]$ vpkg_require singularity Adding package `singularity/2.5.1` to your environment [traine@r00n45 ~]$ singularity shell tensorflow.simg Singularity: Invoking an interactive shell within container... Singularity tensorflow.simg:~> tensorboard --help USAGE: /usr/local/bin/tensorboard [flags] Try --helpfull to get a list of all flags. Singularity tensorflow.simg:~> grep Cpus_allowed_list /proc/$$/status Cpus_allowed_list: 0-3 Singularity tensorflow.simg:~>
Common Singularity commands
Some common commands are listed here. For more information about Singularity, see the Singularity web site.
shell | Start a shell within your container using the operating system you have set up your container to use. |
exec | Run a single command within your container. |
run | Run a recipe script you have set up within your container. Using a recipe script forces users of your container to use a pre-established workflow. |
help | Provides help on Singularity. |