MPI on Caviness
IT provides three job script (batch) template variants for mpi parallel environments located in /opt/shared/templates/slurm/generic/mpi
on Caviness:
- Open MPI: openmpi.qs
- Found in
/opt/shared/templates/slurm/generic/mpi/openmpi
.
- Mpich: mpich.qs
- Found in
/opt/shared/templates/slurm/generic/mpi/mpich
.
- Generic: mpi.qs
- Found in
/opt/shared/templates/slurm/generic/mpi/generic
.
In general, we suggest using the Open MPI job script template, however
- Each job script template is intended for a program using multiple processor cores/threads on one or more nodes. Each variant should be used with Open MPI or another MPI library that is tightly-integrated with Slurm.
- Sections of each script that can/should be edited are delimited by a
[EDIT]
tag. All Slurm job options are denoted by a line that starts with#SBATCH
followed by flags that would otherwise be passed on the command line. Slurm job options can easily be disabled in a script by inserting a space in the prefix, e.g.# SLURM
and reenabled by deleting that space.
You may copy and customize these templates to provide the best performance when running your MPI applications, however you should keep the job script complete (i.e. do not delete lines or sections in the job script template). See Running Applications on Caviness for details about resources.
It is a good idea to periodically check in
/opt/shared/templates/slurm/generic/mpi
for changes in existing templates, or the addition of new templates, designed to provide the best performance on Caviness.