Workgroup Software Installs

Each workgroup receives its own dedicated storage space. If a workgroup uses software that IT does not maintain, then its likely that software is present within that storage space. Often times every individual workgroup member might have his or her own copy that he or she maintains – this may be necessary if frequent redimensioning and recompiling is necessary to the job workflow.

This document assumes that there is software necessary to a workgroup's productivity that do not require per-job recompilation. While the scheme presented here works best when individuals do not require their own copy of the software, the scheme can be extended to include that situation, as well.

After using the workgroup command (either explicitly or within your login files) the WORKDIR environment variable is set to the workgroup's work directory. Create a directory to contain all workgroup-specific software:

$ pwd
/lustre/work/it_nss
 
$ mkdir -p $WORKDIR/sw
$ chmod 02775 $WORKDIR/sw
 
$ ls -ld sw
drwxrwsr-x 4 frey it_nss 4096 Aug  1  2013 sw

The privileges assigned to the sw directory allow any member of the workgroup to create directories/files inside it.

To streamline VALET usage, a directory named valet can be created inside sw to contain any VALET package definition files representing the software you've installed. The VALET software will automatically check this directory if present:

$ mkdir -p $WORKDIR/sw/valet
$ chmod 02775 $WORKDIR/sw/valet

Again, the directory is made writeable by all workgroup members.

The IT-managed software organized under /opt/shared is a good example of how individual software packages will be installed inside $WORKDIR/sw. Each package gets its own subdirectory, named in lowercase. Individual versions of that software package are installed in subdirectories of that:

$WORKDIR/sw
 |- supercfd/
     |- attic/
     |- 1.0
     |- 1.2
     |- 2.1
 |- valet/

You may have seen an attic directory in some of the packages in /opt/shared – it is used to hold the original source/installer components for a software package.

Some workgroups may elect to have one individual maintain their software – call him or her the buildmeister. A workgroup may have several buildmeisters who each maintain some subset of the software. At the total opposite extreme, every group member acts as buildmeister for his/her own software. However your workgroup decides to divide that responsibility, it is best to leave package version directories and their contents ONLY writable by the buildmeister for that package (or version of a package).

Let's say that version 2.2 of "SuperCFD" has just been released and I've logged-in and downloaded the Unix/Linux source code to my desktop. I copy the source package to Mills using scp (or sftp) and the remote directory /lustre/work/it_nss/sw/supercfd/attic. Note that this is the value of $WORKDIR plus paths created in the previous section.

On Mills, I change to the SuperCFD software directory and prepare for the build of version 2.2:

$ cd $WORKDIR/sw/supercfd
$ mkdir 2.2
$ cd 2.2

With the source code uploaded to the attic directory, I can unpack inside the version 2.2 directory I just created:

$ pwd
/lustre/work/it_nss/sw/supercfd/2.2
 
$ tar --bzip -xf ../attic/supercfd-2.2.tar.bz2
 
$ ls
supercfd-2.2
 
$ mv supercfd-2.2 src

In this case, the source was a bzip-compressed tar file, so it was unpacked using the tar command. Other software might be packed as a gzip-compressed tar file or a ZIP file, so the actual command will vary.

The authors of SuperCFD organize their source code along the GNU autoconf guidelines. The source code directory was renamed from supercfd-2.2 to src; when running ./configure the install prefix will be set to $WORKDIR/sw/supercfd/2.2 which will then have e.g. bin, etc, share, lib directories accompanying the src directory. For any software package that uses autoconf or CMake, this is the organizational strategy IT has adopted.

The SuperCFD package requires Open MPI 1.6 and NetCDF:

$ vpkg_require openmpi/1.6.1-intel64
Adding dependency `intel/2011-8.273-64bit` to your environment
Adding package `openmpi/1.6.1-intel64` to your environment
 
$ vpkg_devrequire netcdf/4.1.3-intel64
Adding dependency `hdf5/1.8.8-intel64` to your environment
Adding package `netcdf/4.1.3-intel64` to your environment

The vpkg_devrequire command is used for NetCDF because it will set the CPPFLAGS and LDFLAGS environment variables to include the appropriate -I and -L arguments, respectively. For source code that is configured for build using GNU autoconf or CMake this is usually good enough.

Some software packages will require the buildmeister to pass arguments to the ./configure script or set environment variables. Keeping track of what parameters went into a build can be difficult. It's often a good idea to make note of the commands involved in a text file:

setup.sh

vpkg_require openmpi/1.6.1-intel64
vpkg_devrequire netcdf/4.1.3-intel64
 
export SUPERCFD_UNITS=METRIC
 
./configure --prefix=$WORKDIR/sw/supercfd/2.2 \
        --solver=auto

In the future the nature of the build is easily determined by looking at this file, versus examining build logs, etc. The process with CMake would be similar to what is outlined above.

After the software is built (e.g. using make and possibly a make check or make test) and installed (probably using make install) the software is ready for use:

$ cd ..
$ pwd
/lustre/work/it_nss/sw/supercfd/2.2
 
$ ls -l
total 16
drwxr-sr-x 2 user it_nss 4096 Sep 12  2012 bin
drwxr-sr-x 2 user it_nss  145 Dec 18  2012 etc
drwxr-sr-x 4 user it_nss 4096 Sep 12  2012 include
drwxr-sr-x 4 user it_nss 4096 Sep 12  2012 lib
drwxr-sr-x 5 user it_nss   48 Sep 12  2012 share
drwxr-sr-x 9 user it_nss 4096 Sep 12  2012 src