Each workgroup receives its own dedicated storage space. If a workgroup uses software that IT does not maintain, then its likely that software is present within that storage space. Often times every individual workgroup member might have his or her own copy that he or she maintains – this may be necessary if frequent redimensioning and recompiling is necessary to the job workflow.
This document assumes that there is software necessary to a workgroup's productivity that do not require per-job recompilation. While the scheme presented here works best when individuals do not require their own copy of the software, the scheme can be extended to include that situation, as well.
/lustre/work and on Farber it is /home/work so references to /lustre/work/it_nss should be /home/work/it_nss if using Farber. Thank you.
After using the workgroup command (either explicitly or within your login files) the WORKDIR environment variable is set to the workgroup's work directory. Create a directory to contain all workgroup-specific software:
$ pwd /lustre/work/it_nss $ mkdir -p $WORKDIR/sw $ chmod 02775 $WORKDIR/sw $ ls -ld sw drwxrwsr-x 4 frey it_nss 4096 Aug 1 2013 sw
The privileges assigned to the sw directory allow any member of the workgroup to create directories/files inside it.
To streamline VALET usage, a directory named valet can be created inside sw to contain any VALET package definition files representing the software you've installed. The VALET software will automatically check this directory if present:
$ mkdir -p $WORKDIR/sw/valet $ chmod 02775 $WORKDIR/sw/valet
Again, the directory is made writeable by all workgroup members.
The IT-managed software organized under /opt/shared is a good example of how individual software packages will be installed inside $WORKDIR/sw. Each package gets its own subdirectory, named in lowercase. Individual versions of that software package are installed in subdirectories of that:
$WORKDIR/sw
|- supercfd/
|- attic/
|- 1.0
|- 1.2
|- 2.1
|- valet/
You may have seen an attic directory in some of the packages in /opt/shared – it is used to hold the original source/installer components for a software package.
Some workgroups may elect to have one individual maintain their software – call him or her the buildmeister. A workgroup may have several buildmeisters who each maintain some subset of the software. At the total opposite extreme, every group member acts as buildmeister for his/her own software. However your workgroup decides to divide that responsibility, it is best to leave package version directories and their contents ONLY writable by the buildmeister for that package (or version of a package).
$WORKDIR/sw directory writable only by that user (but still owned and readable by the workgroup). Rather than doing chmod 02775 one would do chmod 02755 when creating the sw directory.
Let's say that version 2.2 of "SuperCFD" has just been released and I've logged-in and downloaded the Unix/Linux source code to my desktop. I copy the source package to Mills using scp (or sftp) and the remote directory /lustre/work/it_nss/sw/supercfd/attic. Note that this is the value of $WORKDIR plus paths created in the previous section.
On Mills, I change to the SuperCFD software directory and prepare for the build of version 2.2:
$ cd $WORKDIR/sw/supercfd $ mkdir 2.2 $ cd 2.2
With the source code uploaded to the attic directory, I can unpack inside the version 2.2 directory I just created:
$ pwd /lustre/work/it_nss/sw/supercfd/2.2 $ tar --bzip -xf ../attic/supercfd-2.2.tar.bz2 $ ls supercfd-2.2 $ mv supercfd-2.2 src
In this case, the source was a bzip-compressed tar file, so it was unpacked using the tar command. Other software might be packed as a gzip-compressed tar file or a ZIP file, so the actual command will vary.
The authors of SuperCFD organize their source code along the GNU autoconf guidelines. The source code directory was renamed from supercfd-2.2 to src; when running ./configure the install prefix will be set to $WORKDIR/sw/supercfd/2.2 which will then have e.g. bin, etc, share, lib directories accompanying the src directory. For any software package that uses autoconf or CMake, this is the organizational strategy IT has adopted.
The SuperCFD package requires Open MPI 1.6 and NetCDF:
$ vpkg_require openmpi/1.6.1-intel64 Adding dependency `intel/2011-8.273-64bit` to your environment Adding package `openmpi/1.6.1-intel64` to your environment $ vpkg_devrequire netcdf/4.1.3-intel64 Adding dependency `hdf5/1.8.8-intel64` to your environment Adding package `netcdf/4.1.3-intel64` to your environment
The vpkg_devrequire command is used for NetCDF because it will set the CPPFLAGS and LDFLAGS environment variables to include the appropriate -I and -L arguments, respectively. For source code that is configured for build using GNU autoconf or CMake this is usually good enough.
vpkg_devrequire as above and examine the value of CPPFLAGS or LDFLAGS e.g. using echo $CPPFLAGS.
Some software packages will require the buildmeister to pass arguments to the ./configure script or set environment variables. Keeping track of what parameters went into a build can be difficult. It's often a good idea to make note of the commands involved in a text file:
vpkg_require openmpi/1.6.1-intel64 vpkg_devrequire netcdf/4.1.3-intel64 export SUPERCFD_UNITS=METRIC ./configure --prefix=$WORKDIR/sw/supercfd/2.2 \ --solver=auto
In the future the nature of the build is easily determined by looking at this file, versus examining build logs, etc. The process with CMake would be similar to what is outlined above.
After the software is built (e.g. using make and possibly a make check or make test) and installed (probably using make install) the software is ready for use:
$ cd .. $ pwd /lustre/work/it_nss/sw/supercfd/2.2 $ ls -l total 16 drwxr-sr-x 2 user it_nss 4096 Sep 12 2012 bin drwxr-sr-x 2 user it_nss 145 Dec 18 2012 etc drwxr-sr-x 4 user it_nss 4096 Sep 12 2012 include drwxr-sr-x 4 user it_nss 4096 Sep 12 2012 lib drwxr-sr-x 5 user it_nss 48 Sep 12 2012 share drwxr-sr-x 9 user it_nss 4096 Sep 12 2012 src