====== Workgroup Software Installs on Caviness======

Each workgroup receives its own dedicated storage space.  If a workgroup uses software that IT does not maintain, then its likely that software is present within that storage space.  Often times every individual workgroup member might have his or her own copy that he or she maintains -- this may be necessary if frequent redimensioning and recompiling is necessary to the job workflow.

This document assumes that there is software necessary to a workgroup's productivity that do not require per-job recompilation.  While the scheme presented here works best when individuals do not require their own copy of the software, the scheme can be extended to include that situation, as well.

<note>For the sake of examples, the workgroup name used in this document will be //it_nss//.</note>

===== Directory Structure =====

After using the ''workgroup'' command (either explicitly or within your login files) the ''WORKDIR'' environment variable is set to the workgroup's work directory.  Create a directory to contain all workgroup-specific software:

<code bash>
$ mkdir -p $WORKDIR/sw
$ chmod 02775 $WORKDIR/sw

$ ls -ld sw
drwxrwsr-x 4 frey it_nss 4096 Aug  1  2013 sw
</code>

The privileges assigned to the //sw// directory allow any member of the workgroup to create directories/files inside it.

To streamline VALET usage, a directory named //valet// can be created inside //sw// to contain any VALET package definition files representing the software you've installed.  The VALET software will automatically check this directory if present:

<code bash>
$ mkdir -p $WORKDIR/sw/valet
$ chmod 02775 $WORKDIR/sw/valet
</code>

Again, the directory is made writeable by all workgroup members.

The IT-managed software organized under ''/opt/shared'' is a good example of how individual software packages will be installed inside ''$WORKDIR/sw''.  Each package gets its own subdirectory, named in lowercase.  Individual versions of that software package are installed in subdirectories of that:

<code>
$WORKDIR/sw
 |- supercfd/
     |- attic/
     |- 1.0
     |- 1.2
     |- 2.1
 |- valet/
</code>

You may have seen an ''attic'' directory in some of the packages in ''/opt/shared'' -- it is used to hold the original source/installer components for a software package.

==== "Buildmeister" ====

Some workgroups may elect to have one individual maintain their software -- call him or her the //buildmeister//.  A workgroup may have several buildmeisters who each maintain some subset of the software.  At the total opposite extreme, every group member acts as buildmeister for his/her own software.  However your workgroup decides to divide that responsibility, it is best to leave package version directories and their contents ONLY writable by the buildmeister for that package (or version of a package).

<note important>If a single buildmeister for all software is chosen, then it's best to leave everything within the ''$WORKDIR/sw'' directory writable only by that user (but still owned and readable by the workgroup).  Rather than doing ''chmod 02775'' one would do ''chmod 02755'' when creating the ''sw'' directory.</note>

===== Building from Source =====

Let's say that version 2.2 of "SuperCFD" has just been released and I've logged-in and downloaded the Unix/Linux source code to my desktop.  I copy the source package to Caviness using ''scp'' (or ''sftp'') and the remote directory ''/work/it_nss/sw/supercfd/attic''.  Note that this is the value of ''$WORKDIR'' plus paths created in the previous section.

On Caviness, I change to the SuperCFD software directory and prepare for the build of version 2.2:

<code bash>
$ cd $WORKDIR/sw/supercfd
$ mkdir 2.2
$ cd 2.2
</code>

With the source code uploaded to the ''attic'' directory, I can unpack inside the version 2.2 directory I just created:

<code bash>
$ pwd
/work/it_nss/sw/supercfd/2.2

$ tar --bzip -xf ../attic/supercfd-2.2.tar.bz2

$ ls
supercfd-2.2

$ mv supercfd-2.2 src
</code>

In this case, the source was a bzip-compressed tar file, so it was unpacked using the ''tar'' command.  Other software might be packed as a gzip-compressed tar file or a ZIP file, so the actual command will vary.

The authors of SuperCFD organize their source code along the GNU autoconf guidelines.  The source code directory was renamed from //supercfd-2.2// to //src//; when running ''./configure'' the install prefix will be set to ''$WORKDIR/sw/supercfd/2.2'' which will then have e.g. ''bin'', ''etc'', ''share'', ''lib'' directories accompanying the ''src'' directory.  For any software package that uses autoconf or CMake, this is the organizational strategy IT has adopted.

==== Building for Autoconf/CMake ====

The SuperCFD package requires Open MPI 1.6 and NetCDF:

<code bash>
$ vpkg_require openmpi/1.6.1-intel64
Adding dependency `intel/2011-8.273-64bit` to your environment
Adding package `openmpi/1.6.1-intel64` to your environment

$ vpkg_devrequire netcdf/4.1.3-intel64
Adding dependency `hdf5/1.8.8-intel64` to your environment
Adding package `netcdf/4.1.3-intel64` to your environment
</code>

The ''vpkg_devrequire'' command is used for NetCDF because it will set the ''CPPFLAGS'' and ''LDFLAGS'' environment variables to include the appropriate ''-I'' and ''-L'' arguments, respectively.  For source code that is configured for build using GNU autoconf or CMake this is usually good enough.

<note important>Some autoconf or CMake packages may require that the actual location of required libraries be specified.  In such cases, use ''vpkg_devrequire'' as above and examine the value of ''CPPFLAGS'' or ''LDFLAGS'' e.g. using ''echo $CPPFLAGS''.</note>

Some software packages will require the buildmeister to pass arguments to the ''./configure'' script or set environment variables.  Keeping track of what parameters went into a build can be difficult.  It's often a good idea to make note of the commands involved in a text file:

<file bash setup.sh>
vpkg_require openmpi/1.6.1-intel64
vpkg_devrequire netcdf/4.1.3-intel64

export SUPERCFD_UNITS=METRIC

./configure --prefix=$WORKDIR/sw/supercfd/2.2 \
        --solver=auto
</file>

In the future the nature of the build is easily determined by looking at this file, versus examining build logs, etc.  The process with CMake would be similar to what is outlined above.

After the software is built (e.g. using ''make'' and possibly a ''make check'' or ''make test'') and installed (probably using ''make install'') the software is ready for use:

<code bash>
$ cd ..
$ pwd
/work/it_nss/sw/supercfd/2.2

$ ls -l
total 16
drwxr-sr-x 2 user it_nss 4096 Sep 12  2012 bin
drwxr-sr-x 2 user it_nss  145 Dec 18  2012 etc
drwxr-sr-x 4 user it_nss 4096 Sep 12  2012 include
drwxr-sr-x 4 user it_nss 4096 Sep 12  2012 lib
drwxr-sr-x 5 user it_nss   48 Sep 12  2012 share
drwxr-sr-x 9 user it_nss 4096 Sep 12  2012 src
</code>