hpc documentation - technical:recipes:software-managment

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Software Management ======

This document offers an introduction to the mechanisms — organizational and procedural — used by IT RCI staff to build and manage the software made available to users on the HPC systems:

=== Organizational ===

It is most often the case that you do not use a single version of a piece of software.  Bug fixes and feature additions drive change in software over time, leading to //versioned releases// of the product.  In many cases the same versioned release may have mutually exclusive features that require multiple //variants// of that version to be maintained (simplest example: differing compiler toolchains, GCC versus Intel).

Organization is also important in maintaining //reproducibility// of your work.  An unstable, difficult to maintain computing platform is the product of:
  * Maintaining a full copy of your source code with each calculation:  minor to massive amounts of storage space can be wasted, and keeping multiple source trees in-sync is a challenge
  * Copying executables and libraries into each working directory:  difficult to remain consistent on release version, feature set, build parameters, etc. involved in producing the executable
  * Including extensive Unix environment manipulation in every job script:  any correction to the environment manipulation must be applied to every affected job script to remain consistent

=== Procedural ===

For many versions and variants of software to coexist on a system and be used properly, users cannot just modify their login files (e.g. ''.bashrc'') to alter environment variables:  such change affects //every shell// the user subsequently launches.  Environment management tools like VALET encapsulate the changes so they can be made consistently and easily.  Corrections can be made to the encapsulated change information to affect globally (for that version or variant of the software).

VALET also makes it easier to build software.  Standard development paths — like the ''<prefix>/lib'' and ''<prefix>/include'' directories — are automatically added to key environment variables when the //development context// is used:
<code bash>
[user@login00.darwin ~]$ vpkg_devrequire udunits/2.2.28
Adding package `udunits/2.2.28` to your environment
[user@login00.darwin ~]$ echo $UDUNITS_PREFIX
/opt/shared/udunits/2.2.28
[user@login00.darwin ~]$ echo $LDFLAGS
-L/opt/shared/udunits/2.2.28/lib
[user@login00.darwin ~]$ echo $CPPFLAGS
-I/opt/shared/udunits/2.2.28/include
</code>
The GNU Autoconf build system makes use of these environment variables when searching for and building software, and other tools can make use of the value of the variable (e.g. ''cmake -DUDUNITS_ROOT=$UDUNITS_PREFIX'').

VALET is not just a tool that system administrators can use to describe environment changes:  all users can create their own //package definitions// for themselves (in their ''~/.valet'' directory) or for their workgroup (in the ''$WORKDIR/sw/valet'' directory).  A package definition can reference other packages as //dependencies// and have those dependencies automatically loaded into the environment.

===== Preparations =====

In this example we will build the latest version of the [[https://tddft.org/programs/libxc/|libxc]] library using various compilers.

First and foremost, decide where on the file system you are going to organize the set of versions/variants of the software.  For a user maintaining personal builds of the software the home directory can be used:
<code bash>
[user@login00.darwin ~]$ LIBXC_BASEDIR=~/sw/libxc
</code>
For shared installations made available to your entire workgroup:

On DARWIN:
<code bash>
[(workgroup:user)@login00.darwin ~]$ LIBXC_BASEDIR="${WORKDIR_SW}/libxc"
</code>

On Caviness:
<code bash>
[(workgroup:user)@login00 ~]$ LIBXC_BASEDIR="${WORKDIR}/sw/libxc"
</code>

Ensure the base directory exists:
<code bash>
[user@login00.darwin ~]$ mkdir -p "$LIBXC_BASEDIR"
</code>

Often the software being built is distributed as one or more downloadable files (e.g. tar.gz archive).  IT RCI staff create a directory to hold (organize!) these files.  For **libxc** we will create that directory and download the source archive:

<code bash>
[user@login00.darwin ~]$ mkdir -p "$LIBXC_BASEDIR/attic"
[user@login00.darwin ~]$ wget -O "$LIBXC_BASEDIR/attic/libxc-5.1.0.tar-gz" \
       "http://www.tddft.org/programs/libxc/down.php?file=5.1.0/libxc-5.1.0.tar.gz"
</code>

At this point the source code has been downloaded, our directory hierarchy has been established, and we are ready to build a variant of the 5.1.0 release.

===== Builds =====

Each of the compilers chosen in building //variants// of the 5.1.0 release will follow the same general procedure.

<note>The notation '':'' (colon) in output examples throughout this document indicates lines of output are displayed by the command entered but have been omitted from this documentation.</note>

==== System GCC ====

The cluster comes with a native GNU toolchain present (gcc, g++, gfortran).  No environment changes are necessary to use these compilers.  Since it is the default compiler toolchain, IT RCI staff do not mention it when naming a software variant:  this variant will have a //version identifier// of ''5.1.0''.

Prepare the variant's installation directory and unpack the source code inside it:
<code bash>
[user@login00.darwin ~]$ mkdir "$LIBXC_BASEDIR/5.1.0"
[user@login00.darwin ~]$ cd "$LIBXC_BASEDIR/5.1.0"
[user@login00.darwin 5.1.0]$ tar -xf "$LIBXC_BASEDIR/attic/libxc-5.1.0.tar-gz"
[user@login00.darwin 5.1.0]$ mv libxc-5.1.0 src
[user@login00.darwin 5.1.0]$ cd src
</code>

The **libxc** build system uses GNU Autoconf or CMake.  For this variant, we will use Autoconf:
<code>
[user@login00.darwin src]$ ./configure --prefix="$LIBXC_BASEDIR/5.1.0"
</code>
Additional flags may be necessary, but the most important is the ''--prefix'', telling the build system the software is intended to be installed into the chosen directory for the variant.
<code bash>
[user@login00.darwin src]$ make
   :
xc-threshold.c: In function ‘check_xc’:
xc-threshold.c:808:3: error: ‘for’ loop initial declarations are only allowed in C99 mode
   for (int i = 0; i < (int) (sizeof(xc_values_type) / sizeof(double)); i++)
   ^
xc-threshold.c:808:3: note: use option -std=c99 or -std=gnu99 to compile your code
</code>
This build encountered an error:  the system GNU C compiler defaults to an older language standard than what the source code implies, so the compiler has mentioned that a flag is necessary to choose that standard:
<code bash>
[user@login00.darwin src]$ CFLAGS="-std=gnu99" ./configure --prefix="$LIBXC_BASEDIR/5.1.0"
   :
[user@login00.darwin src]$ make
   :
[user@login00.darwin src]$ make install
   :
</code>

==== Intel ====

The Intel compilers offer advanced optimizations to target the specific processor models' features.  To use the Intel compilers they must be first added to the environment:
<code bash>
[user@login00.darwin ~]$ cd
[user@login00.darwin ~]$ vpkg_require intel/2020u4
Adding package `intel/2020u4` to your environment
[user@login00.darwin ~]$ which icc
/opt/shared/intel/2020u4/compilers_and_libraries_2020.4.304/linux/bin/intel64/icc
</code>
We will also use CMake for this build; though the OS does provide a version of CMake, it is relatively old so a newer version is often necessary:
<code bash>
[user@login00.darwin ~]$ vpkg_require cmake/3.19.1
Adding package `cmake/3.19.1` to your environment
</code>

Since this variant of 5.1.0 uses the Intel compilers, it will have a version identifier that includes a //feature// named ''intel-2020''.  The appropriate directory name for that identifier can be determined using:
<code bash>
[user@login00.darwin ~]$ vpkg_id2path --version="5.1.0:intel-2020"
5.1.0-intel-2020
</code>
Leading to the unpack procedure:
<code bash>
[user@login00.darwin ~]$ mkdir "$LIBXC_BASEDIR/5.1.0-intel-2020"
[user@login00.darwin ~]$ cd "$LIBXC_BASEDIR/5.1.0-intel-2020"
[user@login00.darwin 5.1.0-intel-2020]$ tar -xf "$LIBXC_BASEDIR/attic/libxc-5.1.0.tar-gz"
[user@login00.darwin 5.1.0-intel-2020]$ mv libxc-5.1.0 src
[user@login00.darwin 5.1.0-intel-2020]$ cd src
</code>

CMake builds usually request that you create an empty directory to hold all of the files generated by CMake.  The commands analogous to the Autoconf setup of the build look like:
<code bash>
[user@login00.darwin 5.1.0-intel-2020]$ mkdir build
[user@login00.darwin 5.1.0-intel-2020]$ cd build
[user@login00.darwin build]$ CC=icc FC=ifort CXX=icpc cmake \
       -DCMAKE_INSTALL_PREFIX="$LIBXC_BASEDIR/5.1.0-intel-2020" \
       -DBUILD_TESTING=FALSE \
       ..
   :
</code>
The CMake build system doesn't yet support Fortran compilation 100%, so by default that language is omitted (and the ''FC=ifort'' is probably extraneous).  By the same token, none of the source is written in C++ so specifying ''CXX=icpc'' is also extraneous, but in neither case does it hurt to have your intentions clear.
<code bash>
[user@login00.darwin build]$ make
   :
[user@login00.darwin build]$ make install
   :
</code>

==== GCC 10 ====

Finally, a variant using the GCC 10 compiler will be built using Autoconf.  First, remove all environment changes made for the Intel build
<code bash>
[user@login00.darwin build]$ cd
[user@login00.darwin ~]$ vpkg_rollback all
</code>
and configure the environment for GCC 10 and the newer CMake:
<code bash>
[user@login00.darwin ~]$ vpkg_versions gcc

Available versions in package (* = default version):

[/opt/shared/valet/2.1/etc/gcc.vpkg_yaml]
gcc       GCC Compiler Suite
  4.8     alias to gcc/4.8.5
  4.8.5   CentOS system GCC with C, C++, Obj-C, Obj-C++, and Fortran
  7.3     alias to gcc/7.3.0
  7.3.0   GCC with C, C++, Obj-C, Obj-C++, Fortran, and GO
  10.1.0  GCC with C, C++, Obj-C, Obj-C++, Fortran, and GO
  10.1    alias to gcc/10.1.0
* system  alias to gcc/4.8.5
[user@login00.darwin ~]$ vpkg_require cmake/3.19.1 gcc/10.1.0
Adding package `cmake/3.19.1` to your environment
Adding package `gcc/10.1.0` to your environment
</code>
<WRAP center round tip 60%>
Notice that multiple packages can be specified in a single ''vpkg_require'' command — this is more efficient than running them as multiple ''vpkg_require'' commands.
</WRAP>
As with the Intel example, we will add a ''gcc-10.1'' feature to the version id to indicate the compiler choice, so the preparations look like:
<code bash>
[user@login00.darwin ~]$ vpkg_id2path --version="5.1.0:gcc-10.1"
5.1.0-gcc-10.1
[user@login00.darwin ~]$ mkdir "$LIBXC_BASEDIR/5.1.0-gcc-10.1"
[user@login00.darwin ~]$ cd "$LIBXC_BASEDIR/5.1.0-gcc-10.1"
[user@login00.darwin 5.1.0-gcc-10.1]$ tar -xf "$LIBXC_BASEDIR/attic/libxc-5.1.0.tar-gz"
[user@login00.darwin 5.1.0-gcc-10.1]$ mv libxc-5.1.0 src
[user@login00.darwin 5.1.0-gcc-10.1]$ cd src
</code>
Sometimes Autoconf allows the same approach as CMake:  creating a build directory to hold the files produced by the build system, leaving the source directory undisturbed.
<code bash>
[user@login00.darwin src]$ mkdir build
[user@login00.darwin src]$ cd build
[user@login00.darwin build]$ CC=gcc FC=gfortran ../configure --prefix="$LIBXC_BASEDIR/5.1.0-gcc-10.1"
   :
[user@login00.darwin build]$ make
   :
[user@login00.darwin build]$ make install
   :
</code>
<WRAP center round info 60%>
Note that with GCC 10 the additional ''CFLAGS=-std=gnu99'' was not present on the ''../configure'' command.  With the 10.1 gcc compiler, the default language standard is listed as **gnu18**.  The default for GCC 4.8.5 is **gnu89** — older than the C99 standard the code requires.
</WRAP>

==== Results ====

Through the course of the above sections three distinct variants of **libxc** 5.1.0 were produced.  The base directory currently looks like:
<code bash>
[user@login00.darwin build]$ cd
[user@login00.darwin ~]$ vpkg_rollback all
[user@login00.darwin ~]$ ls -l "$LIBXC_BASEDIR"
total 21
drwxr-xr-x 6 user everyone 6 Feb  8 11:35 5.1.0
drwxr-xr-x 6 user everyone 6 Feb  8 12:11 5.1.0-gcc-10.1
drwxr-xr-x 7 user everyone 7 Feb  8 11:56 5.1.0-intel-2020
drwxr-xr-x 2 user everyone 3 Feb  8 11:10 attic
</code>
Each variant is structured similarly, with the typical installation directory layout:
<code bash>
[user@login00.darwin ~]$ ls -l "$LIBXC_BASEDIR/5.1.0-intel-2020"
total 40
drwxr-xr-x  2 user everyone  3 Feb  8 11:56 bin
drwxr-xr-x  2 user everyone  7 Feb  8 11:56 include
drwxr-xr-x  3 user everyone  4 Feb  8 11:56 lib64
drwxr-xr-x  3 user everyone  3 Feb  8 11:56 share
drwxr-xr-x 11 user everyone 40 Feb  8 11:50 src
</code>

To make use of one of these variants of **libxc**, these directories must be added to specific environment variables: a task for which VALET is designed to assist.

===== VALET Setup =====

Since **libxc** adheres to the standard directory layout for software on Linux (''bin'', ''lib'' or ''lib64'', ''include'') it is relatively easy to setup the runtime environment:
  * ''$LIBXC_BASEDIR/5.1.0-intel-2020/bin'' must be added to ''$PATH''
  * ''$LIBXC_BASEDIR/5.1.0-intel-2020/lib64'' must be added to ''$LD_LIBRARY_PATH''
In addition, for software-building the linker could be told to check ''$LIBXC_BASEDIR/5.1.0-intel-2020/lib64'' for required libraries and the C/C++ compiler told to look in ''$LIBXC_BASEDIR/5.1.0-intel-2020/include'' for header files — more on that in a moment.

VALET automatically recognizes the standard directory layout, so configuring these variants of ''libxc'' is very straightforward.  First, note where the variants were collocated on the file system:
<code bash>
[user@login00.darwin ~]$ echo $LIBXC_BASEDIR 
/home/user/sw/libxc
</code>
Since these builds were done in the user's home directory, they were personal copies of the software and should use a //VALET package definition file// stored in ''~/.valet''
<code bash>
[user@login00.darwin ~]$ VALET_PKG_DIR=~/.valet ; VALET_PKG_DIR_MODE=0700
</code>
versus an installation made for an entire workgroup would store VALET package definition files in
<code bash>
[user@login00.darwin ~]$ VALET_PKG_DIR="$WORKDIR_SW/valet"
</code>
on DARWIN and in
<code bash>
[user@login00.darwin ~]$ VALET_PKG_DIR="$WORKDIR/sw/valet" ; VALET_PKG_DIR_MODE=2770
</code>
on Caviness.

Whichever scheme is in-use, ensure the directory exists for personal use on Caviness and DARWIN, and entire workgroup on Caviness (entire workgroup on DARWIN is automatically created for each allocation so this is not necessary)
<code bash>
[user@login00.darwin ~]$ mkdir -p --mode=$VALET_PKG_DIR_MODE "$VALET_PKG_DIR"
</code>

VALET allows package definitions in a variety of formats (XML, JSON, YAML) but YAML tends to be the simplest format so we will use it here.

==== Package section ====

The //package section// of the definition file includes items that apply to all versions/variants of the software:
<code yaml>
libxc:
    prefix: /home/user/sw/libxc
    description: a library of exchange-correlation functionals for density-functional theory
    url: "https://tddft.org/programs/libxc/"
</code>
The //package identifier// is the top-level key in the document — ''libxc'' — and the value of ''$LIBXC_BASEDIR'' is the value of the ''prefix'' key in this section.  The URL and description are information taken from the official ''libxc'' web site.

==== Versions ====

The ''versions'' key is used to provide a list of the versions/variants of the software.  The simplest version that was built used the system GCC compiler:
<code yaml>
libxc:
    prefix: /home/user/sw/libxc
    description: a library of exchange-correlation functionals for density-functional theory
    url: "https://tddft.org/programs/libxc/"
    
    versions:
        "5.1.0":
            description: compiled with system gcc/gfortran (4.8.5)
</code>

<WRAP center round tip 60%>
Since we used ''vpkg_id2path'' to determine the path associated with the version identifiers and each variant is installed under ''$LIBXC_BASEDIR'', there's no need to specify a prefix for the version definitions: package's prefix (''/home/user/sw/libxc'') with the version identifier appended (''/home/user/sw/libxc/5.1.0'') is implicit.

The implicit behavior is overridden by providing a ''prefix'' key in the version definition:  a relative path is appended to the package's prefix, an absolute path is used as-is.
</WRAP>

For the other two variants, a dependency exists with respect to the compiler toolchain that was used.  Each of those variants should be defined with that dependency described:
<code yaml>
libxc:
    prefix: /home/user/sw/libxc
    description: a library of exchange-correlation functionals for density-functional theory
    url: "https://tddft.org/programs/libxc/"
    
    versions:
        "5.1.0":
            description: compiled with system gcc/gfortran (4.8.5)
        "5.1.0:intel-2020":
            description: compiled with Intel icc (2020)
            dependencies:
                - intel/2020
        "5.1.0:gcc-10.1":
            description: compiled with gcc (10.1)
            dependencies:
                - gcc/10.1
</code>

<WRAP center round tip 60%>
We could have been more specific about the version of the Intel and GCC compiler that was used, e.g. ''intel/2020u4'' and ''gcc/10.1.0''.  Minor releases of software do not usually significantly change its behavior or alter APIs, so by being less specific (''intel/2020'') if the Intel compiler were upgraded to ''2020u5'' (and the version identifier ''2020'' promoted to point to that by the system administrator) then our ''libxc/5.1.0:intel-2020'' would automatically have the new version of Intel as its dependency.  Note that this automatic "upgrade" does not work for static executables and libraries (they would need to be rebuilt to use the new runtime libraries, for example).
</WRAP>

Finally, it is a good idea to specify which version definition should act as the default.  This yields the following package definition file
<file libxc.vpkg_yaml>
libxc:
    prefix: /home/user/sw/libxc
    description: a library of exchange-correlation functionals for density-functional theory
    url: "https://tddft.org/programs/libxc/"
    default-version: "5.1.0:gcc-10.1"
    versions:
        "5.1.0":
            description: compiled with system gcc/gfortran (4.8.5)
        "5.1.0:intel-2020":
            description: compiled with Intel icc (2020)
            dependencies:
                - intel/2020
        "5.1.0:gcc-10.1":
            description: compiled with gcc (10.1)
            dependencies:
                - gcc/10.1
</file>
saved at ''$VALET_PKG_DIR/libxc.vpkg_yaml''.

==== Checking the definition file ====

The package definition file can be syntax-checked:
<code bash>
[user@login00.darwin ~]$ vpkg_check "$VALET_PKG_DIR/libxc.vpkg_yaml"
/home/user/.valet/libxc.vpkg_yaml is OK

[libxc] {
  contexts: all
  actions: {
    LIBXC_PREFIX=${VALET_PATH_PREFIX} (contexts: development)
  }
  https://tddft.org/programs/libxc/
  a library of exchange-correlation functionals for density-functional theory
  prefix: /home/user/sw/libxc
  source file: /home/user/.valet/libxc.vpkg_yaml
  default version: libxc/5.1.0:intel-2020
  versions: {
    [libxc/5.1.0] {
      contexts: all
      compiled with system gcc/gfortran (4.8.5)
      prefix: /home/user/sw/libxc/5.1.0
      standard paths: {
        bin: /home/user/sw/libxc/5.1.0/bin
        lib: /home/user/sw/libxc/5.1.0/lib
        include: /home/user/sw/libxc/5.1.0/include
        pkgConfig: /home/user/sw/libxc/5.1.0/lib/pkgconfig
      }
    }
    [libxc/5.1.0:gcc-10.1] {
      contexts: all
      dependencies: {
        gcc/10.1
      }
      compiled with gcc (10.1)
      prefix: /home/user/sw/libxc/5.1.0-gcc-10.1
      standard paths: {
        bin: /home/user/sw/libxc/5.1.0-gcc-10.1/bin
        lib: /home/user/sw/libxc/5.1.0-gcc-10.1/lib
        include: /home/user/sw/libxc/5.1.0-gcc-10.1/include
        pkgConfig: /home/user/sw/libxc/5.1.0-gcc-10.1/lib/pkgconfig
      }
    }
    [libxc/5.1.0:intel-2020] {
      contexts: all
      dependencies: {
        intel/2020
      }
      compiled with Intel icc (2020)
      prefix: /home/user/sw/libxc/5.1.0-intel-2020
      standard paths: {
        bin: /home/user/sw/libxc/5.1.0-intel-2020/bin
        lib: /home/user/sw/libxc/5.1.0-intel-2020/lib64
        include: /home/user/sw/libxc/5.1.0-intel-2020/include
      }
    }
  }
}
</code>
The file had no errors in its YAML syntax.  Notice also that the standard paths (''bin'', ''lib64'', ''include'') are found and noted by VALET!

==== Runtime environment ====

To load a specific variant of **libxc** 5.1.0 into the runtime environment, the ''vpkg_require'' command is used:
<code bash>
[user@login00.darwin ~]$ vpkg_require libxc/5.1.0:intel-2020
Adding dependency `intel/2020u4` to your environment
Adding package `libxc/5.1.0:intel-2020` to your environment
[user@login00.darwin ~]$ which xc-info
~/sw/libxc/5.1.0-intel-2020/bin/xc-info
</code>
The ''xc-info'' command is used //without a leading path// which implies that the shell with check directories in the ''$PATH'' environment variable for an executable with that name.  If a different version/variant of **libxc** is chosen:
<code bash>
[frey@login00.darwin ~]$ vpkg_rollback all
[frey@login00.darwin ~]$ vpkg_require libxc/5.1.0
Adding package `libxc/5.1.0` to your environment
[frey@login00.darwin ~]$ which xc-info
~/sw/libxc/5.1.0/bin/xc-info
</code>
The command is still ''xc-info'' but the shell finds it at a different location.  This abstraction (no full paths to executables) makes it easier to alter complex job scripts by simply changing which variant is added using ''vpkg_require''.

Note that the ''$LD_LIBRARY_PATH'' is augmented, as well:
<code bash>
[user@login00.darwin ~]$ echo $LD_LIBRARY_PATH
/home/user/sw/libxc/5.1.0/lib:/opt/shared/slurm/lib
</code>
versus
<code bash>
[user@login00.darwin ~]$ vpkg_rollback all
[user@login00.darwin ~]$ vpkg_require libxc/5.1.0:intel-2020
Adding dependency `intel/2020u4` to your environment
Adding package `libxc/5.1.0:intel-2020` to your environment
[user@login00.darwin ~]$ echo $LD_LIBRARY_PATH
/home/user/sw/libxc/5.1.0-intel-2020/lib64:/opt/shared/intel/2020u4/compilers_and_libraries_2020.4.304[....]
</code>

==== Development context ====

All actions taken by VALET occur in a //context//.  The //context// is an arbitrary string, like ''development'', that limits what actions are taken in the runtime environment.  By default, an action happens regardless of the context.  But there are several actions VALET produces by default that apply to the aforementioned ''development'' context:
  * An additional environment variable is set to the prefix directory for the version/variant
  * The ''<prefix>/include'' directory is appended to the ''$CPPFLAGS'' environment variable (e.g. ''-I<prefix>/include'')
  * The ''<prefix>/lib'' and/or ''<prefix>/lib64'' directories are appended to the ''$LDFLAGS'' environment variable (e.g. ''-L<prefix>/lib64'')
For the **libxc** package:
<code bash>
[user@login00.darwin ~]$ vpkg_rollback all
[user@login00.darwin ~]$ vpkg_require --context=development libxc/5.1.0
Adding package `libxc/5.1.0` to your environment
[user@login00.darwin ~]$ echo $LIBXC_PREFIX
/home/user/sw/libxc/5.1.0
[user@login00.darwin ~]$ echo $CPPFLAGS
-I/home/user/sw/libxc/5.1.0/include
[user@login00.darwin ~]$ echo $LDFLAGS
-L/home/user/sw/libxc/5.1.0/lib
</code>
As mentioned at the start of this document, the ''$LIBXC_PREFIX'' can be particularly helpful when configuring an Autoconf or CMake build of software that depends on this version/variant of **libxc**.  The development context is the most common use of context in VALET that it can be shortened from ''vpkg_require --context=development'' to just ''vpkg_devrequire''.

===== Recipes for Specific Software =====

All recipes are provided based on a specific cluster, Caviness and/or DARWIN.  However each recipe may be used as a reference from one cluster to another by using the appropriate file system (directory structure) and VALET packages for the cluster you are trying to install the software.

==== MCFOST ====
[[technical:recipes:mcfost|Building MCFOST on Caviness]]


==== Python ====

  * [[technical:recipes:pyqt5-in-virtualenv|Building PyQt5 in a Python Virtual Environment]]
  * [[technical:recipes:jupyter-notebook|Jupyter Notebook Python Virtual Environment]]
  * [[technical:recipes:keras-in-virtualenv|Keras Python Virtual Environment]]
  * Gurobi Python Virtual Environments
    * [[software:gurobi:caviness#serial-or-threaded-python-environment|Gurobi Python Threaded and Serial]]
    * [[software:gurobi:caviness#mpi-python-environment|Gurobi Python MPI]]
  * [[technical:recipes:emcee-in-virtualenv|Python Virtualenv: emcee and pyKLIP]]
  * [[technical:recipes:mpi4py-in-virtualenv|Python Virtual Environments with mpi4py]]
  * [[technical:recipes:tensorflow-in-virtualenv|TensorFlow Python Virtual Environment]]

==== WRF ====

  * [[software:wrf:caviness|WRF on Caviness]]
  * [[software:wrf:darwin|WRF on DARWIN]]

==== VASP ====
[[technical:recipes:vasp-6-darwin|Building VASP 6 on Caviness/DARWIN]]