XML Package Definition Files

In version 2 of VALET, packages (and their versions) can still be defined in XML files (with an augmented grammar for new features). VALET 2 can use .vpkg files from version 1 for backward compatibility, but many version 2 features are only available using the newer XML grammar.

VALET package definitions in the XML format use the file extension .vpkg or .vpkg_xml.

A VALET package is represented in XML as a document with the following overall structure:

<?xml version="1.0" encoding="UTF-8"?>
<package
  xmlns="http://www.udel.edu/xml/valet/1.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.udel.edu/xml/valet/1.0
                      http://www.udel.edu/xml/valet/1.0/schema.xsd"
  id="packageid">
    :
  <version id="versionid">
    :
  </version>
</package>

A set of scalar properties can be used both in the package and version elements of the configuration. Scalar properties are:

Typically, the package-global prefix is set to a directory containing each version of the package, and each versioned package defines a prefix relative to that directory:

$ ls -l /opt/shared/openmpi
total 14
drwxr-sr-x 8 user   sysadmin 8 Aug 27 16:10 1.8.2
drwxr-sr-x 8 user   sysadmin 8 Aug 27 16:10 1.8.2-gcc-4.8.3
drwxr-sr-x 8 user   sysadmin 8 Aug 28 09:26 1.8.2-intel64
drwxrwsr-x 2 user   sysadmin 3 Aug 27 17:10 attic

The package's prefix = /opt/shared/openmpi and versions would use e.g. prefix = 1.8.2-intel64. If no prefix is provided for a versioned package, then VALET implicitly uses the version id.

Versioned packages that do not specify the development-env, no-development-env, standard-paths, or no-standard-paths properties inherit the value of their parent package.

Example:

<?xml version="1.0" encoding="UTF-8"?>
<package
  xmlns="http://www.udel.edu/xml/valet/1.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.udel.edu/xml/valet/1.0
                      http://www.udel.edu/xml/valet/1.0/schema.xsd"
  id="gaussian">
  <description>Gaussian Quantum Chemistry Suite</description>
  <url>http://gaussian.com/</url>
  <prefix>/opt/shared/gaussian</prefix>
 
  <no-development-env/>
  <no-standard-paths/>
 
  <version id="g09">
    <prefix>g09d01</prefix>
  </version>
  <version id="g09d01">
  </version>
</package>

In this example, version g09d01 has a prefix path of /opt/shared/gaussian/g09d01 since it did not specify a prefix. Version g09 provided a relative path that is appended to the prefix of its parent package to also yield /opt/shared/gaussian/g09d01. Both versions inherit the development-env and standard-paths behavior of the parent package.

The full versioned package identifiers available in the gaussian package defined above would be:

• gaussian/g09
• gaussian/g09d01

Since the g09 version targets the same executables as g09d01, the g09 version can be considered to be an alias of g09d01. VALET allows for aliased versions of packages:

  <version id="g09" alias-to="g09d01"/>
  <version id="g09d01">
  </version>

Once the overall versioning structure of a package has been created, the environment tests and changes associated with the package and versions of the package can be added. The three kinds of tests/changes are:

If present in the package-global area of the configuration, the tests/changes affect all versions of the package and are satisfied/applied before version-specific dependencies/incompatibilities and actions.

Example:

<?xml version="1.0" encoding="UTF-8"?>
<package
  xmlns="http://www.udel.edu/xml/valet/1.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.udel.edu/xml/valet/1.0
                      http://www.udel.edu/xml/valet/1.0/schema.xsd"
  id="gaussian">
  <description>Gaussian Quantum Chemistry Suite</description>
  <url>http://gaussian.com/</url>
  <prefix>/opt/shared/gaussian</prefix>
 
  <no-development-env/>
  <no-standard-paths/>
 
  <dependencies>
    <predicate stage="pre-condition">
      <variable>GAUSS_SCRDIR</variable>
      <operator>not-is-set</operator>
      <message>If GAUSS_SCRDIR is not set, the working directory will be used; you do not want that.</message>
    </predicate>
  </dependencies>
 
  <actions>
    <script shell="all" action="exec" order="success-first" succeed="0">mk-gaussian-scrdir</script>
  </actions>
 
  <version id="g09" alias-to="g09d01"/>
  <version id="g09d01">
    <dependencies>
      <package id="pgi/14"/>
    </dependencies>
    <actions>
      <export variable="GAUSSIAN_VERSION" action="set">G09</export>
      <script action="source" shell="sh">g09.sh</script>
    </actions>
  </version>
</package>

Given the package definition above, typing vpkg_require gaussian/g09 yields the following course of events:

1. g09 is an alias to g09d01, so version g09d01 becomes the target versioned package being configured
2. Has the pgi/14 package been loaded into the environment yet?
   • If not, do all of its tests and attempt to affect all of its environment changes before proceeding
     • If that fails, this configuration fails and must exit immediately
3. The installation prefix for the software is constructed as /opt/shared/gaussian/g09d01
4. The GAUSS_SCRDIR environment variable is examined
   • If it does not exist in the environment the configuration fails and the message text is printed to stdout
5. The mk-gaussian-scrdir program found in VALET's libexec directory is executed
   • If the result code returned by mk-gaussian-scrdir is zero, continue
   • Otherwise, the configuration fails and exits
6. The GAUSSIAN_VERSION environment variable is assigned the value G09
7. For Bash and other sh-like shells, the script g09.sh in VALET's libexec directory is sourced
   • The return code is not checked for success or failure

If control reaches the end of this pseudo-code, then the gaussian/g09 package has been successfully configured in the environment.

An XML actions element can be specified either in a package or in a version of a package and contains zero or more action elements of the following kinds:

• variable – make changes to environment variables
• special directories – shorthand for alterations to PATH, LD_LIBRARY_PATH, etc.
• script – source/execute a script
• shell alias – set/unset an alias

Each action is represented as an XML element. Actions attached to the package itself will be performed when any version of the package is selected for addition to the environment, and in addition to actions associated with the version itself.

Actions will be performed in the order they are specified in the block.

Example:

  <actions>
    <incdir>/opt/include</incdir>
    <export variable="PATH" action="scrub-path">/opt/bin</export>
    <bindir>/opt/bin</bindir>
    <libdir>/opt/lib</libdir>
    <libdir>/opt/lib64</libdir>
    <mandir>/opt/share/man</mandir>
    <export variable="USER_SW_PATH">/opt/bin</export>
    <export variable="CFLAGS" action="append-space" development-env="false">-O3 -g</export>
  </actions>

Variable actions alter the value of an environment variable. A variable action is an XML export element with the following attributes:

The text inside the export element is the value to add/remove/etc.

The actions are self-explanatory for the most part. The scrub action removes all occurrences of value from the variable's current value in the environment. The scrub-path action treats the value of variable as a search path and removes from that search path all occurrences (by exact match) of value.

The value may contain references to other environment variables using the syntax ${ENV VAR NAME}.

Example:

<export variable="PATH" action="scrub-path">/opt/bin</export>

All special-directory actions are enclosed in an XML element within the actions element. These actions are shortcuts that produce variable actions that affect standard environment variables (like PATH or LD_LIBRARY_PATH). The following XML element tags are available:

The text inside the XML element is the path to be added to the variable in question. Multiple elements of the same tag are allowed.

Example:

<bindir>/opt/bin</bindir>
<libdir>/opt/lib</libdir>
<libdir>/opt/lib64</libdir>
<mandir>/opt/share/man</mandir>

At times VALET may not be able to fully express the modifications necessary for a package, or the package includes its own extensive environment configuration scripts. In such instances, a VALET package can be configured to simply execute (or source) that external script. A script action is an XML script element with the following attributes:

The text inside the XML script element is the path to the script to be sourced/executed. Multiple script elements are allowed.

The order determines which is tested first, the success return code or the failure return code. If no success or failure are specified, then the return code is not tested. Many shell scripts return code 0 on success and a non-zero value in case of a failure; this could be tested by supplying order = success_first and success = 0.

Example:

<script action="source" order="success-first" success="0" shell="sh">/opt/mathematica/bin/remote_kernel.sh</script>
<script action="source" order="success-first" success="0" shell="csh">/opt/mathematica/bin/remote_kernel.csh</script>

An alias is a word that represents an arbitrarily complex command in the shell environment. Rather than typing the command ls -l | less repeatedly, a user might create an alias named lll that when typed yields the same results as if ls -l | less had been typed. A shell alias action is an XML shell-alias element with the following attributes:

The text inside the XML shell-alias element is the command to be executed for the alias. A blank command implies that the named alias should be removed from the environment if it exists. Multiple shell-alias elements are allowed.

Example:

<shell-alias name="lll">/bin/ls -l | /usr/bin/less</shell-alias>
<!-- Remove the 'll' alias: -->
<shell-alias name="ll"/>

Dependencies and incompatibilities are XML elements that contain one or more package identifiers, package identifier pattern matches, or predicates that must:

• dependencies:
  • package id/pattern must be satisfied (present in environment or successfully loads and configures)
  • predicate evaluates to true
• incompatibilities:
  • package id/pattern must not be configured in the current environment
  • prediate evaluates to false

A pre-condition is tested before any environment changes are made for the sake of the package. A post-condition is tested after all environment changes have been made for the sake of the package.

Example:

<dependencies>
  <package id="pgi/14"/>
  <package id="fftw/^^3\."/>
</dependencies>
<incompatibilities>
  <predicate id="gaussian-scratch-test" stage="post-condition">
    <variable>GAUSS_SCRDIR</variable>
    <operator>starts-with</operator>
    <value>/home</value>
    <message>Home directories should not be used as scratch for Gaussian.</message>
  </predicate>
</incompatibilities>

A package id pattern is a string that is similar in construction to a regular package id but contains a regular expression for the package side, version side, or both sides of the versioned package id. A regular expression is indicated in either half by prefixing with a carat (^) character. For example, the pattern

openmpi/^^1\.[0-9]*[02468]\.

includes a regular expression in the version half of the id. Discarding the leading carat, the regular expression is

^1\.[0-9]*[02468]\.

This pattern will match with any even-numbered release of the openmpi package in the 1.x series: 1.4.4, 1.6.3, 1.8.2 would all be matches, but 1.5.1 would not.

As a dependency, id patterns are tested against packages that have already been loaded into the environment. If no matching package has been loaded, VALET locates an available package that matches the pattern and attempts to load it into the environment. Note that VALET simply chooses the first versioned package it finds as a match, which may or may not be a new enough version, etc. Ideally, the user will have loaded an appropriate version of the package prior to loading the package that uses an id pattern dependency.

A predicate is a test performed against an environment variable's value to determine whether package configuration should succeed or fail. Minimally, an XML predicate element must contain an environment variable name and an operator that specifies what test is to be performed. Most operators are binary and thus require a value against which to test; the is-set and not-is-set operators are unary and take no additional test value.

The predicate element may contain the following child elements:

Example:

<predicate id="gaussian-scratch-test" stage="post-condition">
  <variable>GAUSS_SCRDIR</variable>
  <operator>starts-with</operator>
  <value>/home</value>
  <message>Home directories should not be used as scratch for Gaussian.</message>
</predicate>

Prev: XML Package Files | Next: JSON/YAML Package Files