Table of Contents

XML Package Definition Files

In version 2 of VALET, packages (and their versions) can still be defined in XML files (with an augmented grammar for new features). VALET 2 can use .vpkg files from version 1 for backward compatibility, but many version 2 features are only available using the newer XML grammar.

VALET package definitions in the XML format use the file extension .vpkg or .vpkg_xml.

Grammar

A VALET package is represented in XML as a document with the following overall structure:

<?xml version="1.0" encoding="UTF-8"?>
<package
  xmlns="http://www.udel.edu/xml/valet/1.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.udel.edu/xml/valet/1.0
                      http://www.udel.edu/xml/valet/1.0/schema.xsd"
  id="packageid">
    :
  <version id="versionid">
    :
  </version>
</package>

A set of scalar properties can be used both in the package and version elements of the configuration. Scalar properties are:

ElementValue
<description>text…</description>Human-readable description of the software package
<url>hyperlink</url>A web site documenting the package, for example
<prefix>path</prefix>A filesystem path containing the package or the version of the package
<development-env/>VALET should emit LDFLAGS and CPPFLAGS
<no-development-env/>VALET should not emit LDFLAGS and CPPFLAGS
<standard-paths/>VALET should check for standard sub-paths like bin and lib under the prefix directory
<no-standard-paths/>VALET should not check for standard sub-paths like bin and lib under the prefix directory

Typically, the package-global prefix is set to a directory containing each version of the package, and each versioned package defines a prefix relative to that directory:

$ ls -l /opt/shared/openmpi
total 14
drwxr-sr-x 8 user   sysadmin 8 Aug 27 16:10 1.8.2
drwxr-sr-x 8 user   sysadmin 8 Aug 27 16:10 1.8.2-gcc-4.8.3
drwxr-sr-x 8 user   sysadmin 8 Aug 28 09:26 1.8.2-intel64
drwxrwsr-x 2 user   sysadmin 3 Aug 27 17:10 attic

The package's prefix = /opt/shared/openmpi and versions would use e.g. prefix = 1.8.2-intel64. If no prefix is provided for a versioned package, then VALET implicitly uses the version id.

Versioned packages that do not specify the development-env, no-development-env, standard-paths, or no-standard-paths properties inherit the value of their parent package.

Example:

<?xml version="1.0" encoding="UTF-8"?>
<package
  xmlns="http://www.udel.edu/xml/valet/1.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.udel.edu/xml/valet/1.0
                      http://www.udel.edu/xml/valet/1.0/schema.xsd"
  id="gaussian">
  <description>Gaussian Quantum Chemistry Suite</description>
  <url>http://gaussian.com/</url>
  <prefix>/opt/shared/gaussian</prefix>
 
  <no-development-env/>
  <no-standard-paths/>
 
  <version id="g09">
    <prefix>g09d01</prefix>
  </version>
  <version id="g09d01">
  </version>
</package>

In this example, version g09d01 has a prefix path of /opt/shared/gaussian/g09d01 since it did not specify a prefix. Version g09 provided a relative path that is appended to the prefix of its parent package to also yield /opt/shared/gaussian/g09d01. Both versions inherit the development-env and standard-paths behavior of the parent package.

The full versioned package identifiers available in the gaussian package defined above would be:

Since the g09 version targets the same executables as g09d01, the g09 version can be considered to be an alias of g09d01. VALET allows for aliased versions of packages:

  <version id="g09" alias-to="g09d01"/>
  <version id="g09d01">
  </version>

Once the overall versioning structure of a package has been created, the environment tests and changes associated with the package and versions of the package can be added. The three kinds of tests/changes are:

dependenciesother packages that must be present or loaded into the environment; environment variable tests that must evaluate to true for configuration to happen
incompatibilitiesother packages that must NOT be present in the environment; environment variable tests that must evaluate to false for configuration to happen
actionschanges to be made to the environment if all dependencies and incompatibilities are satisfied

If present in the package-global area of the configuration, the tests/changes affect all versions of the package and are satisfied/applied before version-specific dependencies/incompatibilities and actions.

Example:

<?xml version="1.0" encoding="UTF-8"?>
<package
  xmlns="http://www.udel.edu/xml/valet/1.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.udel.edu/xml/valet/1.0
                      http://www.udel.edu/xml/valet/1.0/schema.xsd"
  id="gaussian">
  <description>Gaussian Quantum Chemistry Suite</description>
  <url>http://gaussian.com/</url>
  <prefix>/opt/shared/gaussian</prefix>
 
  <no-development-env/>
  <no-standard-paths/>
 
  <dependencies>
    <predicate stage="pre-condition">
      <variable>GAUSS_SCRDIR</variable>
      <operator>not-is-set</operator>
      <message>If GAUSS_SCRDIR is not set, the working directory will be used; you do not want that.</message>
    </predicate>
  </dependencies>
 
  <actions>
    <script shell="all" action="exec" order="success-first" succeed="0">mk-gaussian-scrdir</script>
  </actions>
 
  <version id="g09" alias-to="g09d01"/>
  <version id="g09d01">
    <dependencies>
      <package id="pgi/14"/>
    </dependencies>
    <actions>
      <export variable="GAUSSIAN_VERSION" action="set">G09</export>
      <script action="source" shell="sh">g09.sh</script>
    </actions>
  </version>
</package>

Given the package definition above, typing vpkg_require gaussian/g09 yields the following course of events:

  1. g09 is an alias to g09d01, so version g09d01 becomes the target versioned package being configured
  2. Has the pgi/14 package been loaded into the environment yet?
    • If not, do all of its tests and attempt to affect all of its environment changes before proceeding
      • If that fails, this configuration fails and must exit immediately
  3. The installation prefix for the software is constructed as /opt/shared/gaussian/g09d01
  4. The GAUSS_SCRDIR environment variable is examined
    • If it does not exist in the environment the configuration fails and the message text is printed to stdout
  5. The mk-gaussian-scrdir program found in VALET's libexec directory is executed
    • If the result code returned by mk-gaussian-scrdir is zero, continue
    • Otherwise, the configuration fails and exits
  6. The GAUSSIAN_VERSION environment variable is assigned the value G09
  7. For Bash and other sh-like shells, the script g09.sh in VALET's libexec directory is sourced
    • The return code is not checked for success or failure

If control reaches the end of this pseudo-code, then the gaussian/g09 package has been successfully configured in the environment.

Actions

An XML actions element can be specified either in a package or in a version of a package and contains zero or more action elements of the following kinds:

Each action is represented as an XML element. Actions attached to the package itself will be performed when any version of the package is selected for addition to the environment, and in addition to actions associated with the version itself.

Actions will be performed in the order they are specified in the block.

Example:

  <actions>
    <incdir>/opt/include</incdir>
    <export variable="PATH" action="scrub-path">/opt/bin</export>
    <bindir>/opt/bin</bindir>
    <libdir>/opt/lib</libdir>
    <libdir>/opt/lib64</libdir>
    <mandir>/opt/share/man</mandir>
    <export variable="USER_SW_PATH">/opt/bin</export>
    <export variable="CFLAGS" action="append-space" development-env="false">-O3 -g</export>
  </actions>

Variable action

Variable actions alter the value of an environment variable. A variable action is an XML export element with the following attributes:

AttributeValueOptional
variableEnvironment variable namerequired
actionset unset append prepend append-path prepend-path append-space prepend-space scrub scrub-pathdefault = set
development-envOnly set this variable when vpkg_devrequire is useddefault = no

The text inside the export element is the value to add/remove/etc.

The actions are self-explanatory for the most part. The scrub action removes all occurrences of value from the variable's current value in the environment. The scrub-path action treats the value of variable as a search path and removes from that search path all occurrences (by exact match) of value.

The value may contain references to other environment variables using the syntax ${ENV VAR NAME}.

Example:

<export variable="PATH" action="scrub-path">/opt/bin</export>

Special directories

All special-directory actions are enclosed in an XML element within the actions element. These actions are shortcuts that produce variable actions that affect standard environment variables (like PATH or LD_LIBRARY_PATH). The following XML element tags are available:

Element TagValue
bindirPaths to add to PATH
libdirPaths to add to LD_LIBRARY_PATH (and LDFLAGS under vpkg_devrequire)
incdirPaths to add to CPPFLAGS under vpkg_devrequire
mandirPaths to add to MANPATH
infodirPaths to add to INFOPATH
pkgconfigdirPaths to add to PKG_CONFIG_PATH

The text inside the XML element is the path to be added to the variable in question. Multiple elements of the same tag are allowed.

Example:

<bindir>/opt/bin</bindir>
<libdir>/opt/lib</libdir>
<libdir>/opt/lib64</libdir>
<mandir>/opt/share/man</mandir>

Scripts

At times VALET may not be able to fully express the modifications necessary for a package, or the package includes its own extensive environment configuration scripts. In such instances, a VALET package can be configured to simply execute (or source) that external script. A script action is an XML script element with the following attributes:

AttributeValue
actionsource or exec
orderfailure-first or success-first
successinteger return code indicating successful execution/sourcing
failureinteger return code indicating failed execution/sourcing
shellshell type for which this script is executed

The text inside the XML script element is the path to the script to be sourced/executed. Multiple script elements are allowed.

The order determines which is tested first, the success return code or the failure return code. If no success or failure are specified, then the return code is not tested. Many shell scripts return code 0 on success and a non-zero value in case of a failure; this could be tested by supplying order = success_first and success = 0.

Example:

<script action="source" order="success-first" success="0" shell="sh">/opt/mathematica/bin/remote_kernel.sh</script>
<script action="source" order="success-first" success="0" shell="csh">/opt/mathematica/bin/remote_kernel.csh</script>

Shell Aliases

An alias is a word that represents an arbitrarily complex command in the shell environment. Rather than typing the command ls -l | less repeatedly, a user might create an alias named lll that when typed yields the same results as if ls -l | less had been typed. A shell alias action is an XML shell-alias element with the following attributes:

AttributeValue
shellshell type for which this alias is valid
development-envset to boolean true to restrict this alias to only be set on vpkg_devrequire

The text inside the XML shell-alias element is the command to be executed for the alias. A blank command implies that the named alias should be removed from the environment if it exists. Multiple shell-alias elements are allowed.

Example:

<shell-alias name="lll">/bin/ls -l | /usr/bin/less</shell-alias>
<!-- Remove the 'll' alias: -->
<shell-alias name="ll"/>

Dependencies & Incompatibilities

Dependencies and incompatibilities are XML elements that contain one or more package identifiers, package identifier pattern matches, or predicates that must:

A pre-condition is tested before any environment changes are made for the sake of the package. A post-condition is tested after all environment changes have been made for the sake of the package.

Example:

<dependencies>
  <package id="pgi/14"/>
  <package id="fftw/^^3\."/>
</dependencies>
<incompatibilities>
  <predicate id="gaussian-scratch-test" stage="post-condition">
    <variable>GAUSS_SCRDIR</variable>
    <operator>starts-with</operator>
    <value>/home</value>
    <message>Home directories should not be used as scratch for Gaussian.</message>
  </predicate>
</incompatibilities>

Package Id Patterns

A package id pattern is a string that is similar in construction to a regular package id but contains a regular expression for the package side, version side, or both sides of the versioned package id. A regular expression is indicated in either half by prefixing with a carat (^) character. For example, the pattern

openmpi/^^1\.[0-9]*[02468]\.

includes a regular expression in the version half of the id. Discarding the leading carat, the regular expression is

^1\.[0-9]*[02468]\.

This pattern will match with any even-numbered release of the openmpi package in the 1.x series: 1.4.4, 1.6.3, 1.8.2 would all be matches, but 1.5.1 would not.

As a dependency, id patterns are tested against packages that have already been loaded into the environment. If no matching package has been loaded, VALET locates an available package that matches the pattern and attempts to load it into the environment. Note that VALET simply chooses the first versioned package it finds as a match, which may or may not be a new enough version, etc. Ideally, the user will have loaded an appropriate version of the package prior to loading the package that uses an id pattern dependency.

Using a regular expression that is the negation of what you would use for a dependency and employing an incompatibility forces the user to load the dependency prior to loading this package. After all, package ids and id patterns inside an incompatibilities element do not cause VALET to attempt to load a package into the environment.

Predicates

A predicate is a test performed against an environment variable's value to determine whether package configuration should succeed or fail. Minimally, an XML predicate element must contain an environment variable name and an operator that specifies what test is to be performed. Most operators are binary and thus require a value against which to test; the is-set and not-is-set operators are unary and take no additional test value.

AttributeValueOptional
operatoris-set: has a valuerequired
is-not-set: does not have a value
eq, ==: lexigraphically equivalent to value
ne, !=: not lexigraphically equivalent to value
lt, <: lexigraphically less-than value
le, <=: lexigraphically less-than or equivalent to value
gt, >: lexigraphically greater-than value
ge, >=: lexigraphically greater-than or equivalent to value
starts-with, <<: value as prefix
not-starts-with, !<<: not value as prefix
ends-with, >>: value as suffix
not-ends-with, !>>: not value as suffix
contains, <>: contains value
not-contains, !<>: does not contain value
matches, ~: matches regular expression in value
not-matches, !~: does not match regular expression in value
stagepre-conditiondefault = pre-condition
post-condition

The predicate element may contain the following child elements:

Element TagPurpose Optional
variableEnvironment variable namerequired
operatorThe attribute can instead be specified as a child element
valueValue to compare against for binary operatorsrequired except for operator = is-set or not-is-set
messageString that will be displayed if the predicate is not satisfied

Example:

<predicate id="gaussian-scratch-test" stage="post-condition">
  <variable>GAUSS_SCRDIR</variable>
  <operator>starts-with</operator>
  <value>/home</value>
  <message>Home directories should not be used as scratch for Gaussian.</message>
</predicate>

Prev: XML Package Files | Next: JSON/YAML Package Files