hpc documentation

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== XML Package Definition Files (version 1.0) ======

{{:software:valet:valet-icon.png?128 |}}VALET packages (and their versions) are defined in XML files; each package available on a system has its own package definition file, named as <<''package-id''>>''.vpkg''.  For example, all versions of Mathematica would be defined in a file named ''mathematica.vpkg''.

<note important>In all examples only lowercase letters are used in package identifiers.  This behavior is not enforced by VALET, and the identifier ''Mathematica'' is actually unique relative to ''mathematica''.  It is recommended that a strict policy toward case in package names be adopted by computing sites in order to promote consistency for the end user.

The valid character set for package and version identifiers consists of the alphanumeric ASCII characters plus the dot, dash, and underscore.  Ids must match the regular expression ''[A-Za-z0-9_.-]+''.</note>

Package definition files are collocated in a directory.  By default (and minimally), the ''etc'' subdirectory inside the ''valet'' installation directory is used.

===== File Format =====

A VALET package definition is expressed in XML.  The XML schema that explicitly declares the XML language can be found [[http://www.udel.edu/xml/valet/1.0/schema.xsd|here]].

The XML document element is named ''package'' and has a single required attribute:

<code xml>
<package id="mathematica">
  <description>Wolfram Mathematica</description>
  <url>http://www.wolfram.com/</url>
  <prefix>/opt/mathematica</prefix>
    :
</package>
</code>

The ''id'' must match that of the package definition filename.  A package can optionally have a human-readable description and a reference URL (e.g. the vendor/product web site).

Ideally, all versions of a package are installed in unique directories under a common parent directory.  If this is indeed the case, then a ''prefix'' element can be added to the ''package'' element as shown above.  When the individual package versions are later defined, they can provide a relative path in their own ''prefix'' element which VALET takes as being relative to the package's ''prefix'' path.

==== Versions ====

Each version of a package is defined as a ''version'' XML element with a required ''id'' attribute:

<code xml>
    :
  <version id="6">
    <description>Wolfram Mathematica version 6</description>
    <url>http://www.wolfram.com/6</url>
    <prefix>6.0.1</prefix>
    <bindir>Executables</bindir>
    <dependencies>
      <package id="pgi/10"/>
    </dependencies>
    <incompatibilities>
      <package id="matlab/*"/>
    </incompatibilities>
      :
  </version>
    :
</code>

The optional ''description'' and ''url'' elements override the value presented in the parent ''package'' element.

As mentioned earlier, the ''prefix'' element specified here is a relative path that VALET assumes is relative to the ''prefix'' provided by the parent ''package'':  ''/opt/mathematica/6.0.1'' would be the full path in this case.

<note tip>For ''version'' elements lacking a ''prefix'', the version id will be used in its stead as a path component relative to the parent package's ''prefix''.</note>

Zero or more ''bindir'' elements specify paths that should be added to the ''PATH'' environment variable.  Relative paths are taken to be relative to the ''prefix'' directory.  The directories that can be specified are:

^Element Name^Env Var Affected^Description^Defaults^
|''bindir''|''PATH''|Directories containing executables|''bin'', ''sbin''|
|''libdir''|''LD_LIBRARY_PATH'', ''LDFLAGS''|Directories containing libraries|''lib'', ''libso''|
|''mandir''|''MANPATH''|Directories containing manual pages|''man'', ''share/man''|
|''infodir''|''INFOPATH''|Directories containing manual pages|''share/info''|
|''incdir''|''CPPFLAGS''|Directories containing header files|''include''|
|''pkgconfigdir''|''PKG_CONFIG_PATH''|Directories containing .pc files for ''pkg-config''|''lib/pkgconfig'', ''share/pkgconfig''|

If any of these elements is not present for a ''version'', then VALET automatically considers the default directory(s).  In all cases, only directories that actually exist will be setup in the environment.

<note tip>The ''LDFLAGS'' and ''CPPFLAGS'' environment variables are set to affect software compilation and linking.  The user must explicitly request their inclusion in the environment setup.</note>

For software packages that are built using other packages managed by VALET, a list of ''dependencies'' can be specified.  Each dependency is expressed as an empty ''package'' element with the ''id'' of the other (versioned) package; in the example above, Mathematica 6 for some reason depends on version 6 of the Portland compiler suite.  When configuring the environment, VALET will automatically attempt to include all dependency packages' configuration details, as well.

A list of ''incompatibilities'' can be included to express that a versioned package conflicts with some other versioned package.  Each incompatibility is expressed as an empty ''package'' element with the ''id'' of the other (versioned) package.  Note in the example above Mathematica 6 conflicts (somehow) with //any// version of Matlab:  the ''*'' acts as a wildcard((The use of a version wildcard in a versioned package id is //only// valid in the context of the ''incompatibilities'' element.)).  Inclusion of any version of Matlab either prior to or after this versioned package's inclusion will yield an error.  Of course, actual version ids can be present, and as always the lack of a version id implies the default version of the package.

Both the ''dependencies'' and ''incompatibilities'' elements can also contain environment variable tests.  The ''predicate'' element is used to model such tests and is described [[#predicates|later in this document]].

Multiple ''version'' elements may appear in a ''package'' document; all ''version'' elements are children of the ''package'' element.  A ''default-version'' element in the ''package'' document indicates which ''version'' should be considered the default; omitting the ''default-version'' element, the first ''version'' element will be the default for the ''package'':

<code xml>
<package id="mathematica">
    :
  <default-version>7</default-version>
  <version id="6">..</version>
  <version id="7">..</version>
  <version id="8">..</version>
    :
</package>
</code>

==== Aliases ====

A version //alias// is a unique package version that points to a sibling version.  For example, Gaussian releases their G09 product and its incremental updates as a series, e.g. ''g09a01'', ''g09a02'', ''g09b03'', etc.  Most users probably don't care what revision of G09 they are using, so long as they are using G09.  By specifying a version alias

<code xml>
    :
  <version id="g09" alias-to="g09b03"/>
    :
</code>

users can always just request ''gaussian/g09'' and the system administrator can change the alias target as future incremental updates are installed on the system.  As illustrated, a version alias starts like a regular ''version'' element but has no content and includes the additional attribute ''alias-to''.  The alias must have its own unique version identifier in the ''id'' attribute, and the identifier of some other version in the ''alias-to'' attribute.
==== Static Exports ====

In some cases there may be additional environment variables that need to be modified for a package/version.  To facilitate this, zero or more ''export'' elements can be added to a ''package'' or ''version'' element:

<code xml>
<package id="mathematica">
    :
  <export variable="MATHEMATICA_IN_USE">1</export>
  <version id="8">
      :
    <export variable="MATHEMATICA_DOCS" action="path-append">${HOME}/mma_8</export>
      :
  </version>
    :
</package>
</code>

An ''export'' element inside a ''version'' element that has the same ''variable'' name as one in the parent ''package'' overrides the behavior defined in the ''package''.  The ''action'' can be:

^Action^Description^
|''set''|Sets the value|
|''path-prepend''|Sets or prefixes (with a colon separator) the value a'la the ''PATH'' variable|
|''path-append''|Sets or suffixes (with a colon separator) the value a'la the ''PATH'' variable|

If no ''action'' attribute is specified the ''set'' action is assumed.

<note important>To embed references to the value of other environment variables in an ''export'' element the ''${VAR-NAME}'' syntax //must// be used.  VALET will not recognize ''$VAR-NAME'' style references in values!</note>
==== Scripts ====

If there is additional "work" that cannot be encapsulated in VALET configuration directives, the system administrator can write an external script that will be sourced into the user's shell.  A ''version'' element can have zero or more ''script'' elements within it that provide the path to each such script:

<code xml>
    :
  <version id="6">
      :
    <script shell="sh">/opt/mathematica/bin/remote_kernel.sh</script>
      :
  </version>
    :
</code>

<note tip>VALET includes a ''libexec'' directory into which you can install helper scripts; relative script paths are taken to be relative to the ''libexec'' directory.</note>

VALET defines some environment variables that scripts may want to use:

^Variable^Description^
|''VALET_PKG_ID''|The package id associated with the version for which the script was invoked.|
|''VALET_PATH_PREFIX''|The path prefix associated with the version for which the script was invoked.|

Scripts are invoked at the end of the environment configuration process, so all VALET-oriented changes to the environment are in-place when scripts are executed.

==== Predicates ====

In some cases it may be necessary to test the value of one or more environment variables to determine whether or not the package setup should succeed or fail.  For example, if the user has no ''GAUSS_SCRDIR'' defined then Gaussian will have no directory in which to write scratch files.  This could be described as either a dependency or an incompatibility.

Such tests are modeled with a ''predicate'' element.  A ''predicate'' element is valid inside the ''dependencies'' and ''incompatibilities'' elements of a ''version'' element.  The ''predicate'' element requires a ''variable'' attribute; an ''operator'' and ''stage'' attribute may also be specified:

<code xml>
     :
    <dependencies>
       :
      <predicate variable="GAUSS_SCRDIR" operator="is-set" stage="post-condition"/>
       :
    </dependencies>
     :
</code>

In this case, the predicate could also be configured as an incompatibility:

<code xml>
     :
    <incompatibilities>
       :
      <predicate variable="GAUSS_SCRDIR" operator="not-is-set" stage="post-condition"/>
       :
    </incompatibilities>
     :
</code>

The ''stage'' attribute dictates whether the test should be performed //before// (''pre-condition'') or //after// (''post-condition'') VALET makes changes to the environment.  If no ''stage'' attribute is specified, the predicate will be a pre-condition.

Valid operators are:

^Name ^Meaning ^ Unary/Binary ^
|''is-set''|Non-empty value|unary|
|''is-not-set''|Empty value|unary|
|''=='', ''eq''|Equivalent to the given string|binary|
|''!='', ''ne''|Not equivalent to the given string|binary|
|''<'', ''lt''|Lexigraphically less-than the given string|binary|
|''%%<=%%'', ''le''|Lexigraphically less-than or equivalent to the given string|binary|
|''>'', ''gt''|Lexigraphically greater-than the given string|binary|
|''%%>=%%'', ''ge''|Lexigraphically greater-than or equivalent to the given string|binary|
|''%%<<%%'', ''starts-with''|Starts with the given string|binary|
|''!%%<<%%'', ''not-starts-with''|Does not start with the given string|binary|
|''%%>>%%'', ''ends-with''|Ends with the given string|binary|
|''!%%>>%%'', ''not-ends-with''|Does not end with the given string|binary|
|''%%<>%%'', ''contains''|Contains the given string|binary|
|''!%%<>%%'', ''not-contains''|Does not contain the given string|binary|
|''~'', ''matches''|Matches the given Python regular expression|binary|
|''!~'', ''not-matches''|Does not match the given Python regular expression|binary|

For binary operators, the required argument is provided in a ''<value>'' element inside the ''<predicate>'' element.  For example, to ensure that setup does not proceed if a user has pre-set ''GAUSS_SCRDIR'' to a filesystem that should not be used as scratch, the following incompatibility predicate might be used:

<code xml>
     :
    <incompatibilities>
       :
      <predicate variable="GAUSS_SCRDIR" operator="matches">
        <value>^/(home|archive)/</value>
        <explanation>Storing Gaussian scratch files on /home or /archive is forbidden.</explanation>
      </predicate>
       :
    </incompatibilities>
     :
</code>

If a user subsequently attempts to ''vpkg_require gaussian'' the ''explanation'' will be displayed if they have ''GAUSS_SCRDIR'' misconfigured:

<file>
ERROR: an error occurred while altering your environment
REASON: Storing Gaussian scratch files on /home or /archive is forbidden.
</file>

All predicates must pass in order for environment configuration to succeed; essentially, success is the logical AND of all predicates specified.

<note>Like package/version dependencies and incompatibilities, predicates are preserved across calls to ''vpkg_require''.  If a later ''vpkg_require'' produces changes to the environment that conflict with any predicates, old or new, the changes will be reverted and an error message displayed.</note>

----

[[software:valet:02_packages|Prev: Packages]] | [[software:valet:03b_packagefile|Next: XML Package Files, version 2]]