hpc documentation

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== WGSS:  WorkGroup-Sponsored Software ======

<WRAP center round important 60%>
This page documents a pilot project to increase inter-workgroup software visibility on the Caviness and DARWIN clusters.  The policies and terms discussed herein are open to change for the duration of the pilot program.
</WRAP>


IT-RCI staff maintain an extensive catalog of software under the ''/opt/shared'' file system on the clusters.  This software is organized in a hierarchy by package name and version, with VALET facilitating their usage in the shell environment.

The standing policy regarding what software qualifies for management by IT-RCI is essentially one of demand:  once requests for a package reach a threshold, the discussion of its support by IT-RCI can be effected.  In some cases, though, a cluster workgroup may want to take upon itself the burden of managing a publically-visible software catalog.  For example, several workgroups' collaborating on a software development project may want visibility without exposing their workgroup storage to all users of the cluster.  This is what **Workgroup-Sponsored Software** (or WGSS/wgss) hopes to address.

====== Creating a WGSS ======

The workgroup seeking to provision a WGSS must submit a [[https://services.udel.edu/TDClient/32/Portal/Requests/TicketRequests/NewForm?ID=D5ZRIgFlfLw_|Research Computing High Performance Computing (HPC) Clusters Help Request]] and supply the following:

  - All requests MUST come from the primary stakeholder of the workgroup or a group member designated by the primary stakeholder.
  - The stakeholder (or designated group member acting on the stakeholder's behalf) assumes all responsibility for maintaining the WGSS.
  - A unique, concise identifier («name») for the software project MUST be provided (e.g. "frey-dft").
  - An anticipated maximum byte size to act as the WGSS quota MAY be provided; a default quota of 25 GiB will be effected otherwise.
  - Please state which cluster, DARWIN and/or Caviness.

The WGSS storage will be provisioned as ''/opt/shared/wgss/«name»'' and owned by user "root" and the sponsoring workgroup.  Privileges on the base directory will be set as ''drwxrwsr-x'' (02775), allowing the workgroup to create whatever directory hierarchy they deem appropriate thereunder.  IT-RCI does **recommend** that the best-practices utilized in organizing ''/opt/shared'' be adopted in WGSS directories.

There will also be created by default a ''/opt/shared/wgss/«name»/valet'' directory for package definitions associated with the WGSS.  IT-RCI will add a VALET package, ''wgss/«name»'', whose sole purpose will be the addition of the WGSS' VALET directory to the VALET search path in the shell environment.  Any user seeking to utilize a specific WGSS can issue the command:

<code bash>
$ vpkg_require wgss/«name»
</code>

and thereafter will be able to work with packages defined in ''/opt/shared/wgss/«name»/valet'' in the current login or runtime shell.  WGSS maintainers are asked to exercise good judgment in the naming and versioning of packages they provide.  Choosing a ''«package-id»/«version-id»'' already provided by IT-RCI is likely to produce confusion or runtime issues with the software being provided; a feature tag can always be added to distinguish the WGSS package, e.g. ''«package-id»/«version-id»:«name»''.  For example, the "frey-dft" WGSS needing a specially-built Open MPI 4.1.0 might use the VALET id ''openmpi/4.1.0:frey-dft'' to clearly distinguish it from other versions/variants.

====== Storage Policies ======

Sponsoring workgroups MUST agree to the following policies with respect to the WGSS storage:

  - WGSS storage is granted for the sake of providing software in collaborative fashion.  The storage is NOT to be used as working directories for jobs executed on the cluster, nor as a repository for data generated by jobs executed on the cluster.  The storage is also NOT to be used in a high-throughput or high-turnover capacity (files altered very frequently).
  - IT-RCI reserves the right to suspend access to the WGSS storage if usage outside the WGSS scope is suspected.
  - IT-RCI does NOT offer support to the sponsor workgroup or other workgroups seeking to use the software present in the WGSS.  It is the sole responsibility of the sponsoring workgroup to provide VALET package definitions or at the least documentation on the software's use.  IT-RCI does NOT condone the modification of end users' Bash login files (''.bashrc'' or ''.bash_profile'') as a mechanism for adding software to the environment, so sponsor workgroups are asked to also foster this best-use policy in the software they provide.
  - Augmentation of the WGSS storage quota may be requested via IT trouble ticket.  As the sole owner of the storage hardware on which ''/opt/shared'' is resident, IT-RCI reserves the right to grant/deny such requests.
  - WGSS storage will be subject to periodic snapshotting and replication off-site.  IT-RCI reserves the right to manage those snapshots (removing if necessary) in pursuit of optimum and balanced performance of the storage hardware.
  - The sponsor workgroup may request the removal of the WGSS at any time WITHOUT the consent of other workgroups present on the cluster.
  - Removal of a WGSS shall encompass the deprovisioning of the granted storage and all snapshots on both production and off-site hardware.