Differences

This shows you the differences between two versions of the page.

--- software [2017-09-07 13:30] – created sraskar
+++ software [2017-09-07 16:54] (current) – removed sraskar
@@ Line 1: / Line 1: @@
-======= Software =======
-IT maintains a wide variety of software on the clusters it supports.  There are several categories of software involved:
-{{:software:floppy_disk.png?128 |}}
-^Category  ^Description  ^Examples  ^
-|**operating system (OS)**|Programs that collectively provide a working environment for computation|RedHat Enterprise Linux (RHEL)|
-|**development**|Programs that are used to create and refine other programs|Portland Compiler Suite, GNU Debugger (gdb)|
-|**libraries**|Reusable program fragments that provide additional functionality to programs|OpenMPI, ScaLAPACK|
-|**applications**|In HPC, a program that facilitates computation or data analysis|Matlab, Gaussian|
-IT is solely responsible for maintaining the OS on the HPC systems.  A number of development tools, libraries, and applications that are requested by multiple HPC users are also installed/compiled and managed by the IT HPC staff.  This area of the site is dedicated to documentation associated with the latter three categories of software.
-====== Availability ======
-Each cluster has a unique complement of development tools, libraries, and applications.  Visit the [[:clusters:start|Clusters]] page and view the documentation for a specific cluster to determine what software is available.
-====== Using software ======
-Preparing the Linux environment for a particular software title usually involves modification of the ''PATH'' or ''LD_LIBRARY_PATH'' variables, setting additional variables in the environment, or even executing scripted commands.  Often the documentation for installing such software suggests that the user edit his/her ''.bashrc'' or ''.bash_profile'' file so the changes are made automatically at login.  On UD HPC clusters, these tasks are best accomplished using [[:software:valet:start|VALET]], not by editing your login files.
-When IT adds new software to a cluster or upgrades existing software titles the work necessary to configure the environment for that software is modeled in a VALET //package file//.  Later, when a user issues the ''vpkg_require'' command to prepare his/her environment, VALET uses the package file to affect the necessary changes.
-===== VALET commands =====
-The following table provides a brief summary of the most often-used VALET commands.  Additional information is available in the [[:software:valet:start|VALET software documentation]].  These commands are available in login shells and batch scripts (since they are run as login shells).
-^Command  ^Description  ^
-|''vpkg_list''|List all available //packages// by their <<package-id>>|
-|''vpkg_versions'' <<package-id>>|List the versions/variants of a specific package by their <<version-id>>|
-|''vpkg_require'' <<versioned-package-id>>|Attempt to configure the environment to use the given software|
-|''vpkg_rollback'' <<number or all>>|Roll back a number of package additions, use all for every package|
-A <<versioned-package-id>> is the combination of a package identifier (found using the ''vpkg_list'' command) with a version/variant identifier (via the ''vpkg_versions'' command).  The two pieces are joined using a forward slash (/).  For example, the commercial software "Matlab" has a <<package-id>> of ''matlab'' and the 2012 release has a <<version-id>> of ''r2012a'', producing the <<versioned-package-id>> ''matlab/r2012a''.
-<note>In the context of a <<versioned-package-id>>, a blank <<version-id>> implies the default version of the package.  If the 2012 release of Matlab were the default, then using ''matlab'' (or ''matlab/default'') as the <<versioned-package-id>> would imply ''matlab/r2012a''.</note>
-VALET environment changes apply to the shell in which the ''vpkg_require'' command is issued; the changes do not "stick" after exiting that shell (by logging-out, for example).  Thus, one way to discard changes affected by a ''vpkg_require'' command is to just exit the shell.  The ''vpkg_rollback'' command is a more graceful alternative and iteratively discards the changes produced by prior ''vpkg_require'' commands issued in the shell.  To undo all changes, ''vpkg_require all'' can be used.
-A script contains commands to be executed in a shell.
-==== VALET source files ====
-When sourced, the commands in the script are executed in the current shell and any changes to the environment with be in affect when done. You can put VALET commands in a sourced script.
-<note tip>In source files containing VALET commands, it is good idea to rollback all packages before the VALET commands.
-<code bash>
-vpkg_rollback all
-</code>
-</note>
-When the script is done the environment may have been changed.  Only source VALET scripts when you want the changes made by VALET to stay in affect.
-==== VALET executable scripts ====
-When executing a script, a new shell is started with all exported variables and functions.  This shell does not have the VALET commands, unless it was executed as a login shell.
-<note tip>In an executable, non-login script containing VALET commands, the VALET functions must be setup. Put this source statement before the VALET commands:
-<code bash>
-source /etc/profile.d/valet.sh
-</code>
-</note>
-When the script is done, or encounters an exit command, the new shell is terminated and the environment is back to the state it was in before execution.
-===== Development ====
-IT staff has created and hosted HPC workshops, which frequently are archived as a tutorials. The exacted steps to follow along on IT managed clusters may be different.
-==== LaPlace Examples ====
-See [[general/training#recommended-hpc-tutorials|UD training materials]].
-  * [[software:tutorial:laplace| Laplace equation solver Serial version to OpenMP, MPI and OpenACC solutions]]
-==== Fortran Examples ====
-  * [[software:tutorial:fortran| Projects in Fortran]]
-===== Libraries =====
-IT staff install commercial and open-source libraries that can be  used in developing your own code. Use of some commercial libraries may require that your research group purchase a right-to-use license and provide of the appropriate license server information.
-==== FFTW ====
-See [[http://www.fftw.org/doc/| FFTW Users manual]].
-  *[[software:fftw | Compiling and linking]]
-==== LAPACK ====
-See [[http://www.netlib.org/lapack/lug/ | LAPACK Users' Guide -- Third Edition]].
-  * [[clusters:mills:lapack:dgels| Compiling and testing LAPACK]] with a //Real Linear Least Squares Routine// from [[http://www.nag.com/lapack-ex/| NAG Example programs for LAPACK drivers]].
-==== Open MPI ====
-**Details by cluster**
-  * [[software:openmpi:Mills]]
-  * [[software:openmpi:Farber]]
-==== ScaLAPACK ====
-See [[http://www.netlib.org/scalapack/slug/|ScaLAPACK user guide]].
-  * [[:clusters:mills:scalapack-example1|ScaLAPACK example1 from the userguide]]
-  * [[:clusters:mills:scalapack-example2|ScaLAPACK example2 from the userguide]]
-  * [[:clusters:mills:linsolve-benchmark|ScaLAPACK linsolve benchmark]]
-===== Applications =====
-IT staff install commercial and open-source applications that do not require you to develop your own code. Use of some commercial software may require that your research group purchase a right-to-use license and provide of the appropriate license server information.
-==== Abaqus ====
-**Vendor documentation:** [[http://www.3ds.com/products/simulia/portfolio/abaqus/abaqus-portfolio/|Abaqus]]
-  * [[software:abaqus| Using Abaqus]]
-==== ANSYS Fluent ====
-Vender documentaion: [[http://www.ansys.com/Products/Simulation+Technology/Fluid+Dynamics/ANSYS+Fluent|ANSYS home → Fluid dynamics → ANSYS Fluent]]
-  * [[:clusters:mills:ansys-fluent|Using ANSYS Fluent]]
-==== ArcGIS ====
-The Python shell embedded in ArcGIS 10.1 Server can be used on a Linux cluster to execute GIS tasks.  This is a (relatively) undocumented feature, and the steps necessary to install and configure the package for this mode of operation proved difficult.
-  * [[:software:arcgis:procedural_install|The installation procedure for ArcGIS]]:  what worked, what did not work, and the solution we came up with.  This document may be useful for other system administrators looking to get ArcGIS 10.1 Server to work in their Linux environment.
-  * [[:software:arcgis|Using ArcGIS shell]]
-==== Gaussian ====
-See [[http://gaussian.com| Gaussian]] - Expanding the limits of computational chemistry
-  * [[software:gaussian| Using Gaussian]]
-==== GROMACS ====
-See [[http://www.gromacs.org| GROMACS]] (GROningen MAchine for Chemical Simulations)
-  * [[software:gromacs| Using GROMACS]]
-==== HPCC ====
-HPCC version 1.4.2 a program to run a series of benchmarks, which comprises the //High Performance Computing Challenge//.
-[[/technical/whitepaper/start?&#hpc-challenge-awards-competition-at-sc-conference|HPC Challenge Awards Competition at SC Conference]]
-This program will read a file that contains the parameters needed to run the benchmarks.  This then runs all be benchmarks and
-report the results using the appropriate metric - Floating point in Tflops (or Gflops), latency in usec, bandwidth in GBS.
-Two benchmarks (HPL and DGEMM) use the ACML library for its BLAS. Version 5.2 of the ACML library has fftw wrappers, which you can use for the challenge.
-  * [[:clusters:mills:benchmark:hpcc:open64-acml|mills: hpcc with open64 compiler, ACML and base FFT]]
-  * [[:clusters:farber:benchmark:hpcc:intel-mkl|farber: hpcc with intel compiler, MKL and base FFT]]
-==== Matlab ====
-See [[http://www.mathworks.com/support/ | Matlab - Mathworks support]] and [[general/training#matlab|UD training materials]].
-  * [[software:matlab| Using Matlab]]
-==== Matplotlib ====
-See [[http://matplotlib.org/index.html|matplotlib - Official Website]] and [[:general:training#python|UD Training materials]]
-  * [[clusters:mills:matplotlib| Using Matplotlib]]
-==== Mpi4py ====
-See [[http://pythonhosted.org//mpi4py/usrman/index.html|MPI for Python v1.3 documentation - Official website]].
-  * [[clusters:mills:mpi4py| Using Mpi4py on Mills]]
-==== NAMD ====
-See [[http://www.ks.uiuc.edu/Research/namd/| NAMD Scalable Molecular Dynamics]]
-  * [[software:namd| Using NAMD]]
-==== OpenFOAM ====
-See [[http://www.openfoam.org/| The OpenFOAM Foundation]]
-  * [[clusters:mills:openfoam| Using OpenFOAM]]
-  * [[clusters:mills:openfoam/application| Installing an OpenFOAM application]]
-  * Running your Openfoam application
-    * [[clusters:mills:openfoam/running|Mills]]
-    * [[clusters:farber:openfoam/running|Farber]]
-==== ParaView ====
-"ParaView is an open-source, multi-platform application designed to visualize data sets of varying sizes from small to very large."
-See [[http://www.paraview.org/Wiki/ParaView/Users_Guide/Table_Of_Contents|ParaView Users Guide – Public Wiki]]
-One of the main purposes of ParaView is to allow users to visualize large data sets thatreside on HPC clusters without first collecting and downloading the data to a single machine.  The local workstation, which is good at visualization, may not have the memory or computing power for rending jobs that benefit from MPI on a cluster.  The solution is a client server model
-See [[http://www.paraview.org/Wiki/Users_Guide_Client-Server_Visualization|Client-Server chapter of Users Guide]]
-==== Python ====
-"Python is a programming language that lets you work more quickly and integrate your systems more effectively."
-See [[http://www.python.org/|Python Programming Language – Official Website]] and [[:general:training#python|UD Training materials]]
-  * [[:clusters:mills:python-htseq|Installing HTSeq: Analysing high-throughput sequencing data with Python]]
-  * [[:clusters:mills:python-canopy|Using Enthought Canopy Express: EPD (Enthought's Python Distribution) Free Academic License]]
-==== R ====
-"R is an integrated suite of software facilities for data manipulation, calculation and graphical display."
-See [[http://cran.r-project.org/manuals.html|R manuals]] for documentation available on using R in various formats.
-  * [[software:R| Using R]]
-==== TAU - Tuning and Analysis Utilities ====
-TAU Performance System® is a portable profiling and tracing toolkit for performance analysis of parallel programs written in Fortran, C, C++, Java, Python. See [[http://www.cs.uoregon.edu/Research/tau/doc.php|TAU's documentation]].
-==== Tecplot ====
-Tecplot 360 lets you quickly plot and animate your CFD data exactly the way you want it. You can analyze complex data, arrange multiple layouts, and communicate your results with professional images and animations.  See [[http://www.tecplot.com/products/tecplot-360/|Learn more about Tecplot 360]].
-  * [[:software:tecplot|Using Tecplot]]
-==== VASP ====
-The Vienna Ab initio Simulation Package (VASP) is a computer program for atomic scale materials modelling, e.g., electronic structure calculations and quantum-mechanical molecular dynamics.
-See [[http://cms.mpi.univie.ac.at/wiki/index.php/The_VASP_Manual|VASP online manual]].
-  * [[https://software.intel.com/en-us/articles/building-vasp-with-intel-mkl-and-intel-compilers|Building VASP with intel mkl and intel compilers]]