hpc documentation

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== R on Caviness =====

==== Learning R ====

== SWIRL ===
In addition to other resources, SWIRL is installed on the Caviness cluster and is available as an interactive learning guide
inside R:

<code>
$ vpkg_require r-cran
$ R -q --no-save
> library(swirl)
> swirl()
</code>



==== R libraries and extensions ====

=== Installed library bundles ===
The cluster also has the majority of [[http://cran.us.r-project.org/|CRAN]]
and [[http://www.bioconductor.org/|Bioconductor]] R libraries already
insalled.  These are installed as point-in-time snapshots of their
respective catalogs.  These libraries are broken down into different valet
packages based on dependencies.  The current bundles are below.  Together
these bundles provide access to over 6,600 R modules, pre-compiled and ready
for use.

^r-cran         |All CRAN modules in CRAN which compile and install cleanly without any additional dependencies.  N.B. all below library packs require this CRAN modle as a base.|
^r-cdf          |CRAN modules which need NetCDF, HDF4, HDF5, and UDUNITS libraries. |
^r-bioc         |The full suite of[[http://www.bioconductor.org/|Bioconductor]] modules. |
^r-fftw         |CRAN modules which need FFTW |
^r-geo          |CRAN modules which need GEOS(Geometry Engine, Open Source), GDAL(Geospatial Data Abstraction Library), or PROJ (Cartographic Projections Library)   |
^r-gnumath      |CRAN modules which need GSL(GNU Scientific Library), GLPK(GNU Linear Programming Kit), or MPFR(GNU MPFR Library)  |
^r-jags         |CRAN modules which need JAGS(Just Another Gibbs Sampler) and the r-gnumath library mentioned above. |
^r-graph        |CRAN modules which need Graphviz or GNUplot |
^r-mpi          |CRAN modules which need the OpenMPI libraries for parallel computing.  |
^r-all          |In addition to loading all the previously mentioned bundles, and CRAN module with multiple dependencies from the above list is also included. |
^r-cuda         |CRAN modules which need CUDA/GPUs |

=== Loading library bundles for use ===
<code>
$ vpkg_require r-geo
Adding dependency `r-bioc/3.5.1:20180715` to your environment
Adding dependency `gsl/1.16` to your environment
Adding dependency `gmp/6.1.2` to your environment
Adding dependency `glpk/4.65` to your environment
Adding dependency `mpfr/4.0.1` to your environment
Adding dependency `r-gnumath/3.5.1:20180715` to your environment
Adding dependency `fftw/3.3.8` to your environment
Adding dependency `r-fftw/3.5.1:20180715` to your environment
Adding dependency `szip/2.1.1` to your environment
Adding dependency `hdf4/4.2.13` to your environment
Adding dependency `hdf5/1.10.2` to your environment
Adding dependency `netcdf/4.6.1` to your environment
Adding dependency `udunits/2.2.26` to your environment
Adding dependency `r-cdf/3.5.1:20180715` to your environment
Adding dependency `geos/3.6.2` to your environment
Adding dependency `gdal/2.3.0` to your environment
Adding dependency `proj/5.1.0` to your environment
Adding package `r-geo/3.5.1:20180715` to your environment
$
</code>

Now using the library in R can be done as normal.

<code>
$ R --no-save -q
> library(CopulaRegression)
Loading required package: MASS
Loading required package: VineCopula
>
</code>

=== Learning about modules ===
IT provides a small script called ''r-info'' which will display the internal
documentation of R modules.  This is helpful to get basic information on
a module to decide if it requires more research.  To use this tool, the library
must be installed, and the module bundle must be loaded with ''vpkg_require''.
For example:

<code>
$ vpkg_require r-cran
$ r-info car
Loading required package: carData

                Information on package ‘car’

Description:

Package:            car
Version:            3.0-0
Date:               2018-03-23
Title:              Companion to Applied Regression

...

Further information is available in the following vignettes in
directory ‘/opt/shared/r/add-ons/r3.5.1/cran/20180715/car/doc’:

embedding: Using car functions inside user functions (source, pdf)     
$
</code>

==== personal/program specific R libraries and extensions ====
You can create your own library of R modules which contains different
versions than provided through VALET, or modules not available via VALET.

R looks in an environment variable called ''R_LIBS'' to obtain a list of
locations to search for modules.  You should ensure your entry is first
in the list, this will allow your library to override any conflicts which
may be installed on the system.  This is also important, because R installs
modules into the first entry in this list by default.

=== Simple example ===
Once this is done, you can install by using ''install.packages''.  Make sure you are in your workgroup (e.g. ''workgroup -g <<//investing-entity//>>''.  Here
is an example:

<code>
$ workgroup -g it_css
$ vpkg_require r-cran
Adding dependency `r/3.5.1` to your environment
Adding package `r-cran/3.5.1:20180715` to your environment
$ mkdir -p $WORKDIR/sw/r/add-ons/r3.5.1/testing/default
$ echo $R_LIBS
/opt/shared/r/add-ons/r3.5.1/cran/20180715
$ export R_LIBS="$WORKDIR/sw/r/add-ons/r3.5.1/testing/default:$R_LIBS"
$ R -q --no-save
> .libPaths()
[1] "/work/it_css/sw/r/add-ons/r3.5.1/testing/default"
[2] "/opt/shared/r/add-ons/r3.5.1/cran/20180715"
[3] "/opt/shared/r/3.5.1/lib64/R/library"
> chooseCRANmirror(all)
Secure CRAN mirrors

 1: 0-Cloud [https]                   2: Algeria [https]
 3: Australia (Canberra) [https]      4: Australia (Melbourne 1) [https]
 5: Australia (Melbourne 2) [https]   6: Australia (Perth) [https]
 7: Austria [https]                   8: Belgium (Ghent) [https]
 9: Brazil (PR) [https]              10: Brazil (RJ) [https]
11: Brazil (SP 1) [https]            12: Brazil (SP 2) [https]
13: Bulgaria [https]                 14: Chile [https]
15: China (Hong Kong) [https]        16: China (Lanzhou) [https]
17: China (Shanghai) [https]         18: Colombia (Cali) [https]
19: Czech Republic [https]           20: Denmark [https]
21: Ecuador (Cuenca) [https]         22: Ecuador (Quito) [https]
23: Estonia [https]                  24: France (Lyon 2) [https]
25: France (Marseille) [https]       26: France (Montpellier) [https]
27: Germany (Erlangen) [https]       28: Germany (Göttingen) [https]
29: Germany (Münster) [https]        30: Germany (Regensburg) [https]
31: Greece [https]                   32: Hungary [https]
33: Iceland [https]                  34: Indonesia (Jakarta) [https]
35: Italy (Padua) [https]            36: Japan (Tokyo) [https]
37: Japan (Yonezawa) [https]         38: Korea (Busan) [https]
39: Korea (Gyeongsan-si) [https]     40: Korea (Seoul 1) [https]
41: Korea (Ulsan) [https]            42: Malaysia [https]
43: Mexico (Mexico City) [https]     44: Norway [https]
45: Philippines [https]              46: Serbia [https]
47: Spain (Madrid) [https]           48: Sweden [https]
49: Switzerland [https]              50: Turkey (Denizli) [https]
51: Turkey (Mersin) [https]          52: UK (Bristol) [https]
53: UK (London 1) [https]            54: USA (CA 1) [https]
55: USA (IA) [https]                 56: USA (KS) [https]
57: USA (MI 1) [https]               58: USA (MI 2) [https]
59: USA (OR) [https]                 60: USA (TN) [https]
61: USA (TX 1) [https]               62: Uruguay [https]
63: (other mirrors)

Selection: 55
> install.packages("KernSmooth", dependencies=TRUE)
Installing package into ‘/work/it_css/sw/r/add-ons/r3.5.1/testing/default’
(as ‘lib’ is unspecified)

trying URL 'https://ftp.osuosl.org/pub/cran/src/contrib/KernSmooth_2.23-15.tar.gz'
Content type 'application/x-gzip' length 24572 bytes (23 KB)
==================================================
downloaded 23 KB

* installing *source* package ‘KernSmooth’ ...
** package ‘KernSmooth’ successfully unpacked and MD5 sums checked
** libs
gfortran   -fpic  -g -O2  -c blkest.f -o blkest.o
gfortran   -fpic  -g -O2  -c cp.f -o cp.o
gfortran   -fpic  -g -O2  -c dgedi.f -o dgedi.o
gfortran   -fpic  -g -O2  -c dgefa.f -o dgefa.o
gfortran   -fpic  -g -O2  -c dgesl.f -o dgesl.o
gcc -std=gnu99 -I"/opt/shared/r/3.5.1/lib64/R/include" -DNDEBUG   -I/opt/shared/gcc/4.9.4/include   -fpic  -g -O2  -c init.c -o init.o
gfortran   -fpic  -g -O2  -c linbin.f -o linbin.o
gfortran   -fpic  -g -O2  -c linbin2D.f -o linbin2D.o
gfortran   -fpic  -g -O2  -c locpoly.f -o locpoly.o
gfortran   -fpic  -g -O2  -c rlbin.f -o rlbin.o
gfortran   -fpic  -g -O2  -c sdiag.f -o sdiag.o
gfortran   -fpic  -g -O2  -c sstdiag.f -o sstdiag.o
gcc -std=gnu99 -shared -L/opt/shared/r/3.5.1/lib64/R/lib -L/opt/shared/gcc/4.9.4/lib -L/opt/shared/gcc/4.9.4/lib64 -o KernSmooth.so blkest.o cp.o dgedi.o dgefa.o dgesl.o init.o linbin.o linbin2D.o locpoly.o rlbin.o sdiag.o sstdiag.o -L/opt/shared/r/3.5.1/lib64/R/lib/atlas -lRblas -lgfortran -lm -lquadmath -lgfortran -lm -lquadmath -L/opt/shared/r/3.5.1/lib64/R/lib -lR
installing to /work/it_css/sw/r/add-ons/r3.5.1/testing/default/KernSmooth/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (KernSmooth)

The downloaded source packages are in
        ‘/tmp/RtmpVq5oBb/downloaded_packages’
> library(KernSmooth)
KernSmooth 2.23 loaded
Copyright M. P. Wand 1997-2009
>
</code>

Notice that the output of ''.libPaths()'' specifies my personal library directory first?  Also it is very important to make sure all dependencies for a library are installed by using the option ''dependencies=TRUE'' for ''install.packages''.  It may be necessary to install a dependency first before trying to install the library you want if the developers of a particular library did not provide the dependency as part of the install.  In the complex example below, even though ''terra'' doesn't indicate a dependency for ''quantregForest'' it may need to be installed using ''install.packages("quantregForest", dependencies=TRUE)'' first before doing ''install.packages("terra", dependencies=TRUE)''.  Unfortunately much of this requires reviewing the developer documentation for each library to determine system requirements, dependencies and noting these dependencies may not be available or automatically installed. On top of that some dependencies require other packages to be installed first, so it can become a catch 22 situation and require trial and error to find the correct order such as needing to install ''bslib'' which is needed for ''rmarkdown'' which is needed for ''quantregForest'' installation. Lastly, make sure you look at what VALET packages are already available for installing R libraries such as the version of the compiler necessary and other system libraries like ''gdal'', ''proj'', ''geos'', etc. Using ''vpkg_devrequire'' will load other dependencies as well set environment variables such as ''LDFLAGS'', ''CPPFLAGS'', ''LIBRARY_PATH'' and ''LD_LIBRARY_PATH'' as well as ''PREFIX'' variables that are necessary in order to find the correct files during the installation of a particular R library.  For example, by loading ''gdal'' and ''geos'' we get a number of other dependencies loaded in our environment as well the appropriate environment variables that will be needed by R to install a library. As part of this trial and error, ''udunits'' failed which required quitting from R, then using ''vpkg_devrequire udunits/2.2'' to load into the environment, setting the environment variables ''UDUNITS2_INCLUDE'' and ''UDUNITS2_LIBS'' based on the environment variable ''UDUNITS_PREFIX'' by VALET, and getting back in to R to try the install again.  This is basically the trial and error process involved. Finally you will need to load all the same packages in VALET using ''vpkg_require'' and set any environment variables before trying to use the libraries you installed in your ''R_LIBS'' for this session.

=== Complex example ===
<code>
[traine@login01.caviness ~]$ workgroup -g it_css
[(it_css:traine)@login01.caviness ~]$ vpkg_devrequire r/4.1 gdal/3.4.3 geos/3.9.1
Adding dependency `binutils/2.35` to your environment
Adding dependency `gcc/11.2.0` to your environment
Adding dependency `atlas/3.10.3` to your environment
Adding package `r/4.1.3` to your environment
Adding dependency `szip/2.1.1` to your environment
Adding dependency `hdf4/4.2.13` to your environment
Adding dependency `hdf5/1.10.2` to your environment
Adding dependency `netcdf/4.6.1` to your environment
Adding dependency `sqlite3/3.34.1` to your environment
Adding dependency `proj/8.2.1` to your environment
Adding package `gdal/3.4.3` to your environment
Adding package `geos/3.9.1` to your environment
[(it_css:traine)@login01.caviness ~]$ echo $LDFLAGS
-L/opt/shared/binutils/2.35/lib -L/opt/shared/gcc/11.2.0/lib -L/opt/shared/gcc/11.2.0/lib64 -L/opt/shared/atlas/30.3/lib -L/opt/shared/r/4.1.3/lib64 -L/opt/shared/r/4.1.3/lib64/R/lib -L/opt/shared/r/4.1.3/lib64/R/lib/atlas -L/t/shared/szip/2.1.1/lib -L/opt/shared/hdf4/4.2.13/lib -L/opt/shared/hdf5/1.10.2/lib -L/opt/shared/netcdf/4.6.1/li-L/opt/shared/sqlite3/3.34.1/lib -L/opt/shared/proj/8.2.1/lib -L/opt/shared/gdal/3.4.3/lib -L/opt/shared/geos/3.9/lib
[(it_css:traine)@login01.caviness ~]$ echo $CPPFLAGS
-I/opt/shared/binutils/2.35/include -I/opt/shared/gcc/11.2.0/include -I/opt/shared/atlas/3.10.3/include -I/opt/shared/r/4.1.3/include -I/opt/shared/r/4.1.3/lib64/R/include -I/opt/shared/szip/2.1.1/include -I/opt/shared/hdf4/4.2.13/include -I/opt/shared/hdf5/1.10.2/include -I/opt/shared/netcdf/4.6.1/include -I/opt/shared/sqlite3/3.34.1/include -I/opt/shared/proj/8.2.1/include -I/opt/shared/gdal/3.4.3/include -I/opt/shared/geos/3.9.1/include
[(it_css:traine)@login01.caviness ~]$ echo $LIBRARY_PATH
/opt/shared/geos/3.9.1/lib:/opt/shared/gdal/3.4.3/lib:/opt/shared/proj/8.2.1/lib:/opt/shared/sqlite3/3.34.1/lib:/opt/shared/netcdf/4.6.1/lib:/opt/shared/hdf5/1.10.2/lib:/opt/shared/hdf4/4.2.13/lib:/opt/shared/szip/2.1.1/lib:/opt/shared/r/4.1.3/lib64/R/lib/atlas:/opt/shared/r/4.1.3/lib64/R/lib:/opt/shared/r/4.1.3/lib64:/opt/shared/atlas/3.10.3/lib:/opt/shared/gcc/11.2.0/lib64:/opt/shared/gcc/11.2.0/lib:/opt/shared/binutils/2.35/lib
[(it_css:traine)@login01.caviness ~]$ echo $LD_LIBRARY_PATH
/opt/shared/geos/3.9.1/lib:/opt/shared/gdal/3.4.3/lib:/opt/shared/proj/8.2.1/lib:/opt/shared/sqlite3/3.34.1/lib:/opt/shared/netcdf/4.6.1/lib:/opt/shared/hdf5/1.10.2/lib:/opt/shared/hdf4/4.2.13/lib:/opt/shared/szip/2.1.1/lib:/opt/shared/r/4.1.3/lib64/R/lib/atlas:/opt/shared/r/4.1.3/lib64/R/lib:/opt/shared/r/4.1.3/lib64:/opt/shared/atlas/3.10.3/lib:/opt/shared/gcc/11.2.0/lib64:/opt/shared/gcc/11.2.0/lib:/opt/shared/binutils/2.35/lib:/opt/shared/slurm/lib
[(it_css:traine)@login01.caviness ~]$ env | grep PREFIX
R_PREFIX=/opt/shared/r/4.1.3
GDAL_PREFIX=/opt/shared/gdal/3.4.3
GCC_PREFIX=/opt/shared/gcc/11.2.0
GEOS_PREFIX=/opt/shared/geos/3.9.1
HDF5_PREFIX=/opt/shared/hdf5/1.10.2
SQLITE3_PREFIX=/opt/shared/sqlite3/3.34.1
PROJ_PREFIX=/opt/shared/proj/8.2.1
NETCDF_PREFIX=/opt/shared/netcdf/4.6.1
GQUEUE_PREFIX=/opt/shared/gqueue
HDF4_PREFIX=/opt/shared/hdf4/4.2.13
BINUTILS_PREFIX=/opt/shared/binutils/2.35
SZIP_PREFIX=/opt/shared/szip/2.1.1
ATLAS_PREFIX=/opt/shared/atlas/3.10.3
[(it_css:traine)@login01.caviness ~]$ mkdir -p $WORKDIR/sw/r/add-ons/r4.1.3/terra
[(it_css:traine)@login01.caviness ~]$ export R_LIBS="$WORKDIR/sw/r/add-ons/r4.1.3/terra"
[(it_css:traine)@login01.caviness ~]$ R -q --no-save
> .libPaths()
[1] "/work/it_css/sw/r/add-ons/r4.1.3/terra"
[2] "/opt/shared/r/4.1.3/lib64/R/library"
> chooseCRANmirror(all)
Secure CRAN mirrors

 1: 0-Cloud [https]
 2: Australia (Canberra) [https]
 3: Australia (Melbourne 1) [https]
 4: Australia (Melbourne 2) [https]
 5: Australia (Perth) [https]
 6: Austria [https]
 ...
69: USA (IA) [https]
70: USA (MI) [https]
71: USA (MO) [https]
72: USA (OH) [https]
73: USA (OR) [https]
74: USA (TN) [https]
75: United Arab Emirates [https]
76: Uruguay [https]
77: (other mirrors)

Selection: 69
> install.packages("quantregForest", dependencies=TRUE)
Installing package into ‘/work/it_css/sw/r/add-ons/r4.1.3/terra’
(as ‘lib’ is unspecified)
also installing the dependencies ‘fs’, ‘R6’, ‘rappdirs’, ‘base64enc’, ‘cachem’, ‘lifecycle’, ‘memoise’, ‘mime’, ‘rlang’, ‘sass’, ‘digest’, ‘ellipsis’, ‘fastmap’, ‘cli’, ‘glue’, ‘magrittr’, ‘stringi’, ‘vctrs’, ‘evaluate’, ‘highr’, ‘xfun’, ‘yaml’, ‘bslib’, ‘fontawesome’, ‘htmltools’, ‘jquerylib’, ‘jsonlite’, ‘stringr’, ‘tinytex’, ‘randomForest’, ‘RColorBrewer’, ‘gss’, ‘knitr’, ‘rmarkdown’

trying URL 'https://mirror.las.iastate.edu/CRAN/src/contrib/fs_1.6.3.tar.gz'
Content type 'application/x-gzip' length 1185603 bytes (1.1 MB)
==================================================
downloaded 1.1 MB
...
...
...
ERROR: dependency ‘bslib’ is not available for package ‘rmarkdown’
* removing ‘/work/it_css/sw/r/add-ons/r4.1.3/terra/rmarkdown’

The downloaded source packages are in
        ‘/tmp/RtmpaBulVZ/downloaded_packages’
Warning message:
In install.packages("quantregForest", dependencies = TRUE) :
  installation of package ‘rmarkdown’ had non-zero exit status
> library (quantregForest)
Loading required package: randomForest
randomForest 4.7-1.1
Type rfNews() to see new features/changes/bug fixes.
Loading required package: RColorBrewer
> install.packages("bslib", dependencies=TRUE)
Installing package into ‘/work/it_css/sw/r/add-ons/r4.1.3/terra’
(as ‘lib’ is unspecified)
also installing the dependencies ‘colorspace’, ‘utf8’, ‘prettyunits’, ‘labeling’, ‘munsell’, ‘viridisLite’, ‘fansi’, ‘pillar’, ‘pkgconfig’, ‘Rcpp’, ‘rprojroot’, ‘pkgbuild’, ‘diffobj’, ‘rematch2’, ‘gtable’, ‘isoband’, ‘scales’, ‘tibble’, ‘httpuv’, ‘xtable’, ‘sourcetools’, ‘later’, ‘promises’, ‘crayon’, ‘commonmark’, ‘brio’, ‘callr’, ‘desc’, ‘pkgload’, ‘praise’, ‘processx’, ‘ps’, ‘waldo’, ‘farver’, ‘rstudioapi’, ‘bsicons’, ‘curl’, ‘ggplot2’, ‘rmarkdown’, ‘shiny’, ‘testthat’, ‘thematic’, ‘withr’
...
...
checking for udunits2.h... no
checking for udunits2/udunits2.h... no
checking for ut_read_xml in -ludunits2... yes
configure: error: in `/tmp/Rtmpc5hPQz/R.INSTALL540e8c2492d/units':
configure: error:
--------------------------------------------------------------------------------
  Configuration failed because udunits2.h was not found. Try installing:
    * deb: libudunits2-dev (Debian, Ubuntu, ...)
    * rpm: udunits2-devel (Fedora, EPEL, ...)
    * brew: udunits (OSX)
  If udunits2 is already installed in a non-standard location, use:
    --configure-args='--with-udunits2-lib=/usr/local/lib'
  if the library was not found, and/or:
    --configure-args='--with-udunits2-include=/usr/include/udunits2'
  if the header was not found, replacing paths with appropriate values.
  You can alternatively set UDUNITS2_INCLUDE and UDUNITS2_LIBS manually.
--------------------------------------------------------------------------------

See `config.log' for more details
ERROR: configuration failed for package ‘units’
* removing ‘/work/it_css/sw/r/add-ons/r4.1.3/terra/units’

The downloaded source packages are in
        ‘/tmp/Rtmps25MZ6/downloaded_packages’
Warning message:
In install.packages("units", dependencies = TRUE) :
  installation of package ‘units’ had non-zero exit status
> quit()
[(it_css:traine)@login01.caviness ~]$ vpkg_devrequire udunits/2.2
Adding package `udunits/2.2.26` to your environment
[(it_css:traine)@login01.caviness ~]$ env | grep UDUNITS
UDUNITS_PREFIX=/opt/shared/udunits/2.2.26
[(it_css:traine)@login01.caviness ~]$ export UDUNITS2_INCLUDE=$UDUNITS_PREFIX/include
[(it_css:traine)@login01.caviness ~]$ export UDUNITS2_LIBS=$UDUNITS_PREFIX/lib
[(it_css:traine)@login01.caviness ~]$ R -q --no-save
> .libPaths()
[1] "/work/it_css/sw/r/add-ons/r4.1.3/terra"
[2] "/opt/shared/r/4.1.3/lib64/R/library"
> chooseCRANmirror(all)
Secure CRAN mirrors

 1: 0-Cloud [https]
 2: Australia (Canberra) [https]
 3: Australia (Melbourne 1) [https]
 4: Australia (Melbourne 2) [https]
 5: Australia (Perth) [https]
 6: Austria [https]
...
...
69: USA (IA) [https]
70: USA (MI) [https]
71: USA (MO) [https]
72: USA (OH) [https]
73: USA (OR) [https]
74: USA (TN) [https]
75: United Arab Emirates [https]
76: Uruguay [https]
77: (other mirrors)

Selection: 69
> install.packages("units", dependencies=TRUE)
Installing package into ‘/work/it_css/sw/r/add-ons/r4.1.3/terra’
(as ‘lib’ is unspecified)
trying URL 'https://mirror.las.iastate.edu/CRAN/src/contrib/units_0.8-4.tar.gz'
Content type 'application/x-gzip' length 248024 bytes (242 KB)
==================================================
downloaded 242 KB

* installing *source* package ‘units’ ...
** package ‘units’ successfully unpacked and MD5 sums checked
** using staged installation
configure: units: 0.8-4
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C++... yes
checking whether /opt/shared/gcc/11.2.0/bin/g++ -std=gnu++14 accepts -g... yes
checking for /opt/shared/gcc/11.2.0/bin/g++ -std=gnu++14 option to enable C++11 features... none needed
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for _Bool... no
checking for stdbool.h that conforms to C99... yes
checking for error_at_line... yes
checking for gcc... /opt/shared/gcc/11.2.0/bin/gcc
checking whether the compiler supports GNU C... yes
checking whether /opt/shared/gcc/11.2.0/bin/gcc accepts -g... yes
checking for /opt/shared/gcc/11.2.0/bin/gcc option to enable C11 features... none needed
checking for XML_ParserCreate in -lexpat... yes
checking for udunits2.h... yes
checking for ut_read_xml in -ludunits2... yes
configure: creating ./config.status
config.status: creating src/Makevars
** libs
/opt/shared/gcc/11.2.0/bin/g++ -std=gnu++14 -I"/opt/shared/r/4.1.3/lib64/R/include" -DNDEBUG -DUDUNITS2_DIR=0 -I/opt/shared/udunits/2.2.26/include -I/opt/shared/udunits/2.2.26/include -I/opt/shared/binutils/2.35/include -I/opt/shared/gcc/11.2.0/include -I/opt/shared/r/4.1.3/include -I'/work/it_css/sw/r/add-ons/r4.1.3/terra/Rcpp/include' -I/opt/shared/binutils/2.35/include -I/opt/shared/gcc/11.2.0/include -I/opt/shared/r/4.1.3/include   -fpic  -g -O2  -c RcppExports.cpp -o RcppExports.o
/opt/shared/gcc/11.2.0/bin/g++ -std=gnu++14 -I"/opt/shared/r/4.1.3/lib64/R/include" -DNDEBUG -DUDUNITS2_DIR=0 -I/opt/shared/udunits/2.2.26/include -I/opt/shared/udunits/2.2.26/include -I/opt/shared/binutils/2.35/include -I/opt/shared/gcc/11.2.0/include -I/opt/shared/r/4.1.3/include -I'/work/it_css/sw/r/add-ons/r4.1.3/terra/Rcpp/include' -I/opt/shared/binutils/2.35/include -I/opt/shared/gcc/11.2.0/include -I/opt/shared/r/4.1.3/include   -fpic  -g -O2  -c udunits.cpp -o udunits.o
/opt/shared/gcc/11.2.0/bin/g++ -std=gnu++14 -shared -L/opt/shared/r/4.1.3/lib64/R/lib -L/opt/shared/binutils/2.35/lib -L/opt/shared/gcc/11.2.0/lib -L/opt/shared/gcc/11.2.0/lib64 -I/opt/shared/r/4.1.3/lib64 -o units.so RcppExports.o udunits.o -lexpat -L/opt/shared/udunits/2.2.26/lib -lexpat -ludunits2 -L/opt/shared/r/4.1.3/lib64/R/lib -lR
installing to /work/it_css/sw/r/add-ons/r4.1.3/terra/00LOCK-units/00new/units/libs
** R
** demo
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (units)
> library(terra)
terra 1.7.55
> library(quantregForest)
Loading required package: randomForest
randomForest 4.7-1.1
Type rfNews() to see new features/changes/bug fixes.
Loading required package: RColorBrewer
> quit()
[(it_css:traine)@login01.caviness ~]$
</code>

=== Using IT's udbuild environment ===
IT developed a formalization for installing modules called [[abstract:caviness:install_software:install_software|udbuild]]
which can simplify the installation of modules.  Here is an example ''udbuild''
script which can be used to install a personal R library.

<file sh udbuild-testing-cuda>
#!/bin/bash -l

PKGNAME=testing
VERSION=default

UDBUILD_HOME=$WORKDIR/sw
PKG_LIST='
	WideLM rpud permGPU magma gputools cudaBayesregData cudaBayesreg
	CARramps
'

vpkg_devrequire udbuild r/3.1.1 r-cran/20140905
init_udbuildenv r-addon cuda/6.5

#Sometimes R doesn't properly use CPPFLAGS which is set by VALET, fix that here:
CPATH=$CUDA_PREFIX/include:$CPATH
LIBRARY_PATH=$CUDA_PREFIX/lib64:$CUDA_PREFIX/lib64/stubs:$LIBRARY_PATH

#CRAN_MIRROR='http://cran.cs.wwu.edu/'
CRAN_MIRROR='http://lib.stat.cmu.edu/R/CRAN/'

quote() { printf '"%s", ' "$@" | sed 's/, $/\n/'; }

R -q --no-save <<EOT
	.libPaths()
	options(repos=structure(c(CRAN="$CRAN_MIRROR")))
	for ( pkg in c( `quote $PKG_LIST` ) ) {
		print(pkg)
		install.packages(pkg, dependencies=TRUE)
	}

	warnings()
EOT
</file>

This script will attempt to build the cuda capable R modules using the
cuda 6.5 version into ''$WORKDIR/sw/r/add-ons/r3.1.1/testing/default-cuda-6.5''.

====== R script in batch ======

==== matmul.R script ====

Consider the simple R script file to multiply a small 3x3 matrix

<file R matmul.R>
# Calculate and print small matrix AA'
a <- matrix(1:12,3,4);
a%*%t(a)
</file>

Let's test this R script using ''Rscript'' from the command line on a compute node.  Don't forget to set your [[abstract:caviness:app_dev:compute_env#using-workgroup-and-directories|workgroup]] to define your cluster group or //investing-entity// compute nodes before you use ''salloc'' to get on a compute node. For example,

<code bash>
workgroup -g it_css
salloc
vpkg_require r/3.5
Rscript matmul.R
</code>

The output to the screen:

<code>
     [,1] [,2] [,3]
[1,]  166  188  210
[2,]  188  214  240
[3,]  210  240  270
</code>

To return to the head node, type
<code bash>
exit
</code>

==== matmul.qs file ====

To run a R script in batch instead of on the command line has nearly the same steps. Copy a template job submission script (''/opt/shared/templates/slurm/generic/threads.qs'') for example and call it ''matmul.qs''.  Now edit it to change the job name and add your commands for your job something like this:

<file bash matmul.qs>
#!/bin/bash -l
#
....

#SBATCH --job-name=matmultiply_R

...
#
# [EDIT] Execute your OpenMP/threaded program using the srun command:
#
# Add vpkg_require commands
vpkg_require r/3.5

# Syntax: Rscript [options] filename.R [arguments]
Rscript matmul.R 
</file>

Now to run the R script simply submit the job from the head node with the
''sbatch'' command.

<code>
sbatch matmul.qs
</code>

You should see a notification that your job was submitted.  Something like this

<code bash>
Submitted batch job 983119
</code>

After the code completes the output of the script will appear in the file
''slurm-983119.out'' because the job number is 983119. Type 

<code>
more slurm-983119.out
</code>

to display the contents of the output file on the screen.  For example,

<code>
-- OpenMP job setup complete:
--  OMP_THREAD_LIMIT     = 2
--  OMP_PROC_BIND        = true
--  OMP_PLACES           = cores
--  MP_BLIST             = 5,17

Adding package `r/3.5.1` to your environment
     [,1] [,2] [,3]
[1,]  166  188  210
[2,]  188  214  240
[3,]  210  240  270
</code>

====== Using R script in batch array job ======
===== sweep.R file =====

Consider the simple script to print a fraction from the argument list

<file R sweep.R>
args <- commandArgs(trailingOnly = TRUE)
# print fraction from argument list 
as.numeric(args[1])/as.numeric(args[2])
</file>

This is a R script which can be run from the command line on a compute node the commands 

<code bash>
salloc
vpkg_require r/3.5
Rscript sweep.R 5 200
</code>

The output to the screen:
<code>
[1] 0.025
</code>

===== sweep.qs file =====

Again copy a template job submission script (/opt/shared/templates/slurm/generic/threads.qs) for example and call it ''sweep.qs''. Now edit it to change the job name, this time adding options for an array job and add your commands for your job something like this:

<file bash sweep.qs>
#!/bin/bash -l
#
....

#SBATCH --job-name=sweep_R
#SBATCH --array=1-200

...
#
# [EDIT] Execute your OpenMP/threaded program using the srun command:
#
## Parameter sweep array job to run the sweep.R  with
##    lambda = 0,1,2. ... 199
##
# Add vpkg_require commands
vpkg_require r/3.5

date "+Start %s"
echo "Host $HOSTNAME"

let lambda="$SLURM_ARRAY_TASK_ID-1"
let taskCount=200

# Syntax: Rscript [options] filename.R [arguments]
Rscript --vanilla sweep.R $lambda $taskCount

date "+Finish %s"
</file>

The ''date'' and ''echo Host'' lines are just a way of keeping track of when and where the jobs are run.
There will be 200 array jobs all running the same script with different parameters (arguments).  The ''--vanilla'' option
is used to prevent the multiple jobs from using the same disk space.

To run this in batch you must submit the job from the head node with the
''sbatch'' command.

<code>
sbatch sweep.qs
</code>

And you see the notification of the job submitted, like this:

<code>
Submitted batch job 1170728
</code>

After the code completes the output of the script will appear in the files
''slurm-1170728_1.out'' to ''slurm-1170728_200.out''. The number ''1170728'' is the job ID assigned to your job when submitted, and 1 to 200 is the Task ID (e.g. corresponds to the ''--array=1-200'')

If we look specifically at the array job output that maps to our previous example using ''5 200'' which would be ''slurm-1170728_6.out'' we see similar output 
<code>
-- OpenMP job setup complete:
--  OMP_THREAD_LIMIT     = 2
--  OMP_PROC_BIND        = true
--  OMP_PLACES           = cores
--  MP_BLIST             = 30,31

Adding package `r/3.5.1` to your environment
Start 1567531210
Host r00n15.localdomain.hpc.udel.edu
[1] 0.025
Finish 1567531210
</code>
<note tip>
You will want to do more than just print out one fraction in your script.  The integer parameter can be used for
a one dimensional parameter sweep, to construct unique input and output file names for each task, 
or as a seed for the R Random Number Generator (RNG).</note>

==== Writing files from an array job ====

You are running many jobs in the same directory.  Grid engine handles the standard output by writing to
separate files with "dot taskid" appended to the jobid.  You need to take care of other file output in your R script.

<note important>
You need to make sure no two of your jobs will write to the same file.  Look at your R script to see if you
are writing files.  Look for the ''**sink**'' command or any graphics writing commands such as ''**pdf**'' or ''**png**''.
If you are using these R functions, then use a unique file name constructed from the task id.
</note>

==== vanilla option ====

The command-line option ''--vanilla'' implies --no-site-file, --no-init-file and --no-environ.  This way you will not
be reading or writing to the same files.  If you need initialization command, put them in your R script instead of in
in the init-file ''.Rprofile''.  If you need some environment variables, export them in your bash script instead of assigning
them in your environ file ''.Renviron''.