Both sides previous revision Previous revision Next revision | Previous revisionLast revisionBoth sides next revision |
technical:whitepaper:r-runtime-blas-lapack [2018-12-10 12:31] – [Runtime-configurable substitution] frey | technical:whitepaper:r-runtime-blas-lapack [2018-12-10 12:39] – [VALET configuration] frey |
---|
* [[https://stackoverflow.com/questions/29984141/does-installing-blas-atlas-mkl-openblas-will-speed-up-r-package-that-is-written]] | * [[https://stackoverflow.com/questions/29984141/does-installing-blas-atlas-mkl-openblas-will-speed-up-r-package-that-is-written]] |
The basic idea is: | The basic idea is: |
* copy ''libatlas.so'' to ''**R_PREFIX**/lib64/R/lib'' | * copy ''libatlas.so'' to ''** R_HOME**/lib64/R/lib'' |
* remove ''libRblas.so'' and ''libRlapack.so'' from ''**R_PREFIX**/lib64/R/lib'' | * remove ''libRblas.so'' and ''libRlapack.so'' from ''** R_HOME**/lib64/R/lib'' |
* symlink ''libRblas.so'' and ''libRlapack.so'' to ''libatlas.so'' in ''**R_PREFIX**/lib64/R/lib'' | * symlink ''libRblas.so'' and ''libRlapack.so'' to ''libatlas.so'' in ''** R_HOME**/lib64/R/lib'' |
This copy of R is configured to use ''**R_PREFIX**/lib64/R/lib'' to resolve shared libraries, so when executing the ''R'' command, for example, the symlinks will lead the runtime linker to the ATLAS library when resolving BLAS/LAPACK functions. | This copy of R is configured to use ''** R_HOME**/lib64/R/lib'' to resolve shared libraries, so when executing the ''R'' command, for example, the symlinks will lead the runtime linker to the ATLAS library when resolving BLAS/LAPACK functions. |
| |
This scheme requires two things: | This scheme requires two things: |
| |
^Path ^Description^ | ^Path ^Description^ |
|''**R_PREFIX**/lib64/R/lib'' |base directory where R looks for shared libraries by default| | |''** R_HOME**/lib64/R/lib'' |base directory where R looks for shared libraries by default| |
|''**R_PREFIX**/lib64/R/lib/libRblas.so'' |symlink to chosen BLAS library (from one of the subdirectories herein)| | |''** R_HOME**/lib64/R/lib/libRblas.so'' |symlink to chosen BLAS library (from one of the subdirectories herein)| |
|''**R_PREFIX**/lib64/R/lib/libRlapack.so'' |symlink to chosen LAPACK library (from one of the subdirectories herein)| | |''** R_HOME**/lib64/R/lib/libRlapack.so'' |symlink to chosen LAPACK library (from one of the subdirectories herein)| |
|''**R_PREFIX**/lib64/R/lib/atlas'' |directory to hold ''libatlas.so''| | |''** R_HOME**/lib64/R/lib/atlas'' |directory to hold ''libatlas.so''| |
|''**R_PREFIX**/lib64/R/lib/rblas'' |directory to hold the bundled ''libRblas.so'' and ''libRlapack.so'' produced by R build procedure| | |''** R_HOME**/lib64/R/lib/rblas'' |directory to hold the bundled ''libRblas.so'' and ''libRlapack.so'' produced by R build procedure| |
|''**R_PREFIX**/lib64/R/lib/mkl'' |directory to hold MKL variants| | |''** R_HOME**/lib64/R/lib/mkl'' |directory to hold MKL variants| |
|''**R_PREFIX**/lib64/R/lib/mkl/seq'' |directory to hold sequential MKL variant| | |''** R_HOME**/lib64/R/lib/mkl/seq'' |directory to hold sequential MKL variant| |
|''**R_PREFIX**/lib64/R/lib/mkl/thr'' |directory to hold threaded MKL variant| | |''** R_HOME**/lib64/R/lib/mkl/thr'' |directory to hold threaded MKL variant| |
| |
==== R BLAS/LAPACK ==== | ==== R BLAS/LAPACK ==== |
| |
<code bash> | <code bash> |
$ cd ${R_PREFIX}/lib64/R/lib | $ cd ${R_HOME}/lib64/R/lib |
$ rm -f libR{blas,lapack}.so | $ rm -f libR{blas,lapack}.so |
$ ln -s rblas/libRblas.so . | $ ln -s rblas/libRblas.so . |
==== ATLAS ==== | ==== ATLAS ==== |
| |
The ATLAS library contains both BLAS and LAPACK APIs in a single shared library. With ''libatlas.so'' copied into the ''**R_PREFIX**/lib64/R/lib/atlas'' subdirectory, we configure R to use ATLAS: | The ATLAS library contains both BLAS and LAPACK APIs in a single shared library. With ''libatlas.so'' copied into the ''** R_HOME**/lib64/R/lib/atlas'' subdirectory, we configure R to use ATLAS: |
| |
<code bash> | <code bash> |
$ cd ${R_PREFIX}/lib64/R/lib | $ cd ${R_HOME}/lib64/R/lib |
$ rm -f libR{blas,lapack}.so | $ rm -f libR{blas,lapack}.so |
$ ln -s atlas/libRblas.so . | $ ln -s atlas/libRblas.so . |
==== Sequential MKL ==== | ==== Sequential MKL ==== |
| |
A C source file containing a dummy function was created in ''**R_PREFIX**/lib64/R/lib/mkl/seq'': | A C source file containing a dummy function was created in ''** R_HOME**/lib64/R/lib/mkl/seq'': |
| |
<file C shim.c> | <file C shim.c> |
| |
<code bash> | <code bash> |
$ cd ${R_PREFIX}/lib64/R/lib/mkl/seq | $ cd ${R_HOME}/lib64/R/lib/mkl/seq |
$ icc -shared -o libRblas.so -mkl=sequential shim.c | $ icc -shared -o libRblas.so -mkl=sequential shim.c |
$ ln -s libRblas.so libRlapack.so | $ ln -s libRblas.so libRlapack.so |
| |
<code bash> | <code bash> |
$ cd ${R_PREFIX}/lib64/R/lib | $ cd ${R_HOME}/lib64/R/lib |
$ rm -f libR{blas,lapack}.so | $ rm -f libR{blas,lapack}.so |
$ ln -s mkl/seq/libRblas.so . | $ ln -s mkl/seq/libRblas.so . |
==== Threaded MKL ==== | ==== Threaded MKL ==== |
| |
A C source file containing a dummy function was created in ''**R_PREFIX**/lib64/R/lib/mkl/thr'': | A C source file containing a dummy function was created in ''** R_HOME**/lib64/R/lib/mkl/thr'': |
| |
<file C shim.c> | <file C shim.c> |
| |
<code bash> | <code bash> |
$ cd ${R_PREFIX}/lib64/R/lib/mkl/thr | $ cd ${R_HOME}/lib64/R/lib/mkl/thr |
$ icc -shared -o libRblas.so shim.c -lmkl_gnu_thread -lmkl_core -lmkl_intel_lp64 | $ icc -shared -o libRblas.so shim.c -lmkl_gnu_thread -lmkl_core -lmkl_intel_lp64 |
$ ln -s libRblas.so libRlapack.so | $ ln -s libRblas.so libRlapack.so |
| |
<code bash> | <code bash> |
$ cd ${R_PREFIX}/lib64/R/lib | $ cd ${R_HOME}/lib64/R/lib |
$ rm -f libR{blas,lapack}.so | $ rm -f libR{blas,lapack}.so |
$ ln -s mkl/thr/libRblas.so . | $ ln -s mkl/thr/libRblas.so . |
===== Runtime-configurable substitution ===== | ===== Runtime-configurable substitution ===== |
| |
By stashing each BLAS/LAPACK variant in its own subdirectory, our copy of R is actually fairly close to being runtime-configurable with respect to choice of BLAS/LAPACK. Since all R commands will setup the environment to have the runtime linker check ''**R_PREFIX**/lib64/R/lib'' for shared libraries, the ''libRblas.so'' and ''libRlapack.so'' symlinks in that directory will always have priority over any other path we might add to ''LD_LIBRARY_PATH'' prior to issuing the ''R'' command, for example. However, if ''libRblas.so'' and ''libRlapack.so'' are **not** present in that directory, the runtime linker will be forced to consult other paths present in ''LD_LIBRARY_PATH''. | By stashing each BLAS/LAPACK variant in its own subdirectory, our copy of R is actually fairly close to being runtime-configurable with respect to choice of BLAS/LAPACK. Since all R commands will setup the environment to have the runtime linker check ''** R_HOME**/lib64/R/lib'' for shared libraries, the ''libRblas.so'' and ''libRlapack.so'' symlinks in that directory will always have priority over any other path we might add to ''LD_LIBRARY_PATH'' prior to issuing the ''R'' command, for example. However, if ''libRblas.so'' and ''libRlapack.so'' are **not** present in that directory, the runtime linker will be forced to consult other paths present in ''LD_LIBRARY_PATH''. |
| |
On our Caviness cluster we include no BLAS/LAPACK library symlinks in the base directory which R checks for shared libraries: | On our Caviness cluster we include no BLAS/LAPACK library symlinks in the base directory which R checks for shared libraries: |
| |
<code bash> | <code bash> |
$ cd ${R_PREFIX}/lib64/R/lib | $ cd ${R_HOME}/lib64/R/lib |
$ rm -f libR{blas,lapack}.so | $ rm -f libR{blas,lapack}.so |
</code> | </code> |
</code> | </code> |
| |
This works fine so long as you don't attempt to install any R modules that require the BLAS/LAPACK functionality. The R module-building environment defaults to using link search paths in the base directory (''**R_PREFIX**/lib64/R/lib'') and fails to find ''libRblas.so'' or ''libRlapack.so''. This is easily fixed by altering two lines in the R installation's standard make flags file at ''**R_PREFIX**/lib64/R/etc/Makeconf'': | This works fine so long as you don't attempt to install any R modules that require the BLAS/LAPACK functionality. The R module-building environment defaults to using link search paths in the base directory (''** R_HOME**/lib64/R/lib'') and fails to find ''libRblas.so'' or ''libRlapack.so''. This is easily fixed by altering two lines in the R installation's standard make flags file at ''** R_HOME**/lib64/R/etc/Makeconf'': |
| |
<code> | <code> |
LAPACK_LIBS = -L"$(R_HOME)/lib$(R_ARCH)/$(R_BLAS_VARIANT)" -lRlapack | LAPACK_LIBS = -L"$(R_HOME)/lib$(R_ARCH)/$(R_BLAS_VARIANT)" -lRlapack |
</code> | </code> |
| |
| The link search added here will default to the usual path if ''R_BLAS_VARIANT'' is not set in the user's environment. But in our VALET configuration of each of the variants, we set the appropriate relative path for ''R_BLAS_VARIANT'': |
| |
| <code bash> |
| $ vpkg_require r/3.5.1:rblas |
| Adding package `r/3.5.1:rblas` to your environment |
| $ echo $R_BLAS_VARIANT |
| rblas |
| $ vpkg_rollback all |
| $ vpkg_require r/3.5.1:mkl-seq |
| Adding package `r/3.5.1:mkl-seq` to your environment |
| $ echo $R_BLAS_VARIANT |
| mkl/seq |
| </code> |
| |
| We now have a runtime-configurable BLAS/LAPACK for this single installation of R, and any properly-packaged R modules should build fine against it. |
| |
| ===== VALET configuration ===== |
| |
| Here is the VALET configuration for our installation of R 3.5.1 with runtime-configurable BLAS/LAPACK: |
| |
| <file yaml> |
| r: |
| description: The R Project for Statistical Computing |
| url: http://www.r-project.org/ |
| prefix: /opt/shared/r |
| |
| default-version: "3.5.1" |
| |
| versions: |
| "3.5": |
| alias-to: 3.5.1 |
| "3.5.1": |
| description: R 3.5.1 with system compilers, ATLAS |
| actions: |
| - libdir: lib64/R/lib |
| - incdir: lib64/R/include |
| - libdir: lib64/R/lib/atlas |
| - variable: R_BLAS_VARIANT |
| operator: set |
| value: atlas |
| |
| "3.5.1:rblas": |
| description: R 3.5.1 with system compilers, R reference BLAS/LAPACK |
| prefix: 3.5.1 |
| actions: |
| - libdir: lib64/R/lib |
| - incdir: lib64/R/include |
| - libdir: lib64/R/lib/rblas |
| - variable: R_BLAS_VARIANT |
| operator: set |
| value: rblas |
| |
| "3.5.1:mkl-seq": |
| description: R 3.5.1 with system compilers, MKL (sequential) |
| prefix: 3.5.1 |
| actions: |
| - libdir: lib64/R/lib |
| - incdir: lib64/R/include |
| - libdir: lib64/R/lib/mkl/seq |
| - variable: R_BLAS_VARIANT |
| operator: set |
| value: mkl/seq |
| |
| "3.5.1:mkl-thr": |
| description: R 3.5.1 with system compilers, MKL (multithread) |
| prefix: 3.5.1 |
| actions: |
| - libdir: lib64/R/lib |
| - incdir: lib64/R/include |
| - libdir: lib64/R/lib/mkl/thr |
| - variable: R_BLAS_VARIANT |
| operator: set |
| value: mkl/thr |
| </file> |