Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
technical:whitepaper:r-runtime-blas-lapack [2018-12-10 11:23] – frey | technical:whitepaper:r-runtime-blas-lapack [2018-12-10 12:40] (current) – [R: Runtime-configuration BLAS/LAPACK] frey | ||
---|---|---|---|
Line 8: | Line 8: | ||
* can require on the order of //N// times the disk space of a single build | * can require on the order of //N// times the disk space of a single build | ||
* puts a greater burden on the sysadmin to maintain all //N// similarly-outfitted copies | * puts a greater burden on the sysadmin to maintain all //N// similarly-outfitted copies | ||
- | - R only makes use of standardized BLAS/LAPACK APIs, so any standard BLAS/LAPACK library should be able to be chosen at runtime (not just build time)/ | + | - R only makes use of standardized BLAS/LAPACK APIs, so any standard BLAS/LAPACK library should be able to be chosen at runtime (not just build time). |
===== Substituting an alternate library ===== | ===== Substituting an alternate library ===== | ||
Line 16: | Line 16: | ||
* [[https:// | * [[https:// | ||
The basic idea is: | The basic idea is: | ||
- | * copy '' | + | * copy '' |
- | * remove '' | + | * remove '' |
- | * symlink '' | + | * symlink '' |
- | This copy of R is configured to use '' | + | This copy of R is configured to use '' |
+ | |||
+ | This scheme requires two things: | ||
+ | - the user must have ownership of the R installation or sufficient privileges to alter the files | ||
+ | - the BLAS/LAPACK substitution will happen on time only (probably shortly after the library is built) | ||
+ | While the first condition is obvious, the second may not seem important, especially for a build of R being maintained by an arbitrary user in an arbitrary location on the filesystem. | ||
+ | * only a single choice of underlying BLAS/LAPACK can be active | ||
+ | * the underlying BLAS/LAPACK can be changed only when that build of R is not being executed/ | ||
+ | |||
+ | A simple way to organize multiple underlying BLAS/LAPACK libraries in a single R installation is to create subdirectories for each variant: | ||
+ | |||
+ | ^Path ^Description^ | ||
+ | |'' | ||
+ | |'' | ||
+ | |'' | ||
+ | |'' | ||
+ | |'' | ||
+ | |'' | ||
+ | |'' | ||
+ | |'' | ||
+ | |||
+ | ==== R BLAS/LAPACK ==== | ||
+ | |||
+ | When we restructured the R '' | ||
+ | |||
+ | <code bash> | ||
+ | $ cd ${R_HOME}/ | ||
+ | $ rm -f libR{blas, | ||
+ | $ ln -s rblas/ | ||
+ | $ ln -s rblas/ | ||
+ | </ | ||
+ | |||
+ | ==== ATLAS ==== | ||
+ | |||
+ | The ATLAS library contains both BLAS and LAPACK APIs in a single shared library. | ||
+ | |||
+ | <code bash> | ||
+ | $ cd ${R_HOME}/ | ||
+ | $ rm -f libR{blas, | ||
+ | $ ln -s atlas/ | ||
+ | $ ln -s atlas/ | ||
+ | </ | ||
+ | |||
+ | ==== Sequential MKL ==== | ||
+ | |||
+ | A C source file containing a dummy function was created in '' | ||
+ | |||
+ | <file C shim.c> | ||
+ | int | ||
+ | mkl_shim_dummy(void) | ||
+ | { | ||
+ | return 0; | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | The shim library is then created thusly: | ||
+ | |||
+ | <code bash> | ||
+ | $ cd ${R_HOME}/ | ||
+ | $ icc -shared -o libRblas.so -mkl=sequential shim.c | ||
+ | $ ln -s libRblas.so libRlapack.so | ||
+ | </ | ||
+ | |||
+ | To configure R to use the sequential MKL: | ||
+ | |||
+ | <code bash> | ||
+ | $ cd ${R_HOME}/ | ||
+ | $ rm -f libR{blas, | ||
+ | $ ln -s mkl/ | ||
+ | $ ln -s mkl/ | ||
+ | </ | ||
+ | |||
+ | ==== Threaded MKL ==== | ||
+ | |||
+ | A C source file containing a dummy function was created in '' | ||
+ | |||
+ | <file C shim.c> | ||
+ | int | ||
+ | mkl_shim_dummy(void) | ||
+ | { | ||
+ | return 0; | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | Since our R build used the GNU C compiler, the threaded MKL variant only works if the shim library is built against the GNU OpenMP runtime. | ||
+ | |||
+ | <code bash> | ||
+ | $ cd ${R_HOME}/ | ||
+ | $ icc -shared -o libRblas.so shim.c -lmkl_gnu_thread -lmkl_core -lmkl_intel_lp64 | ||
+ | $ ln -s libRblas.so libRlapack.so | ||
+ | </ | ||
+ | |||
+ | To configure R to use the threaded MKL: | ||
+ | |||
+ | <code bash> | ||
+ | $ cd ${R_HOME}/ | ||
+ | $ rm -f libR{blas, | ||
+ | $ ln -s mkl/ | ||
+ | $ ln -s mkl/ | ||
+ | </ | ||
+ | |||
+ | ===== Runtime-configurable substitution ===== | ||
+ | |||
+ | By stashing each BLAS/LAPACK variant in its own subdirectory, | ||
+ | |||
+ | On our Caviness cluster we include no BLAS/LAPACK library symlinks in the base directory which R checks for shared libraries: | ||
+ | |||
+ | <code bash> | ||
+ | $ cd ${R_HOME}/ | ||
+ | $ rm -f libR{blas, | ||
+ | </ | ||
+ | |||
+ | In our VALET package definition for R we configure four variants that differ by BLAS/LAPACK (as a feature tag) | ||
+ | |||
+ | <code bash> | ||
+ | $ vpkg_versions r | ||
+ | : | ||
+ | r The R Project for Statistical Computing | ||
+ | 3.5 alias to r/3.5.1 | ||
+ | * 3.5.1 R 3.5.1 with system compilers, ATLAS | ||
+ | 3.5.1: | ||
+ | 3.5.1: | ||
+ | 3.5.1: | ||
+ | </ | ||
+ | |||
+ | with ATLAS as the default (recommended) choice of underlying BLAS/ | ||
+ | |||
+ | <code bash> | ||
+ | $ vpkg_require r/3.5.1 | ||
+ | Adding package `r/3.5.1` to your environment | ||
+ | $ echo $LD_LIBRARY_PATH | ||
+ | / | ||
+ | $ vpkg_rollback all | ||
+ | $ vpkg_require r/ | ||
+ | Adding package `r/ | ||
+ | $ echo $LD_LIBRARY_PATH | ||
+ | / | ||
+ | </ | ||
+ | |||
+ | This works fine so long as you don't attempt to install any R modules that require the BLAS/LAPACK functionality. | ||
+ | |||
+ | < | ||
+ | BLAS_LIBS = -L" | ||
+ | </ | ||
+ | |||
+ | and | ||
+ | |||
+ | < | ||
+ | LAPACK_LIBS = -L" | ||
+ | </ | ||
+ | |||
+ | The link search added here will default to the usual path if '' | ||
+ | |||
+ | <code bash> | ||
+ | $ vpkg_require r/ | ||
+ | Adding package `r/ | ||
+ | $ echo $R_BLAS_VARIANT | ||
+ | rblas | ||
+ | $ vpkg_rollback all | ||
+ | $ vpkg_require r/ | ||
+ | Adding package `r/ | ||
+ | $ echo $R_BLAS_VARIANT | ||
+ | mkl/seq | ||
+ | </ | ||
+ | |||
+ | We now have a runtime-configurable BLAS/LAPACK for this single installation of R, and any properly-packaged R modules should build fine against it. | ||
+ | |||
+ | ===== VALET configuration ===== | ||
+ | |||
+ | Here is the VALET configuration for our installation of R 3.5.1 with runtime-configurable BLAS/ | ||
+ | |||
+ | <file yaml> | ||
+ | r: | ||
+ | description: | ||
+ | url: | ||
+ | prefix: | ||
+ | |||
+ | default-version: | ||
+ | |||
+ | versions: | ||
+ | " | ||
+ | alias-to: | ||
+ | " | ||
+ | description: | ||
+ | actions: | ||
+ | - libdir: | ||
+ | - incdir: | ||
+ | - libdir: | ||
+ | - variable: | ||
+ | operator: | ||
+ | value: | ||
+ | |||
+ | " | ||
+ | description: | ||
+ | prefix: | ||
+ | actions: | ||
+ | - libdir: | ||
+ | - incdir: | ||
+ | - libdir: | ||
+ | - variable: | ||
+ | operator: | ||
+ | value: | ||
+ | |||
+ | " | ||
+ | description: | ||
+ | prefix: | ||
+ | actions: | ||
+ | - libdir: | ||
+ | - incdir: | ||
+ | - libdir: | ||
+ | - variable: | ||
+ | operator: | ||
+ | value: | ||
+ | |||
+ | " | ||
+ | description: | ||
+ | prefix: | ||
+ | actions: | ||
+ | - libdir: | ||
+ | - incdir: | ||
+ | - libdir: | ||
+ | - variable: | ||
+ | operator: | ||
+ | value: | ||
+ | </ |