Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
technical:recipes:vasp-wannier [2023-03-13 21:28] – [Building VASP] frey | technical:recipes:vasp-wannier [2023-03-14 15:17] (current) – anita | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Building VASP with Wannier90 Support ====== | ||
+ | The VASP electronic structure code can be extended with the Wannier90 library. | ||
+ | |||
+ | ===== General Procedure ===== | ||
+ | |||
+ | With an appropriate compiler toolchain loaded into the environment: | ||
+ | |||
+ | - A package directory is created (if it does not yet exist) and a version directory is created under it | ||
+ | - Wannier90 is built and installed to the version directory | ||
+ | - VASP is built and linked against the installed Wannier90 library | ||
+ | - The resulting VASP executables are copied to the version directory | ||
+ | |||
+ | Naturally, this procedure assumes the source packages for Wannier90 and VASP have already been downloaded. | ||
+ | |||
+ | The build procedure was performed on the DARWIN cluster. | ||
+ | |||
+ | ==== Load a Toolchain ==== | ||
+ | |||
+ | VASP works well with the Intel compiler suite (Fortran and C) and MKL libraries for BLAS, LAPACK, ScaLAPACK, and FFT functions. | ||
+ | |||
+ | Wannier90 must use the same Fortran and MPI compiler that will be used to build VASP. It is also recommended to use the same BLAS/LAPACK library, e.g. MKL. | ||
+ | |||
+ | The MKL comes in two base variants: | ||
+ | |||
+ | To that end, the toolchain is loaded into the environment: | ||
+ | |||
+ | <code bash> | ||
+ | $ vpkg_require openmpi/ | ||
+ | Adding dependency `intel/ | ||
+ | Adding dependency `ucx/ | ||
+ | Adding package `openmpi/ | ||
+ | </ | ||
+ | |||
+ | ==== Creating the Package Directory Hierarchy ==== | ||
+ | |||
+ | On DARWIN every user has a home directory in which software can be compiled and stored. | ||
+ | |||
+ | A directory hierarchy is utilized to organize software packages and versions. | ||
+ | |||
+ | <code bash> | ||
+ | $ VASP_PREFIX=" | ||
+ | $ mkdir -p " | ||
+ | $ mkdir " | ||
+ | </ | ||
+ | |||
+ | The '' | ||
+ | |||
+ | <code bash> | ||
+ | $ pushd " | ||
+ | $ wget " | ||
+ | $ popd | ||
+ | </ | ||
+ | |||
+ | This recipe will be creating a variant of VASP 6.3.2 with Wannier90 3.1.0 functionality, | ||
+ | |||
+ | <code bash> | ||
+ | $ VASP_VERSION=" | ||
+ | $ VASP_INSTALL_PATH=" | ||
+ | $ echo " | ||
+ | / | ||
+ | $ mkdir " | ||
+ | </ | ||
+ | |||
+ | The source code will be colocated in that variant directory, so a '' | ||
+ | |||
+ | <code bash> | ||
+ | $ mkdir " | ||
+ | $ cd " | ||
+ | $ tar -xf ../ | ||
+ | $ tar -xf ../ | ||
+ | $ ls -l | ||
+ | total 36858 | ||
+ | drwxr-xr-x | ||
+ | drwxr-xr-x 12 frey everyone | ||
+ | </ | ||
+ | |||
+ | ==== Building Wannier90 ==== | ||
+ | |||
+ | Next, Wannier90 is built. | ||
+ | |||
+ | <code bash> | ||
+ | $ cd wannier90-3.1.0 | ||
+ | $ cp config/ | ||
+ | </ | ||
+ | |||
+ | The '' | ||
+ | |||
+ | <file make make.inc> | ||
+ | |||
+ | # | ||
+ | # For Linux with intel version 11/12 on 64bit machines | ||
+ | # | ||
+ | F90 = ifort | ||
+ | COMMS=mpi | ||
+ | MPIF90=mpif90 | ||
+ | FCOPTS=-O2 | ||
+ | LDOPTS=-O2 | ||
+ | |||
+ | # | ||
+ | # Intel mkl libraries. Set LIBPATH if not in default path | ||
+ | # | ||
+ | |||
+ | LIBDIR = | ||
+ | LIBS | ||
+ | |||
+ | # | ||
+ | # ATLAS Blas and LAPACK | ||
+ | # | ||
+ | #LIBDIR = / | ||
+ | #LIBS = -L$(LIBDIR) | ||
+ | </ | ||
+ | |||
+ | Note the use of the special Intel compiler flag '' | ||
+ | |||
+ | Build is now accomplished using '' | ||
+ | |||
+ | <code bash> | ||
+ | $ make all | ||
+ | </ | ||
+ | |||
+ | If successful, the completed library and executables can be installed in the VASP installation directory: | ||
+ | |||
+ | <code bash> | ||
+ | $ make PREFIX=" | ||
+ | $ ls -l ../../lib | ||
+ | total 1506 | ||
+ | -rw-r--r-- 1 frey everyone 3004552 Feb 8 11:52 libwannier.a | ||
+ | |||
+ | $ ls -l ../../bin | ||
+ | total 77578 | ||
+ | -rwxr-xr-x 1 frey everyone | ||
+ | -rwxr-xr-x 1 frey everyone | ||
+ | -rwxr-xr-x 1 frey everyone | ||
+ | -rwxr-xr-x 1 frey everyone | ||
+ | -rwxr-xr-x 1 frey everyone | ||
+ | -rwxr-xr-x 1 frey everyone | ||
+ | </ | ||
+ | |||
+ | With the Wannier90 library completed, the VASP build can proceed. | ||
+ | |||
+ | ==== Building VASP ==== | ||
+ | |||
+ | The VASP source code is one directory level up; a customized copy of the Intel OpenMP + MKL + MPI architecture-specific file is required: | ||
+ | |||
+ | <code bash> | ||
+ | $ cd ../ | ||
+ | $ cp arch/ | ||
+ | </ | ||
+ | |||
+ | The following patch can be downloaded to the '' | ||
+ | |||
+ | <file text makefile.include.patch> | ||
+ | --- A/ | ||
+ | +++ B/ | ||
+ | @@ -13,7 +13,7 @@ | ||
+ | | ||
+ | |||
+ | | ||
+ | -FCL = mpif90 | ||
+ | +FCL = mpif90 -qopenmp | ||
+ | |||
+ | | ||
+ | |||
+ | @@ -49,19 +49,19 @@ | ||
+ | |||
+ | # When compiling on the target machine itself, change this to the | ||
+ | # relevant target when cross-compiling for another architecture | ||
+ | -VASP_TARGET_CPU ?= -xHOST | ||
+ | +VASP_TARGET_CPU ?= -xcore-avx2 | ||
+ | | ||
+ | | ||
+ | # Intel MKL for FFTW, BLAS, LAPACK, and scaLAPACK | ||
+ | # (Note: for Intel Parallel Studio' | ||
+ | -FCL += -qmkl | ||
+ | -MKLROOT | ||
+ | +FCL += -mkl=sequential | ||
+ | +MKLROOT | ||
+ | | ||
+ | |||
+ | # Use a separate scaLAPACK installation (optional but recommended in combination with OpenMPI) | ||
+ | # Comment out the two lines below if you want to use scaLAPACK from MKL instead | ||
+ | -SCALAPACK_ROOT ?= / | ||
+ | -LLIBS | ||
+ | +SCALAPACK_ROOT ?= | ||
+ | +LLIBS | ||
+ | |||
+ | # HDF5-support (optional but strongly recommended) | ||
+ | # | ||
+ | @@ -70,9 +70,9 @@ | ||
+ | # | ||
+ | |||
+ | # For the VASP-2-Wannier90 interface (optional) | ||
+ | -# | ||
+ | -# | ||
+ | -# | ||
+ | +CPP_OPTIONS | ||
+ | +WANNIER90_ROOT ?= | ||
+ | +LLIBS | ||
+ | |||
+ | # For the fftlib library (hardly any benefit in combination with MKL's FFTs) | ||
+ | # | ||
+ | </ | ||
+ | |||
+ | and applied to the copy of the stock '' | ||
+ | |||
+ | <code bash> | ||
+ | $ patch -p1 < makefile.include.patch | ||
+ | patching file makefile.include | ||
+ | </ | ||
+ | |||
+ | In summary, the changes embodied in that patch are: | ||
+ | |||
+ | * When the object code is linked using the '' | ||
+ | * The AMD nodes in DARWIN have AVX2 capabilities but the '' | ||
+ | * The '' | ||
+ | * '' | ||
+ | * The Intel MKL ScaLAPACK and BLACS libraries are found under the '' | ||
+ | * The appropriate integer bit-depth libraries (lp64) with BLACS for Open MPI runtimes are needed | ||
+ | * The Wannier90 library doesn' | ||
+ | |||
+ | The build of VASP can now be effected: | ||
+ | |||
+ | <code bash> | ||
+ | $ make WANNIER90_ROOT=" | ||
+ | </ | ||
+ | |||
+ | By default all three variants of VASP are produced -- std, gam, and ncl. If the build is successful, the completed executables can be copied into the installation directory: | ||
+ | |||
+ | <code bash> | ||
+ | $ mkdir " | ||
+ | $ cp bin/* " | ||
+ | </ | ||
+ | |||
+ | At this point all executables associated with Wannier90 and VASP are present in the '' | ||
+ | |||
+ | ==== VALET Package Definition ==== | ||
+ | |||
+ | Since this copy of VASP resides in the user's home directory, the appropriate location to install a VALET package definition for it is '' | ||
+ | |||
+ | Key values to know before editing the package definition were assigned to variables throughout this recipe: | ||
+ | |||
+ | <code bash> | ||
+ | $ for V in VASP_VERSION VASP_PREFIX; | ||
+ | VASP_VERSION = 6.3.2: | ||
+ | VASP_PREFIX = / | ||
+ | |||
+ | $ vpkg_history | ||
+ | [standard] | ||
+ | intel/ | ||
+ | ucx/1.13.1 | ||
+ | openmpi/ | ||
+ | </ | ||
+ | |||
+ | Recall that we added '' | ||
+ | |||
+ | <file yaml vasp.vpkg_yaml> | ||
+ | vasp: | ||
+ | prefix: $VASP_PREFIX | ||
+ | description: | ||
+ | url: http:// | ||
+ | | ||
+ | default-version: | ||
+ | versions: | ||
+ | " | ||
+ | description: | ||
+ | dependencies: | ||
+ | - openmpi/ | ||
+ | </ | ||
+ | |||
+ | VALET can now be used to query versions of VASP available: | ||
+ | |||
+ | <code bash> | ||
+ | $ vpkg_versions vasp | ||
+ | |||
+ | Available versions/ | ||
+ | |||
+ | [/ | ||
+ | vasp | ||
+ | * 6.3.2: | ||
+ | </ | ||
+ | |||
+ | ===== Using VASP ===== | ||
+ | |||
+ | Henceforth, this variant of VASP 6.3.2 can be added to your shell environment using VALET: | ||
+ | |||
+ | <code bash> | ||
+ | $ vpkg_require vasp/ | ||
+ | Adding dependency `intel/ | ||
+ | Adding dependency `ucx/ | ||
+ | Adding dependency `openmpi/ | ||
+ | Adding package `vasp/ | ||
+ | |||
+ | $ which vasp_std | ||
+ | / | ||
+ | |||
+ | $ which wannier90.x | ||
+ | / | ||
+ | </ | ||
+ | |||
+ | The same '' | ||
+ | |||
+ | <code text> | ||
+ | : | ||
+ | # | ||
+ | # [EDIT] Do any pre-processing, | ||
+ | # or explicit changes to PATH, LD_LIBRARY_PATH, | ||
+ | # | ||
+ | vpkg_require vasp/ | ||
+ | : | ||
+ | # | ||
+ | # [EDIT] Execute your MPI program | ||
+ | # | ||
+ | ${UD_MPIRUN} vasp_std | ||
+ | mpi_rc=$? | ||
+ | |||
+ | # | ||
+ | # [EDIT] Do any cleanup work here... | ||
+ | # | ||
+ | |||
+ | # | ||
+ | # Be sure to return the mpirun' | ||
+ | # | ||
+ | exit $mpi_rc | ||
+ | </ |