Building VASP* with Intel® MKL and Intel® Compilers

Building VASP* with Intel® MKL and Intel® Compilers

Step 1 – Overview

This guide is intended to help users on how to build VASP (Vienna Ab-Initio Package Simulation) using Intel® Math Kernel Library (Intel® MKL) and Intel® Compilers and Intel® MPI.

VASP is a package for performing ab-initio quantum-mechanical molecular dynamics (MD) using pseudo potentials and a plane wave basis set. The approach implemented in VAMP/VASP is based on a finite-temperature local-density approximation (with the free energy as variational quantity) and an exact evaluation of the instantaneous electronic ground state at each MD-step using efficient matrix diagonalization schemes and an efficient Pulay mixing. These techniques avoid all problems occurring in the original Car-Parrinello method which is based on the simultaneous integration of electronic and ionic equations of motion. The interaction between ions and electrons is described using ultrasoft Vanderbilt pseudopotentials (US-PP) or the projector augmented wave method (PAW). Both techniques allow a considerable reduction of the necessary number of plane-waves per atom for transition metals and first row elements. Forces and stress can be easily calculated with VAMP/VASP and used to relax atoms into their instantaneous groundstate.  [Ref: VASP]

Version Information

This application note was created to help users who benchmark clusters using VASP to also incorporate the latest version of Intel® MKL on Linux platforms on Xeon systems.  Specifically, we'll address Intel® MKL version 11.1

More information on VASP can be found from http://cms.mpi.univie.ac.at/vasp/

Step 2 – Downloading VASP Source Code

VASP is not public-domain or share-ware, and will be distributed only after a license contract has been signed. Please visit VASP homepage to know more details on obtaining the license.

Prerequisites

1. Intel® MKL contains highly optimized FFT and also the wrappers for FFTW, which can be obtained from the following options:
   • Download a FREE evaluation version of the Intel® MKL product.
   • Download the FREE non-commercial* version of the Intel® MKL product.
   • Purchase a copy of Intel® MKL

All of these can be obtained at: http://www.intel.com/software/products/mkl.

1. Intel® C++ and FORTRAN compilers can be obtained from http://www.intel.com/softwre/products/compilers.

1. Intel MPI can be obtained from http://www.intel.com/software/products/cluster.

Open source MPI (MPICH2) can be obtained from http://www-unix.mcs.anl.gov/mpi/mpich/ .

Step 3 - Configuration

Use the following commands to extract the VASPfiles:

$tar –xvzf vasp.tgz vasp.lib.tgz

This will create  vasp and vasp.lib directories

Make sure that MPI, C++ and FORTRAN compilers are installed and in the PATH. Also set LD_LIBRARY_PATH to your compiler (C++ and FORTRAN), MPI, and Intel® MKL libraries.

Note:  This application note is written specifically for use with the Intel compilers and MPI.

Step 4 – Building VASP

1. Build FFTW wrapper library for Intel® MKL
    

Change directory to <your MKL installation>/interfaces/fftw3xf

Assuming default installation

$cd /opt/intel/mkl/interfaces/fftw3xf

Build the fftw 3.x FORTRAN wrapper library for Intel® 64 (previously known as em64t) platform using the Intel compilers.

$make libintel64 compiler=intel

This will create the fftw wrapper library “libfftw3xf_intel.a” in lib/em64t directory of your Intel® MKL installation.

1. Build libdmy.a

Change directory to vasp.x.x.lib

Modify the makefile.linux_ifc_P4 Makefile to point to the correct Intel FORTRAN compiler.

FC=ifort

Run the following command from vasp.x.x.lib, using the Makefile for linux using the Intel compiler.

$make –f makefile.linux_ifc_P4

After a successful compilation, libdmy.a will be built in the same directory.

1. Build VASP

Change directory to vasp.x.x

Export the environment variable MPIFC to point to the Intel MPI Fortran compiler mpiifort.

$export MPIFC=mpiifort

Edit the makefile.linux_ifc_P4 to link with Intel® MKL libraries and change the Fotran compiler

Under FORTRAN compiler and linker part of the makefile

FC=$(MPIFC)

CPP    = $(CPP_) -DMPI  -DHOST=\"LinuxIFC\" -DIFC \
     -DCACHE_SIZE=32000 -DPGF90 -Davoidalloc -DNGZhalf \
     -DMPI_BLOCK=64000 -DscaLAPACK -Duse_collective -Ddo_loops –DMKL_ILP64

Change the FORTRAN flags section as shown here.

FFFLAGS = -FR -names lowercase -assume byterecl -I$(MKLROOT)/include/fftw –xAVX

MKLROOT=/opt/intel/composer_xe_2013_sp1/mkl
MKL_PATH=$(MKLROOT)/lib/intel64
MKL_FFTW_PATH=$(MKL_PATH)/interfaces/fftw3xf

BLAS= -mkl=cluster

LAPACK= -mkl=cluster

…
.……
…………

Comment out the existing FFT3D line

#FFT3D = fftdfurth.o fftdlib.o

FFT3D= fftmpiw.o fftmpi_map.o fft3d.o fft3dlib.o $(MKL_FFTW_PATH)/libfftw3xf_intel.a

INCS = -I$(MKLROOT)/include/fftw

To use Intel® MKL ScaLAPACK, modify additional options for CPP for parallel version adding

-DscaLAPACK to the CPP line as mentioned above and specify Intel® MKL ScaLAPACK as below.

SCA=$(MKL_PATH)/intel64/libmkl_scalapack_lp64.a $(MKL_PATH)/libmkl_blacs_intelmkl_ilp64.a

Run the following command to build vasp

$make –f makefile.linux_ifc_P4

This will create the VASP executable in the current directory.

Step 5 - Running VASP

Run vasp by executing mpiexec command with your required parameters.  Below, for e.g., to run 32 processes use as shown with your workloads.

$mpiexec –n 32 ./vasp

Appendix B - Known Issues and Limitations

There could be a compilation error on nonlr.F file as reported below.

mpif90 -fc=ifort  -FR -names lowercase -assume byterecl  -O2 -ip   -I/opt/intel/compiler/2013_sp1.1.106/composer_xe_2013_sp1.1.106/mkl/include/fftw  -c nonlr.f90

nonlr.F(3069): error #6404: This name does not have a type, and must have an explicit type.   [LM]
             DO LM=1,LMMAXC
nonlr.F(3069): error #6404: This name does not have a type, and must have an explicit type.   [LM]
             DO LM=1,LMMAXC
----------------^
nonlr.F(3069): error #6063: An INTEGER or REAL data type is required in this context.   [LM]
             DO LM=1,LMMAXC
----------------^
nonlr.F(3206): error #6404: This name does not have a type, and must have an explicit type.   [LM]
             DO LM=1,LMMAXC
----------------^
nonlr.F(3206): error #6063: An INTEGER or REAL data type is required in this context.   [LM]
             DO LM=1,LMMAXC
----------------^

By applying the patch mentioned below to the nonlr.F, this compilation error can be fixed.

---
>     INTEGER IP, LMBASE, ISPIRAL, ISPINOR, NLIIND, NIS, NT, LMMAXC, NI, INDMAX, L, IND
3144c3144
<     INTEGER IP, LMBASE, ISPIRAL, ISPINOR, NLIIND, NIS, NT, LMMAXC, NI, INDMAX, L, LM, IND
---
>     INTEGER IP, LMBASE, ISPIRAL, ISPINOR, NLIIND, NIS, NT, LMMAXC, NI, INDMAX, L, IND

Appendix C – References

VASP (Vienna Ab-Initio Package Simulation)

Intel® Math Kernel Library

Intel® Cluster Studio XE