Link Error When Static Linking to Intel® MKL on Linux 6

Problem

Compiling with “-mkl –static” causes a linker error on Red Hat EL 6.x while the same compilation on RHEL 5.x works correctly. Here are the commands to reproduce the problem:

$ source /opt/intel/composer_xe_2013.5.192/bin/iccvars.sh intel64

$ icc -mkl -openmp -static dgemm.c (Copy the code from manual) 

ld: cannot find -lpthread
ld: cannot find -lm

Once we solve the problem by:

$icc -mkl -openmp -static -L/usr/lib/x86_64-redhat-linux5E/lib64/ dgemm.c

New errors produce:

dgemm.c:(.text+0x465): undefined reference to ‘dgemm’ or

xxxmm.c:(.text+0x465): undefined reference to ‘__isoc99_fscanf’ 

Background

Any library that maintains state is potentially an issue when linked statically. It becomes a decision when such a library is linked multiple times against an application. Remember that this can happen indirectly e.g., via a 3^rd party. Therefore the Linux* community is trying to discourage static linking and moving the community to a dynamic linking model. Thus, recent Linux distributions are not shipping/installing GLIBC/static by default (or they are putting the static libraries into a non-default path that is not searched by default).

There are two main solutions for this problem: (1) make all required static libraries available either pointing to the non-default location, by download and installation, or by building them from source, or (2) switch to a different linkage model such as mixed dynamic and static linkage or perhaps a complete dynamic linkage model. 

Solution

In the above case, there are mainly two problems.

1. Root cause: The option –static means you need the static libraries of GLIBC, Intel^® MKL, and OpenMP* (also explained at http://software.intel.com/en-us/articles/error-ld-cannot-find-lm/). The root cause is either that RHEL 6, FC12, 13, 14 do not ship with static versions of GLIBC or that the libraries are in a directory that is not normally searched during the link step. Thus, there is no static version of libm to link against by default.
Solution:“cannot find pthread/ld/m”, and “undefined reference to __isoc99_fscanf”
Please add the path of static versions of GLIBC in command line or LIBRARY_PATH, it may be -L/usr/lib/x86_64-redhat-linux5E/lib64/ or install the static version of GLIBC for your Linux* distribution.
Note: -static-intel is to statically link the Intel libraries and dynamic link system libraries, which is recommended if you want static Intel library only.  

2.  Root cause: the position of –mkl matters

There are 4 kind of command lines:
1. $ icc -mkl dgemm.c
2. $ icc dgemm.c –mkl
3. $ icc dgemm.c -mkl –static-intel
4. $ icc -mkl dgemm.c -static-intel
The first 3 models can be build successfully with composer_xe_2013.5.192 . The 4^th gives an error (see also athttp://software.intel.com/en-us/forums/topic/393920):

Undefined reference to ‘dgemm’

The 4th model does not work because the Intel® MKL library will be located before the source file and the command actually requests to have a combined compile-link step (where the link stage will not see the required library). The static references in dgemm.o can only be resolved by libraries passed after the object.

Solution: try

$ icc dgemm.c –mkl -openmp –static-intel
Please also have a look at the Intel MKL Link Line Advisor:http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/

For example:

$ icc dgemm.c -Wl,--start-group $MKLROOT/lib/intel64/libmkl_intel_lp64.a $(MKLROOT)/lib/intel64/libmkl_intel_thread.a $MKLROOT/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread –lm 

Note: the –mkl problem cannot happen when compile and link stage are separated. Of course, the –mkl option needs to be supplied for both stages/compiler invocations (the last stage uses the Intel Compiler as a linker driver). 

Other problems

You may see several warnings with the command:

/opt/intel/composer_xe_2013.x.x/mkl/lib/intel64/libmkl_core.a (ueaa_prv_load_backend_lib.o): In function `ueaa_prv_load_backend_libs':

../../../../serv/offload/ueaa/core/coi2/ueaa_prv_load_backend_lib.c:(.text+0x233): warning: the 'dlopen' symbol can be made statically available but the statically linked application may link dynamic at runtime if certain components are present. Moreover, those components may also rely on shared libraries such GLIBC. 

Root Cause: such warnings with the -static option appear since the automatic offload support was added for the Xeon phi processor (Intel^® MKL 11.0). The automatic offload infrastructure uses some components of the MPSS if present; hence it is dynamically linked using dlopen.

Note: disabling automatic offload using the mkl_mic_enable() service function or the MKL_MIC_ENABLE environment variable does not link in a “more static” manner – it would just disable this functionality (which might present in a system).

Reference: F393920, Q698895, DPD200345901,DPD200238847