Use of the Intel oneAPI Compilers

A mainstay in the HPC world, Intel's compilers for C/C++/Fortran and the Math Kernel Library (MKL) have been producing highly-optimized code for many years. Circa 2021 Intel began transitioning the toolchain to be based on LLVM and Clang under the new branding of Intel oneAPI¹⁾. Throughout the man pages and software one finds MKL now referred to as oneMKL. For proper branding perhaps the next processor will be the Intel oneCPU, which will make dual-socket systems a two oneCPU design.

The 2022 release of the oneAPI toolchain included both the traditional compilers – icc/icpc/ifort – and the new LLVM-based compilers, icx/icpx/ifx. Starting with the 2023 release of oneAPI, the traditional compilers, though still present, produce a warning message when used:

icc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.

These warnings become extremely annoying very quickly. While the -diag-disable=10441 flag could be added to Makefiles and CMake or Autoconf workflows by the user, the intel-oneapi/2023 packages add the text -diag-disable=10441 to the environment variable __INTEL_PRE_CFLAGS. The Intel compilers will use the contents of this variable as implicit flags to the compiler.

The native version of GCC on Caviness is version 4.8.5 with a patched binutils version 2.25.1. The binutils package provides utilities that create and modify ELF object files, which is the format underlying Linux executable, library, and object files. Over time the ELF format changes, which must be effected via change to the binutils package. As a general rule:

While the Intel C++ compiler has supported increasingly-newer language standards over the years, situating it atop the native stdc++ library accompanying GCC 4.8.5 effectively limits the Intel C++ compilers to C++11²⁾. It can be difficult to diagnose C++ compilation errors when the underlying GCC tools are too old versus the chosen language standard; thus, it is important to choose a minimally-compatible alternate GCC to accompany each Intel release. The same can be said for the binutils package.

Each release of Intel oneAPI currently present on Caviness (intel-oneapi/2022 and intel-oneapi/2023) have as a direct dependency any VALET-managed version of gcc with a version greater-than or equal-to 5. This can be satisfied one of two ways:

• The user can load any version of gcc >= 5.0.0 into the environment before loading e.g. intel-oneapi/2022
• If no gcc package has been loaded, VALET will choose the newest version of the gcc package and load that into the environment along with e.g. intel-oneapi/2022

It is important to note which versions of the gcc package themselves directly depend on binutils/2.35: versions prior to gcc/10.1.0 do not have binutils/2.35 as a dependency. This means that should you do the following:

$ vpkg_require gcc/9.2 intel-oneapi/2023
Adding package `gcc/9.2.0` to your environment
Adding package `intel-oneapi/2023.0.0.25537` to your environment

the environment will use the native binutils – and the icx/icpx/ifx compilers will likely not work properly (though the traditional icc/icpc/ifort tools will). Naturally, this situation is remedied by also adding a newer binutils package to the environment:

$ vpkg_require gcc/9.2 binutils/2.35 intel-oneapi/2023
Adding package `gcc/9.2.0` to your environment
Adding package `binutils/2.35` to your environment
Adding package `intel-oneapi/2023.0.0.25537` to your environment

Without a version of gcc present, the newest version is chosen by VALET and binutils/2.35 is loaded as an indirect dependency (of gcc/12.1.0):

$ vpkg_require intel-oneapi/2023
Adding dependency `binutils/2.35` to your environment
Adding dependency `gcc/12.1.0` to your environment
Adding package `intel-oneapi/2023.0.0.25537` to your environment

The native version of GCC on Caviness is version 4.8.5 with a patched binutils version 2.27.43. That version of binutils is new enough to include the changes that GCC 10 and up require, so currently none of the gcc packages on DARWIN include binutils as a dependency.

Since the versions of GCC are the same on both clusters, the issues w.r.t. C++ version compatibility are present on DARWIN, as well. Each release of Intel oneAPI currently present on DARWIN (intel-oneapi/2022 and intel-oneapi/2023) have as a direct dependency any VALET-managed version of gcc with a version greater-than or equal-to 7. This can be satisfied one of two ways:

• The user can load any version of gcc >= 7.0.0 into the environment before loading e.g. intel-oneapi/2022
• If no gcc package has been loaded, VALET will choose the newest version of the gcc package and load that into the environment along with e.g. intel-oneapi/2022

In distant past versions of the Intel compiler suite the developer used a web form to produce a set of compiler flags – library search paths, libraries, OpenMP enablers – associated with a number of MKL options. Eventually some of this logic was built into the compilers themselves and accessed using the –mkl=<options> flag.

The <options> remain the same in oneAPI but the flag has been altered to -qmkl=<options>. The bare flag is actually accepted without complaint (even though it isn't documented in the man page) where in any other usage form an error is produced:

$ icx -qopenmp -mkl Development/omp/omp_hello.c
$ icx -qopenmp -mkl=sequential Development/omp/omp_hello.c
icx: error: unknown argument '-mkl=sequential'; did you mean '-qmkl=sequential'?

As a result, CMake and Autoconf infrastructure will continue to work with -mkl (implied parallel) but users are encouraged to use the -qmkl form regardless when employing the oneAPI icx/icpx/ifx compilers.