====== Use of the Intel oneAPI Compilers ======

A mainstay in the HPC world, Intel's compilers for C/C++/Fortran and the Math Kernel Library (MKL) have been producing highly-optimized code for many years.  Circa 2021 Intel began transitioning the toolchain to be based on LLVM and Clang under the new branding of **Intel oneAPI**((An ironic name since the product embodies four programming languages and a multitude of libraries providing computational APIs.)).  Throughout the man pages and software one finds MKL now referred to as **oneMKL**.  For proper branding perhaps the next processor will be the Intel **oneCPU**, which will make dual-socket systems a **two oneCPU** design.

===== Deprecation Warnings =====

The 2022 release of the oneAPI toolchain included both the traditional compilers -- ''icc/icpc/ifort'' -- and the new LLVM-based compilers, ''icx/icpx/ifx''.  Starting with the 2023 release of oneAPI, the traditional compilers, though still present, produce a warning message when used:

<code text>
icc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.
</code>

These warnings become extremely annoying very quickly.  While the ''-diag-disable=10441'' flag could be added to Makefiles and CMake or Autoconf workflows by the user, the ''intel-oneapi/2023'' packages add the text ''-diag-disable=10441'' to the environment variable ''__INTEL_PRE_CFLAGS''.  The Intel compilers will use the contents of this variable as implicit flags to the compiler.

===== GCC and binutils Compatibility =====

==== Caviness ====

The native version of GCC on Caviness is version 4.8.5 with a patched binutils version 2.25.1.  The binutils package provides utilities that create and modify ELF object files, which is the format underlying Linux executable, library, and object files.  Over time the ELF format changes, which must be effected via change to the binutils package.  As a general rule:

^binutils version ^ old ELF files ^ new ELF files ^
|native 2.25.1 |  YES  |  NO  |
|VALET binutils/2.35 |  YES  |  YES  |

While the Intel C++ compiler has supported increasingly-newer language standards over the years, situating it atop the native stdc++ library accompanying GCC 4.8.5 effectively limits the Intel C++ compilers to C++11((The same problem existed with the Portland Group C++ compiler; a configuration file specifies the underlying GCC compiler and standard library paths used by the toolchain.)).  It can be difficult to diagnose C++ compilation errors when the underlying GCC tools are too old versus the chosen language standard; thus, it is important to choose a minimally-compatible alternate GCC to accompany each Intel release.  The same can be said for the binutils package.

Each release of Intel oneAPI currently present on Caviness (''intel-oneapi/2022'' and ''intel-oneapi/2023'') have as a direct dependency any VALET-managed version of ''gcc'' with a version greater-than or equal-to 5.  This can be satisfied one of two ways:

  * The user can load any version of ''gcc'' >= 5.0.0 into the environment **before** loading e.g. ''intel-oneapi/2022''
  * If no ''gcc'' package has been loaded, VALET will choose the newest version of the ''gcc'' package and load that into the environment along with e.g. ''intel-oneapi/2022''

It is important to note which versions of the ''gcc'' package themselves directly depend on ''binutils/2.35'':  versions prior to ''gcc/10.1.0'' do **not** have ''binutils/2.35'' as a dependency.  This means that should you do the following:

<code bash>
$ vpkg_require gcc/9.2 intel-oneapi/2023
Adding package `gcc/9.2.0` to your environment
Adding package `intel-oneapi/2023.0.0.25537` to your environment
</code>

the environment will use the native binutils -- and the ''icx/icpx/ifx'' compilers will likely not work properly (though the traditional ''icc/icpc/ifort'' tools will).  Naturally, this situation is remedied by also adding a newer ''binutils'' package to the environment:

<code bash>
$ vpkg_require gcc/9.2 binutils/2.35 intel-oneapi/2023
Adding package `gcc/9.2.0` to your environment
Adding package `binutils/2.35` to your environment
Adding package `intel-oneapi/2023.0.0.25537` to your environment
</code>

Without a version of ''gcc'' present, the newest version is chosen by VALET and ''binutils/2.35'' is loaded as an indirect dependency (of ''gcc/12.1.0''):

<code bash>
$ vpkg_require intel-oneapi/2023
Adding dependency `binutils/2.35` to your environment
Adding dependency `gcc/12.1.0` to your environment
Adding package `intel-oneapi/2023.0.0.25537` to your environment
</code>

==== DARWIN ====

The native version of GCC on Caviness is version 4.8.5 with a patched binutils version 2.27.43.  That version of binutils is new enough to include the changes that GCC 10 and up require, so currently none of the ''gcc'' packages on DARWIN include ''binutils'' as a dependency.

Since the versions of GCC are the same on both clusters, the issues w.r.t. C++ version compatibility are present on DARWIN, as well.  Each release of Intel oneAPI currently present on DARWIN (''intel-oneapi/2022'' and ''intel-oneapi/2023'') have as a direct dependency any VALET-managed version of ''gcc'' with a version greater-than or equal-to 7.  This can be satisfied one of two ways:

  * The user can load any version of ''gcc'' >= 7.0.0 into the environment **before** loading e.g. ''intel-oneapi/2022''
  * If no ''gcc'' package has been loaded, VALET will choose the newest version of the ''gcc'' package and load that into the environment along with e.g. ''intel-oneapi/2022''

===== MKL flags =====

In distant past versions of the Intel compiler suite the developer used a web form to produce a set of compiler flags -- library search paths, libraries, OpenMP enablers -- associated with a number of MKL options.  Eventually some of this logic was built into the compilers themselves and accessed using the ''--mkl=<options>'' flag.

The ''<options>'' remain the same in oneAPI but the flag has been altered to ''-qmkl=<options>''.  The bare flag is actually accepted without complaint (even though it isn't documented in the man page) where in any other usage form an error is produced:

<code bash>
$ icx -qopenmp -mkl Development/omp/omp_hello.c
$ icx -qopenmp -mkl=sequential Development/omp/omp_hello.c
icx: error: unknown argument '-mkl=sequential'; did you mean '-qmkl=sequential'?
</code>

As a result, CMake and Autoconf infrastructure will continue to work with ''-mkl'' (implied ''parallel'') but users are encouraged to use the ''-qmkl'' form regardless when employing the oneAPI ''icx/icpx/ifx'' compilers.