====== Gurobi on Caviness ====== To determine the available versions of Gurobi^TM Optimizer installed use


[traine@login00 ~]$ vpkg_versions gurobi
Available versions in package (* = default version):

[/opt/shared/valet/2.1/etc/gurobi.vpkg_yaml]
gurobi  The most powerful mathematical optimization solver out there
* 9.0.1 release 9.0.1 for Linux

This documentation is based on [[https://www.gurobi.com/documentation/9.0/refman/index.html|Gurobi]]^TM Optimizer version 9.0.1 and explains how to install and use the Gurobi Python language interface. ===== Serial or Threaded Python Environment ===== ==== Adding Gurobi to a Anaconda Environment ==== Based on the instructions on Gurobi's site for installing the Anaconda Python distribution (https://www.gurobi.com/documentation/9.0/quickstart_linux/ins_the_anaconda_python_di.html#section:Anaconda). The following steps can be adapted from those instructions to create an Anaconda Python environment for running Gurobi. The workgroup ''it_css'' will be used in this example. The anaconda environment will be created in the home directory of user ''traine''. The workgroup (''$WORKGROUP'') and home directory (''$HOME'') should be changed to reflect your workgroup and home directory. If Gurobi is going to be used by multiple people in a workgroup, then it might be worth installing the Gurobi anaconda environment in a workgroup directory (''$WORKDIR''), instead of installing in each person's home directory. Installing it in a workgroup directory will allow everyone within a workgroup to share that Gurobi anaconda environment, and not have to create their own. See [[abstract:caviness:install_software:workgroup-sw|workgroup software installs on Caviness]] for more details.


[traine@login00 ~]$ workgroup -g it_css
[(it_css:traine)@login00 ~]$ vpkg_require gurobi
Adding package `gurobi/9.0.1` to your environment
[(it_css:traine)@login00 ~]$ vpkg_require anaconda/5.2.0:python3
Adding package `anaconda/5.2.0:python3` to your environment
[(it_css:traine)@login00 ~]$ conda config --add channels http://conda.anaconda.org/gurobi
[(it_css:traine)@login00 ~]$ conda create -p ~/gurobiproject
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.5.11
  latest version: 4.8.3

Please update conda by running

    $ conda update -n base -c defaults conda



## Package Plan ##

  environment location: /home/1201/gurobiproject


Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use:
# > source activate /home/1201/gurobiproject
#
# To deactivate an active environment, use:
# > source deactivate
#

[(it_css:traine)@login00 ~]$ source activate $HOME/gurobiproject
(/home/1201/gurobiproject)[(it_css:traine)@login00 ~]$ conda install gurobi scipy pandas joblib
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.5.11
  latest version: 4.8.3

Please update conda by running

    $ conda update -n base -c defaults conda



## Package Plan ##

  environment location: /home/2179/gurobiproject

  added / updated specs:
    - gurobi
    - joblib
    - pandas
    - scipy


The following NEW packages will be INSTALLED:

    _libgcc_mutex:    0.1-main
    blas:             1.0-mkl
    ca-certificates:  2020.1.1-0
    certifi:          2019.11.28-py37_1
    gurobi:           9.0.1-py37_0             gurobi
    intel-openmp:     2020.0-166
    joblib:           0.14.1-py_0
    ld_impl_linux-64: 2.33.1-h53a641e_7
    libedit:          3.1.20181209-hc058e9b_0
    libffi:           3.2.1-hd88cf55_4
    libgcc-ng:        9.1.0-hdf63c60_0
    libgfortran-ng:   7.3.0-hdf63c60_0
    libstdcxx-ng:     9.1.0-hdf63c60_0
    mkl:              2020.0-166
    mkl-service:      2.3.0-py37he904b0f_0
    mkl_fft:          1.0.15-py37ha843d7b_0
    mkl_random:       1.1.0-py37hd6b4f25_0
    ncurses:          6.2-he6710b0_0
    numpy:            1.18.1-py37h4f9e942_0
    numpy-base:       1.18.1-py37hde5b4d6_1
    openssl:          1.1.1e-h7b6447c_0
    pandas:           1.0.3-py37h0573a6f_0
    pip:              20.0.2-py37_1
    python:           3.7.7-hcf32534_0_cpython
    python-dateutil:  2.8.1-py_0
    pytz:             2019.3-py_0
    readline:         8.0-h7b6447c_0
    scipy:            1.4.1-py37h0b6359f_0
    setuptools:       46.1.3-py37_0
    six:              1.14.0-py37_0
    sqlite:           3.31.1-h7b6447c_0
    tk:               8.6.8-hbc83047_0
    wheel:            0.34.2-py37_0
    xz:               5.2.4-h14c3975_4
    zlib:             1.2.11-h7b6447c_3

Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
[(it_css:traine)@login00 ~]$ source deactivate

==== Running a Gurobi Serial or Threaded Job ==== The below steps are based on copying a template job script and modifying ''serial.qs'' or ''threads.qs'' script to load run your Gurobi Anaconda environment. It is a good idea to periodically check in ''/opt/shared/templates/slurm/'' for updated or new [[technical:slurm:caviness:templates:start|templates]] to use as job scripts to run generic or specific applications designed to provide the best performance on Caviness. To start, use VALET to load Gurobi and Anaconda. Once those two packages are loaded, the Gurobi Anaconda environment will need to be activated. After the Gurobi Anaconda environment is activated, then the python script that uses Gurobi can be called and ran. After the python script has completed, then the Gurobi Anaconda environment will be need to be deactivated. === Example serial.qs script ===


#
# Do general job environment setup:
#
. /opt/shared/slurm/templates/libexec/common.sh

#
# [EDIT] Add your script statements hereafter, or execute a script or program
#        using the srun command.
#

vpkg_require gurobi
vpkg_require anaconda/5.2.0:python3

source activate ~${USER}/gurobiproject #Directory where the Gurobi Anaconda Environment was installed
python3 serial_test.py
source deactivate

=== Example threads.qs script ===


#
# Do standard OpenMP environment setup:
#
. /opt/shared/slurm/templates/libexec/openmp.sh

#
# [EDIT] Execute your OpenMP/threaded program using the srun command:
#

vpkg_require gurobi
vpkg_require anaconda/5.2.0:python3

source activate ~${USER}/gurobiproject #Directory where the Gurobi Anaconda Environment was installed
python3 sample_parallel.py
source deactivate

===== MPI Python Environment ===== ==== Adding Gurobi and mpi4py to a Python Environment ==== Using Gurobi with MPI requires mpi4py and in order for MPI to work properly on our cluster and to understand our networking, it must be compiled using our the version of Open MPI compiled for our cluster. The instructions below are intended to create a virtual python environment that includes Gurobi and mpi4py in your home directory (or your workgroup) based on the account ''traine'' and ''/home/1201'', and workgroup ''it_css''. You will need to adjust the instructions for your own account and workgroup. Create a directory in which you can organize the Gurobi and mpi4py environment. Make sure you are in your workgroup before you start:


[traine@login01 ~]$ workgroup -g it_css
[(it_css:traine)@login01 ~]$ mkdir gurobi-mpi
[(it_css:traine)@login01  gurobi-mpi]$ cd gurobi-mpi

Now, we'll use the Intel Python distribution to build a Gurobi 9 environment — without mpi4py at this point! Rather than adding the Gurobi channel to our global conda configuration, we'll enable it just on the new environment itself:


 [(it_css:traine)@login01  gurobi-mpi]$ vpkg_require intel-python/2019u5:python3
 Adding package `intel-python/2019u5:python3` to your environment

 (root) [(it_css:anita)@login01  gurobi-mpi]$ conda create --prefix=$(pwd)/envs/20200415 --channel http://conda.anaconda.org/gurobi --channel=intel gurobi=9 scipy pandas joblib python=3
 Fetching package metadata ...............
 Solving package specifications: .

 Package plan for installation in environment /home/1201/gurobi-mpi/envs/20200415:

 The following NEW packages will be INSTALLED:

     bzip2:           1.0.8-1               intel
     certifi:         2019.11.28-py37_0     intel
     cycler:          0.10.0-py37_7         intel
     freetype:        2.10.1-1              intel

     gurobi:          9.0.1-py37_0          gurobi
     
        :

 scipy-1.3.3-py 100% |########################################################################| Time: 0:00:00  43.73 MB/s
 pandas-0.25.3- 100% |########################################################################| Time: 0:00:00  44.97 MB/s
 #
 # To activate this environment, use:
 #  source activate /home/1201/gurobi-mpi/envs/20200415
 #
 # To deactivate an active environment, use:
 #  source deactivate
 #

Now activate the new environment:


(root) [(it_css:traine)@login01  gurobi-mpi]$ source activate /home/1201/gurobi-mpi/envs/20200415
(/home/1201/gurobi-mpi/envs/20200415)  [(it_css:traine)@login01  gurobi-mpi]$

Note your prompt now shows the virtual python environment you have activated ''(/home/1201/gurobi-mpi/envs/20200415)''. Next we need to setup to build mpi4py. The ''binutils/2.25.1'' package is needed because the Anaconda environment's Python was built on a toolset that supported (and used) the ''--sysroot'' option to the ''ld'' command; the native ''binutils'' in CentOS do NOT include that functionality. Naturally, one of our Open MPI package is required to build the binary components of mpi4py:


 (/home/1201/gurobi-mpi/envs/20200415)  [(it_css:traine)@login01  gurobi-mpi]$ vpkg_require binutils/2.38 openmpi/4.1.2
 Adding package `binutils/2.38` to your environment
 Adding dependency `libfabric/1.13.2` to your environment
 Adding package `openmpi/4.1.2` to your environment

 (/home/1201/gurobi-mpi/envs/20200415)  [(it_css:traine)@login01  gurobi-mpi]$ pip install --no-binary :all: --compile mpi4py
 Collecting mpi4py
   Using cached https://files.pythonhosted.org/packages/ec/8f/bbd8de5ba566dd77e408d8136e2bab7fdf2b97ce06cab830ba8b50a2f588/mpi4py-3.1.3.tar.gz
 Skipping wheel build for mpi4py, due to binaries being disabled for it.
 Installing collected packages: mpi4py
     Running setup.py install for mpi4py ... done
 Successfully installed mpi4py-3.1.3

Now rollback all the packages in VALET which also deactivates your environment too and then return to your home directory:


 (/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01  gurobi-mpi]$ vpkg_rollback all
 [(it_css:traine)@login01  gurobi-mpi]$ cd
 [(it_css:traine)@login01  ~]$

==== Creating a VALET package definition ==== Now, create a VALET package to handle the environment setup/teardown and dependencies by using ''nano'' or ''vi'' to create the file ''~/.valet/gurobi-mpi.vpkg_yaml'' and copy the following text into it changing the ''/home/1201'' to your home directory:


 gurobi-mpi:
     prefix: /home/1201/gurobi-mpi/envs
     description: Gurobi and mpi4py Python environments
     
     actions:
         - action: source
           script:
             sh: anaconda-activate.sh
           success: 0
     
     versions:
         "20200415":
             description: Python 3.7 with scipy, pandas, joblib, and mpi4py
             dependencies:
                 - openmpi/3.1.3
                 - intel-python/2019u5:python3
                 - gurobi/9.0.1

Using ''nano ~/.valet/gurobi-mpi.vpkg_yaml'' assumes the directory ''.valet'' exists. You will get an error "No such file or directory" when you go to save, if the directory ''.valet'' doesn't exist. Check if the ''.valet'' directory exists or not, here it shows the directory exists


[(it_css:traine)@login01 ~]$ file .valet
.valet: directory

and in this case the directory does not exist, so create it:


[(it_css:traine)@login01 ~]$ file .valet
.valet: cannot open (No such file or directory)
[(it_css:traine)@login01 ~]$ mkdir .valet

Thereafter, here is a simple test using VALET to load the environment, test we have the correct version of python3, loading python3 and importing the packages gurobipy and mpi4py, exiting python3, and deactivating the environment:


[(it_css:traine)@login01  ~]$ vpkg_require gurobi-mpi/20200415
Adding dependency `libfabric/1.9.0` to your environment
Adding dependency `openmpi/3.1.3` to your environment
Adding dependency `intel-python/2019u5:python3` to your environment
Adding dependency `gurobi/9.0.1` to your environment
Adding package `gurobi-env/20200415` to your environment
(/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 ~]$ which python3
/home/1201/gurobi-mpi/envs/20200415/bin/python3
(/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 ~]$ python3
Python 3.7.7 (default, Mar 26 2020, 15:48:22)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gurobipy
>>> import mpi4py
>>>
(/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 ~]$ vpkg_rollback
[(it_css:traine)@login01 ~]$

==== Running a Gurobi MPI Job ==== First copy the template job script ''openmpi.qs'' and modify it to setup the number of cores, memory and time, and use VALET to load your Gurobi and mpi4py environment that you will need to run your job. It is a good idea to periodically check in ''/opt/shared/templates/slurm/'' for updated or new [[technical:slurm:templates:start|templates]] to use as job scripts to run generic or specific applications, designed to provide the best performance on Caviness. The ''openmpi.qs'' file can be found in ''opt/templates/slurm/generic/mpi'' directory.