Differences

This shows you the differences between two versions of the page.

--- software:gurobi:caviness [2022-05-05 11:49] – anita
+++ software:gurobi:caviness [2022-07-01 13:04] (current) – frey
@@ Line 1: / Line 1: @@
+====== Gurobi on Caviness ======
+To determine the available versions of Gurobi<sup>TM</sup> Optimizer installed use
+<code bash>
+[traine@login00 ~]$ vpkg_versions gurobi
+Available versions in package (* = default version):
+[/opt/shared/valet/2.1/etc/gurobi.vpkg_yaml]
+gurobi  The most powerful mathematical optimization solver out there
+* 9.0.1 release 9.0.1 for Linux
+</code>
+This documentation is based on [[https://www.gurobi.com/documentation/9.0/refman/index.html|Gurobi]]<sup>TM</sup> Optimizer version 9.0.1 and explains how to install and use the Gurobi Python language interface.
+===== Serial or Threaded Python Environment =====
+==== Adding Gurobi to a Anaconda Environment ====
+Based on the instructions on Gurobi's site for installing the Anaconda Python distribution (https://www.gurobi.com/documentation/9.0/quickstart_linux/ins_the_anaconda_python_di.html#section:Anaconda). The following steps can be adapted from those instructions to create an Anaconda Python environment for running Gurobi.
+The workgroup ''it_css'' will be used in this example. The anaconda environment will be created in the home directory of user ''traine''. The workgroup (''$WORKGROUP'') and home directory (''$HOME'') should be changed to reflect your workgroup and home directory. If Gurobi is going to be used by multiple people in a workgroup, then it might be worth installing the Gurobi anaconda environment in a workgroup directory (''$WORKDIR''), instead of installing in each person's home directory. Installing it in a workgroup directory will allow everyone within a workgroup to share that Gurobi anaconda environment, and not have to create their own. See [[abstract:caviness:install_software:workgroup-sw|workgroup software installs on Caviness]] for more details.
+<code bash>
+[traine@login00 ~]$ workgroup -g it_css
+[(it_css:traine)@login00 ~]$ vpkg_require gurobi
+Adding package `gurobi/9.0.1` to your environment
+[(it_css:traine)@login00 ~]$ vpkg_require anaconda/5.2.0:python3
+Adding package `anaconda/5.2.0:python3` to your environment
+[(it_css:traine)@login00 ~]$ conda config --add channels http://conda.anaconda.org/gurobi
+[(it_css:traine)@login00 ~]$ conda create -p ~/gurobiproject
+Solving environment: done
+==> WARNING: A newer version of conda exists. <==
+  current version: 4.5.11
+  latest version: 4.8.3
+Please update conda by running
+    $ conda update -n base -c defaults conda
+## Package Plan ##
+  environment location: /home/1201/gurobiproject
+Proceed ([y]/n)? y
+Preparing transaction: done
+Verifying transaction: done
+Executing transaction: done
+#
+# To activate this environment, use:
+# > source activate /home/1201/gurobiproject
+#
+# To deactivate an active environment, use:
+# > source deactivate
+#
+[(it_css:traine)@login00 ~]$ source activate $HOME/gurobiproject
+(/home/1201/gurobiproject)[(it_css:traine)@login00 ~]$ conda install gurobi scipy pandas joblib
+Solving environment: done
+==> WARNING: A newer version of conda exists. <==
+  current version: 4.5.11
+  latest version: 4.8.3
+Please update conda by running
+    $ conda update -n base -c defaults conda
+## Package Plan ##
+  environment location: /home/2179/gurobiproject
+  added / updated specs:
+    - gurobi
+    - joblib
+    - pandas
+    - scipy
+The following NEW packages will be INSTALLED:
+    _libgcc_mutex:    0.1-main
+    blas:             1.0-mkl
+    ca-certificates:  2020.1.1-0
+    certifi:          2019.11.28-py37_1
+    gurobi:           9.0.1-py37_0             gurobi
+    intel-openmp:     2020.0-166
+    joblib:           0.14.1-py_0
+    ld_impl_linux-64: 2.33.1-h53a641e_7
+    libedit:          3.1.20181209-hc058e9b_0
+    libffi:           3.2.1-hd88cf55_4
+    libgcc-ng:        9.1.0-hdf63c60_0
+    libgfortran-ng:   7.3.0-hdf63c60_0
+    libstdcxx-ng:     9.1.0-hdf63c60_0
+    mkl:              2020.0-166
+    mkl-service:      2.3.0-py37he904b0f_0
+    mkl_fft:          1.0.15-py37ha843d7b_0
+    mkl_random:       1.1.0-py37hd6b4f25_0
+    ncurses:          6.2-he6710b0_0
+    numpy:            1.18.1-py37h4f9e942_0
+    numpy-base:       1.18.1-py37hde5b4d6_1
+    openssl:          1.1.1e-h7b6447c_0
+    pandas:           1.0.3-py37h0573a6f_0
+    pip:              20.0.2-py37_1
+    python:           3.7.7-hcf32534_0_cpython
+    python-dateutil:  2.8.1-py_0
+    pytz:             2019.3-py_0
+    readline:         8.0-h7b6447c_0
+    scipy:            1.4.1-py37h0b6359f_0
+    setuptools:       46.1.3-py37_0
+    six:              1.14.0-py37_0
+    sqlite:           3.31.1-h7b6447c_0
+    tk:               8.6.8-hbc83047_0
+    wheel:            0.34.2-py37_0
+    xz:               5.2.4-h14c3975_4
+    zlib:             1.2.11-h7b6447c_3
+Proceed ([y]/n)? y
+Preparing transaction: done
+Verifying transaction: done
+Executing transaction: done
+[(it_css:traine)@login00 ~]$ source deactivate
+</code>
+==== Running a Gurobi Serial or Threaded Job ====
+The below steps are based on copying a template job script and modifying ''serial.qs'' or ''threads.qs'' script to load run your Gurobi Anaconda environment.
+<note tip>It is a good idea to periodically check in ''/opt/shared/templates/slurm/'' for updated or new [[technical:slurm:caviness:templates:start|templates]] to use as job scripts to run generic or specific applications designed to provide the best performance on Caviness.</note>
+To start, use VALET to load Gurobi and Anaconda. Once those two packages are loaded, the Gurobi Anaconda environment will need to be activated. After the Gurobi Anaconda environment is activated, then the python script that uses Gurobi can be called and ran. After the python script has completed, then the Gurobi Anaconda environment will be need to be deactivated.
+=== Example serial.qs script ===
+<code bash>
+#
+# Do general job environment setup:
+#
+. /opt/shared/slurm/templates/libexec/common.sh
+#
+# [EDIT] Add your script statements hereafter, or execute a script or program
+#        using the srun command.
+#
+vpkg_require gurobi
+vpkg_require anaconda/5.2.0:python3
+source activate ~${USER}/gurobiproject #Directory where the Gurobi Anaconda Environment was installed
+python3 serial_test.py
+source deactivate
+</code>
+=== Example threads.qs script ===
+<code bash>
+#
+# Do standard OpenMP environment setup:
+#
+. /opt/shared/slurm/templates/libexec/openmp.sh
+#
+# [EDIT] Execute your OpenMP/threaded program using the srun command:
+#
+vpkg_require gurobi
+vpkg_require anaconda/5.2.0:python3
+source activate ~${USER}/gurobiproject #Directory where the Gurobi Anaconda Environment was installed
+python3 sample_parallel.py
+source deactivate
+</code>
+===== MPI Python Environment =====
+==== Adding Gurobi and mpi4py to a Python Environment ====
+Using Gurobi with MPI requires mpi4py and in order for MPI to work properly on our cluster and to understand our networking, it must be compiled using our the version of Open MPI compiled for our cluster.
+The instructions below are intended to create a virtual python environment that includes Gurobi and mpi4py in your home directory (or your workgroup) based on the account ''traine'' and ''/home/1201'', and workgroup ''it_css''.  You will need to adjust the instructions for your own account and workgroup.
+Create a directory in which you can organize the Gurobi and mpi4py environment. Make sure you are in your workgroup before you start:
+<code bash>
+[traine@login01 ~]$ workgroup -g it_css
+[(it_css:traine)@login01 ~]$ mkdir gurobi-mpi
+[(it_css:traine)@login01  gurobi-mpi]$ cd gurobi-mpi
+</code>
+Now, we'll use the Intel Python distribution to build a Gurobi 9 environment — without mpi4py at this point!  Rather than adding the Gurobi channel to our global conda configuration, we'll enable it just on the new environment itself:
+<code bash>
+ [(it_css:traine)@login01  gurobi-mpi]$ vpkg_require intel-python/2019u5:python3
+ Adding package `intel-python/2019u5:python3` to your environment
+ (root) [(it_css:anita)@login01  gurobi-mpi]$ conda create --prefix=$(pwd)/envs/20200415 --channel http://conda.anaconda.org/gurobi --channel=intel gurobi=9 scipy pandas joblib python=3
+ Fetching package metadata ...............
+ Solving package specifications: .
+ Package plan for installation in environment /home/1201/gurobi-mpi/envs/20200415:
+ The following NEW packages will be INSTALLED:
+     bzip2:           1.0.8-1               intel
+     certifi:         2019.11.28-py37_0     intel
+     cycler:          0.10.0-py37_7         intel
+     freetype:        2.10.1-1              intel
+     gurobi:          9.0.1-py37_0          gurobi
+        :
+ scipy-1.3.3-py 100% |########################################################################| Time: 0:00:00  43.73 MB/s
+ pandas-0.25.3- 100% |########################################################################| Time: 0:00:00  44.97 MB/s
+ #
+ # To activate this environment, use:
+ #  source activate /home/1201/gurobi-mpi/envs/20200415
+ #
+ # To deactivate an active environment, use:
+ #  source deactivate
+ #
+</code>
+Now activate the new environment:
+<code bash>
+(root) [(it_css:traine)@login01  gurobi-mpi]$ source activate /home/1201/gurobi-mpi/envs/20200415
+(/home/1201/gurobi-mpi/envs/20200415)  [(it_css:traine)@login01  gurobi-mpi]$
+</code>
+Note your prompt now shows the virtual python environment you have activated ''(/home/1201/gurobi-mpi/envs/20200415)''. Next we need to setup to build mpi4py. The ''binutils/2.25.1'' package is needed because the Anaconda environment's Python was built on a toolset that supported (and used) the ''--sysroot'' option to the ''ld'' command; the native ''binutils'' in CentOS do NOT include that functionality.  Naturally, one of our Open MPI package is required to build the binary components of mpi4py:
+<code bash>
+ (/home/1201/gurobi-mpi/envs/20200415)  [(it_css:traine)@login01  gurobi-mpi]$ vpkg_require binutils/2.38 openmpi/4.1.2
+ Adding package `binutils/2.38` to your environment
+ Adding dependency `libfabric/1.13.2` to your environment
+ Adding package `openmpi/4.1.2` to your environment
+ (/home/1201/gurobi-mpi/envs/20200415)  [(it_css:traine)@login01  gurobi-mpi]$ pip install --no-binary :all: --compile mpi4py
+ Collecting mpi4py
+   Using cached https://files.pythonhosted.org/packages/ec/8f/bbd8de5ba566dd77e408d8136e2bab7fdf2b97ce06cab830ba8b50a2f588/mpi4py-3.1.3.tar.gz
+ Skipping wheel build for mpi4py, due to binaries being disabled for it.
+ Installing collected packages: mpi4py
+     Running setup.py install for mpi4py ... done
+ Successfully installed mpi4py-3.1.3
+</code>
+Now rollback all the packages in VALET which also deactivates your environment too and then return to your home directory:
+<code bash>
+ (/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01  gurobi-mpi]$ vpkg_rollback all
+ [(it_css:traine)@login01  gurobi-mpi]$ cd
+ [(it_css:traine)@login01  ~]$
+</code>
+==== Creating a VALET package definition ====
+Now, create a VALET package to handle the environment setup/teardown and dependencies by using ''nano'' or ''vi'' to create the file ''~/.valet/gurobi-mpi.vpkg_yaml'' and copy the following text into it changing the ''/home/1201'' to your home directory:
+<code bash>
+ gurobi-mpi:
+     prefix: /home/1201/gurobi-mpi/envs
+     description: Gurobi and mpi4py Python environments
+     actions:
+         - action: source
+           script:
+             sh: anaconda-activate.sh
+           success: 0
+     versions:
+         "20200415":
+             description: Python 3.7 with scipy, pandas, joblib, and mpi4py
+             dependencies:
+                 - openmpi/3.1.3
+                 - intel-python/2019u5:python3
+                 - gurobi/9.0.1
+</code>
+<note important>Using ''nano ~/.valet/gurobi-mpi.vpkg_yaml'' assumes the directory ''.valet'' exists. You will get an error "No such file or directory" when you go to save, if the directory ''.valet'' doesn't exist.
+Check if the ''.valet'' directory exists or not, here it shows the directory exists
+<code bash>
+[(it_css:traine)@login01 ~]$ file .valet
+.valet: directory
+</code>
+and in this case the directory does not exist, so create it:
+<code bash>
+[(it_css:traine)@login01 ~]$ file .valet
+.valet: cannot open (No such file or directory)
+[(it_css:traine)@login01 ~]$ mkdir .valet
+</code>
+</note>
+Thereafter, here is a simple test using VALET to load the environment, test we have the correct version of python3, loading python3 and importing the packages gurobipy and mpi4py, exiting python3, and deactivating the environment:
+<code bash>
+[(it_css:traine)@login01  ~]$ vpkg_require gurobi-mpi/20200415
+Adding dependency `libfabric/1.9.0` to your environment
+Adding dependency `openmpi/3.1.3` to your environment
+Adding dependency `intel-python/2019u5:python3` to your environment
+Adding dependency `gurobi/9.0.1` to your environment
+Adding package `gurobi-env/20200415` to your environment
+(/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 ~]$ which python3
+/home/1201/gurobi-mpi/envs/20200415/bin/python3
+(/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 ~]$ python3
+Python 3.7.7 (default, Mar 26 2020, 15:48:22)
+[GCC 7.3.0] :: Anaconda, Inc. on linux
+Type "help", "copyright", "credits" or "license" for more information.
+>>> import gurobipy
+>>> import mpi4py
+>>>
+(/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 ~]$ vpkg_rollback
+[(it_css:traine)@login01 ~]$
+</code>
+==== Running a Gurobi MPI Job ====
+First copy the template job script ''openmpi.qs'' and modify it to setup the number of cores, memory and time, and use VALET to load your Gurobi and mpi4py environment that you will need to run your job.
+<note tip>It is a good idea to periodically check in ''/opt/shared/templates/slurm/'' for updated or new [[technical:slurm:templates:start|templates]] to use as job scripts to run generic or specific applications, designed to provide the best performance on Caviness.</note>
+The ''openmpi.qs'' file can be found in ''opt/templates/slurm/generic/mpi'' directory.