Show pageOld revisionsBack to top This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong. ====== Gurobi on Caviness ====== To determine the available versions of Gurobi<sup>TM</sup> Optimizer installed use <code bash> [traine@login00 ~]$ vpkg_versions gurobi Available versions in package (* = default version): [/opt/shared/valet/2.1/etc/gurobi.vpkg_yaml] gurobi The most powerful mathematical optimization solver out there * 9.0.1 release 9.0.1 for Linux </code> This documentation is based on [[https://www.gurobi.com/documentation/9.0/refman/index.html|Gurobi]]<sup>TM</sup> Optimizer version 9.0.1 and explains how to install and use the Gurobi Python language interface. ===== Serial or Threaded Python Environment ===== ==== Adding Gurobi to a Anaconda Environment ==== Based on the instructions on Gurobi's site for installing the Anaconda Python distribution (https://www.gurobi.com/documentation/9.0/quickstart_linux/ins_the_anaconda_python_di.html#section:Anaconda). The following steps can be adapted from those instructions to create an Anaconda Python environment for running Gurobi. The workgroup ''it_css'' will be used in this example. The anaconda environment will be created in the home directory of user ''traine''. The workgroup (''$WORKGROUP'') and home directory (''$HOME'') should be changed to reflect your workgroup and home directory. If Gurobi is going to be used by multiple people in a workgroup, then it might be worth installing the Gurobi anaconda environment in a workgroup directory (''$WORKDIR''), instead of installing in each person's home directory. Installing it in a workgroup directory will allow everyone within a workgroup to share that Gurobi anaconda environment, and not have to create their own. See [[abstract:caviness:install_software:workgroup-sw|workgroup software installs on Caviness]] for more details. <code bash> [traine@login00 ~]$ workgroup -g it_css [(it_css:traine)@login00 ~]$ vpkg_require gurobi Adding package `gurobi/9.0.1` to your environment [(it_css:traine)@login00 ~]$ vpkg_require anaconda/5.2.0:python3 Adding package `anaconda/5.2.0:python3` to your environment [(it_css:traine)@login00 ~]$ conda config --add channels http://conda.anaconda.org/gurobi [(it_css:traine)@login00 ~]$ conda create -p ~/gurobiproject Solving environment: done ==> WARNING: A newer version of conda exists. <== current version: 4.5.11 latest version: 4.8.3 Please update conda by running $ conda update -n base -c defaults conda ## Package Plan ## environment location: /home/1201/gurobiproject Proceed ([y]/n)? y Preparing transaction: done Verifying transaction: done Executing transaction: done # # To activate this environment, use: # > source activate /home/1201/gurobiproject # # To deactivate an active environment, use: # > source deactivate # [(it_css:traine)@login00 ~]$ source activate $HOME/gurobiproject (/home/1201/gurobiproject)[(it_css:traine)@login00 ~]$ conda install gurobi scipy pandas joblib Solving environment: done ==> WARNING: A newer version of conda exists. <== current version: 4.5.11 latest version: 4.8.3 Please update conda by running $ conda update -n base -c defaults conda ## Package Plan ## environment location: /home/2179/gurobiproject added / updated specs: - gurobi - joblib - pandas - scipy The following NEW packages will be INSTALLED: _libgcc_mutex: 0.1-main blas: 1.0-mkl ca-certificates: 2020.1.1-0 certifi: 2019.11.28-py37_1 gurobi: 9.0.1-py37_0 gurobi intel-openmp: 2020.0-166 joblib: 0.14.1-py_0 ld_impl_linux-64: 2.33.1-h53a641e_7 libedit: 3.1.20181209-hc058e9b_0 libffi: 3.2.1-hd88cf55_4 libgcc-ng: 9.1.0-hdf63c60_0 libgfortran-ng: 7.3.0-hdf63c60_0 libstdcxx-ng: 9.1.0-hdf63c60_0 mkl: 2020.0-166 mkl-service: 2.3.0-py37he904b0f_0 mkl_fft: 1.0.15-py37ha843d7b_0 mkl_random: 1.1.0-py37hd6b4f25_0 ncurses: 6.2-he6710b0_0 numpy: 1.18.1-py37h4f9e942_0 numpy-base: 1.18.1-py37hde5b4d6_1 openssl: 1.1.1e-h7b6447c_0 pandas: 1.0.3-py37h0573a6f_0 pip: 20.0.2-py37_1 python: 3.7.7-hcf32534_0_cpython python-dateutil: 2.8.1-py_0 pytz: 2019.3-py_0 readline: 8.0-h7b6447c_0 scipy: 1.4.1-py37h0b6359f_0 setuptools: 46.1.3-py37_0 six: 1.14.0-py37_0 sqlite: 3.31.1-h7b6447c_0 tk: 8.6.8-hbc83047_0 wheel: 0.34.2-py37_0 xz: 5.2.4-h14c3975_4 zlib: 1.2.11-h7b6447c_3 Proceed ([y]/n)? y Preparing transaction: done Verifying transaction: done Executing transaction: done [(it_css:traine)@login00 ~]$ source deactivate </code> ==== Running a Gurobi Serial or Threaded Job ==== The below steps are based on copying a template job script and modifying ''serial.qs'' or ''threads.qs'' script to load run your Gurobi Anaconda environment. <note tip>It is a good idea to periodically check in ''/opt/shared/templates/slurm/'' for updated or new [[technical:slurm:caviness:templates:start|templates]] to use as job scripts to run generic or specific applications designed to provide the best performance on Caviness.</note> To start, use VALET to load Gurobi and Anaconda. Once those two packages are loaded, the Gurobi Anaconda environment will need to be activated. After the Gurobi Anaconda environment is activated, then the python script that uses Gurobi can be called and ran. After the python script has completed, then the Gurobi Anaconda environment will be need to be deactivated. === Example serial.qs script === <code bash> # # Do general job environment setup: # . /opt/shared/slurm/templates/libexec/common.sh # # [EDIT] Add your script statements hereafter, or execute a script or program # using the srun command. # vpkg_require gurobi vpkg_require anaconda/5.2.0:python3 source activate ~${USER}/gurobiproject #Directory where the Gurobi Anaconda Environment was installed python3 serial_test.py source deactivate </code> === Example threads.qs script === <code bash> # # Do standard OpenMP environment setup: # . /opt/shared/slurm/templates/libexec/openmp.sh # # [EDIT] Execute your OpenMP/threaded program using the srun command: # vpkg_require gurobi vpkg_require anaconda/5.2.0:python3 source activate ~${USER}/gurobiproject #Directory where the Gurobi Anaconda Environment was installed python3 sample_parallel.py source deactivate </code> ===== MPI Python Environment ===== ==== Adding Gurobi and mpi4py to a Python Environment ==== Using Gurobi with MPI requires mpi4py and in order for MPI to work properly on our cluster and to understand our networking, it must be compiled using our the version of Open MPI compiled for our cluster. The instructions below are intended to create a virtual python environment that includes Gurobi and mpi4py in your home directory (or your workgroup) based on the account ''traine'' and ''/home/1201'', and workgroup ''it_css''. You will need to adjust the instructions for your own account and workgroup. Create a directory in which you can organize the Gurobi and mpi4py environment. Make sure you are in your workgroup before you start: <code bash> [traine@login01 ~]$ workgroup -g it_css [(it_css:traine)@login01 ~]$ mkdir gurobi-mpi [(it_css:traine)@login01 gurobi-mpi]$ cd gurobi-mpi </code> Now, we'll use the Intel Python distribution to build a Gurobi 9 environment — without mpi4py at this point! Rather than adding the Gurobi channel to our global conda configuration, we'll enable it just on the new environment itself: <code bash> [(it_css:traine)@login01 gurobi-mpi]$ vpkg_require intel-python/2019u5:python3 Adding package `intel-python/2019u5:python3` to your environment (root) [(it_css:anita)@login01 gurobi-mpi]$ conda create --prefix=$(pwd)/envs/20200415 --channel http://conda.anaconda.org/gurobi --channel=intel gurobi=9 scipy pandas joblib python=3 Fetching package metadata ............... Solving package specifications: . Package plan for installation in environment /home/1201/gurobi-mpi/envs/20200415: The following NEW packages will be INSTALLED: bzip2: 1.0.8-1 intel certifi: 2019.11.28-py37_0 intel cycler: 0.10.0-py37_7 intel freetype: 2.10.1-1 intel gurobi: 9.0.1-py37_0 gurobi : scipy-1.3.3-py 100% |########################################################################| Time: 0:00:00 43.73 MB/s pandas-0.25.3- 100% |########################################################################| Time: 0:00:00 44.97 MB/s # # To activate this environment, use: # source activate /home/1201/gurobi-mpi/envs/20200415 # # To deactivate an active environment, use: # source deactivate # </code> Now activate the new environment: <code bash> (root) [(it_css:traine)@login01 gurobi-mpi]$ source activate /home/1201/gurobi-mpi/envs/20200415 (/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 gurobi-mpi]$ </code> Note your prompt now shows the virtual python environment you have activated ''(/home/1201/gurobi-mpi/envs/20200415)''. Next we need to setup to build mpi4py. The ''binutils/2.25.1'' package is needed because the Anaconda environment's Python was built on a toolset that supported (and used) the ''--sysroot'' option to the ''ld'' command; the native ''binutils'' in CentOS do NOT include that functionality. Naturally, one of our Open MPI package is required to build the binary components of mpi4py: <code bash> (/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 gurobi-mpi]$ vpkg_require binutils/2.38 openmpi/4.1.2 Adding package `binutils/2.38` to your environment Adding dependency `libfabric/1.13.2` to your environment Adding package `openmpi/4.1.2` to your environment (/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 gurobi-mpi]$ pip install --no-binary :all: --compile mpi4py Collecting mpi4py Using cached https://files.pythonhosted.org/packages/ec/8f/bbd8de5ba566dd77e408d8136e2bab7fdf2b97ce06cab830ba8b50a2f588/mpi4py-3.1.3.tar.gz Skipping wheel build for mpi4py, due to binaries being disabled for it. Installing collected packages: mpi4py Running setup.py install for mpi4py ... done Successfully installed mpi4py-3.1.3 </code> Now rollback all the packages in VALET which also deactivates your environment too and then return to your home directory: <code bash> (/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 gurobi-mpi]$ vpkg_rollback all [(it_css:traine)@login01 gurobi-mpi]$ cd [(it_css:traine)@login01 ~]$ </code> ==== Creating a VALET package definition ==== Now, create a VALET package to handle the environment setup/teardown and dependencies by using ''nano'' or ''vi'' to create the file ''~/.valet/gurobi-mpi.vpkg_yaml'' and copy the following text into it changing the ''/home/1201'' to your home directory: <code bash> gurobi-mpi: prefix: /home/1201/gurobi-mpi/envs description: Gurobi and mpi4py Python environments actions: - action: source script: sh: anaconda-activate.sh success: 0 versions: "20200415": description: Python 3.7 with scipy, pandas, joblib, and mpi4py dependencies: - openmpi/3.1.3 - intel-python/2019u5:python3 - gurobi/9.0.1 </code> <note important>Using ''nano ~/.valet/gurobi-mpi.vpkg_yaml'' assumes the directory ''.valet'' exists. You will get an error "No such file or directory" when you go to save, if the directory ''.valet'' doesn't exist. Check if the ''.valet'' directory exists or not, here it shows the directory exists <code bash> [(it_css:traine)@login01 ~]$ file .valet .valet: directory </code> and in this case the directory does not exist, so create it: <code bash> [(it_css:traine)@login01 ~]$ file .valet .valet: cannot open (No such file or directory) [(it_css:traine)@login01 ~]$ mkdir .valet </code> </note> Thereafter, here is a simple test using VALET to load the environment, test we have the correct version of python3, loading python3 and importing the packages gurobipy and mpi4py, exiting python3, and deactivating the environment: <code bash> [(it_css:traine)@login01 ~]$ vpkg_require gurobi-mpi/20200415 Adding dependency `libfabric/1.9.0` to your environment Adding dependency `openmpi/3.1.3` to your environment Adding dependency `intel-python/2019u5:python3` to your environment Adding dependency `gurobi/9.0.1` to your environment Adding package `gurobi-env/20200415` to your environment (/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 ~]$ which python3 /home/1201/gurobi-mpi/envs/20200415/bin/python3 (/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 ~]$ python3 Python 3.7.7 (default, Mar 26 2020, 15:48:22) [GCC 7.3.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information. >>> import gurobipy >>> import mpi4py >>> (/home/1201/gurobi-mpi/envs/20200415) [(it_css:traine)@login01 ~]$ vpkg_rollback [(it_css:traine)@login01 ~]$ </code> ==== Running a Gurobi MPI Job ==== First copy the template job script ''openmpi.qs'' and modify it to setup the number of cores, memory and time, and use VALET to load your Gurobi and mpi4py environment that you will need to run your job. <note tip>It is a good idea to periodically check in ''/opt/shared/templates/slurm/'' for updated or new [[technical:slurm:templates:start|templates]] to use as job scripts to run generic or specific applications, designed to provide the best performance on Caviness.</note> The ''openmpi.qs'' file can be found in ''opt/templates/slurm/generic/mpi'' directory. software/gurobi/caviness.txt Last modified: 2022-07-01 13:04by frey