hpc documentation

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Anaconda on Farber ======

<code bash>
$ vpkg_versions anaconda

Available versions in package (* = default version):

[/opt/shared/valet/2.0.1/etc/anaconda.vpkg_json]
anaconda   Open Enterprise Python
* 2.5.0    Anaconda 2.5.0 (Python 2.x, 64-bit)
           + python2
  2.5.0    Anaconda 2.5.0 (Python 3.x, 64-bit)
           + python3
  5.0.1    Anaconda 5.0.1 (Python 2.7, 64-bit)
           + python2
  5.0.1    Anaconda 5.0.1 (Python 3.6, 64-bit)
           + python3
  5.2.0    Anaconda 5.2.0 (Python 2.7, 64-bit)
           + python2
  5.2.0    Anaconda 5.2.0 (Python 3.6, 64-bit)
           + python3
  python2  alias to anaconda/2.5.0:python2
  python3  alias to anaconda/2.5.0:python3
</code>

See [[software:anaconda:farber#Recipes|Recipes]] for examples of building virtual environments for specific packages.
===== Virtual Environment ====

If you are going to follow the instructions for creating an Anaconda virtual environment on Farber consider using a slight modification to the documentation provided by each software package and use VALET to load the proper version of Anaconda for Python 2 or Python3, also specify a path in your home directory not in the system path for Anaconda, and ignore the warnings to update ''conda'' while it is building your virtual environment.

See the example below which creates a virtual Anaconda environment based on Python3 for FEniCS called ''fenicsproject'' in the ''traine'' home directory and installs all the fenics packages and mshr. Once the environment is created, then a simple interactive test is done on a compute node via ''qlogin'' to verify the new ''fenicsproject'' Anaconda environment is working properly. 

<code>
[traine@farber ~]$ vpkg_devrequire anaconda/5.2.0:python3
Adding package `anaconda/5.2.0:python3` to your environment
[traine@farber ~]$ conda create -p ~/fenicsproject -c conda-forge fenics fenics-dijitso fenics-dolfin fenics-ffc fenics-fiat fenics-libdolfin fenics-ufl mshr
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.5.11
  latest version: 4.8.1

Please update conda by running

    $ conda update -n base -c defaults conda


## Package Plan ##

  environment location: /home/1201/fenicsproject

  added / updated specs:
    - fenics
    - fenics-dijitso
    - fenics-dolfin
    - fenics-ffc
    - fenics-fiat
    - fenics-libdolfin
    - fenics-ufl
    - mshr


The following packages will be downloaded:

...
...
...
Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
...
...
...
#
# To activate this environment, use:
# > source activate /home/1201/fenicsproject
#
# To deactivate an active environment, use:
# > source deactivate
#

[traine@farber ~]$ workgroup -g it_css
[(it_css:traine)@farber ~]$ qlogin
$ qlogin
Your job 3985323 ("QLOGIN") has been submitted
waiting for interactive job to be scheduled ...
Your interactive job 3985323 has been successfully scheduled.
Establishing /opt/shared/univa/local/qlogin_ssh session to host n038 ...
[(it_css:traine)@n038 ~]$ vpkg_devrequire anaconda/5.2.0:python3
Adding package `anaconda/5.2.0:python3` to your environment
[(it_css:traine)@n038 ~]$ source activate ~/fenicsproject
(/home/1201/fenicsproject) [(it_css:traine)@n038 ~]$ python3 -c "import mshr"
(/home/1201/fenicsproject) [(it_css:traine)@n038 ~]$ source deactivate
[(it_css:traine)@n038 ~]$ exit
exit
Connection to n038 closed.
/opt/shared/univa/local/qlogin_ssh exited with exit code 0
[(it_css:traine)@farber ~]$
</code>

===== Recipes ====

Examples documented as recipes to be used for specific installations including using a ''workgroup'' directory as well as creating VALET packages for these environments and job scripts setup for batch runs, but can also perhaps help others in solving similar installation dilemmas.

  * [[technical:recipes:pyqt5-in-virtualenv|Building PyQt5 in a Python Virtual Environment]]
  * [[technical:recipes:keras-in-virtualenv|Keras Python Virtual Environment]]
  * [[technical:recipes:emcee-in-virtualenv|Python Virtualenv: emcee and pyKLIP]]
  * [[technical:recipes:mpi4py-in-virtualenv|Python Virtual Environments with mpi4py]]