OpenMM Python Virtual Environment

This is an old revision of the document!

This page documents the creation of a Python virtual environment (virtualenv) containing the OpenMM software on the Caviness HPC system¹⁾. It assumes that the user is adding the software to the workgroup storage.

Prepare to add software in the standard sub-directories of the workgroup storage:

[user@login01 ~]$ workgroup -g my_workgroup
[(my_workgroup:user)@login01 ~]$ mkdir --mode=2775 --parent ${WORKDIR}/sw/openmm
[(my_workgroup:user)@login01 ~]$ mkdir --mode=2775 --parent ${WORKDIR}/sw/valet

These commands create any missing directories. All directories created will have group-write and -inherit permissions.

The Python distribution will form the basis for the anaconda virtualenv, so add it to the environment:

[(my_workgroup:user)@login01 ~]$ vpkg_require anaconda/2024.02
Adding package `anaconda/2024.02` to your environment

The virtual environment is first populated with all packages that do not require OpenMM. Any packages requiring OpenMM must be installed after we build and install our local copy of OpenMM in the virtual environment. In this example, neither Numpy nor Scipy require OpenMM.

The two channel options are present to ensure only the default Anaconda channels are consulted – otherwise the command could still pick packages from the Intel channel, for example, which would still have the binary compatibility issues!

$ conda create --prefix=$WORKDIR/sw/openmm/20240726 --override-channels --channel conda-forge python'=>3.7' numpy scipy
Solving environment: done
    :
Proceed ([y]/n)? y
    :
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#                                                                               
# To activate this environment, use                                             
#                                                                               
#     $ conda activate /work/it_css/sw/openmm/20240726                          
#                                                                               
# To deactivate an active environment, use                                      
#                                                                               
#     $ conda deactivate 
#

Before building and installing OpenMM the environment needs to be activated:

$ conda activate $WORKDIR/sw/openmm/20240726
(/work/it_css/sw/openmm/20240726)$

With the new virtual environment activated, we can now build OpenMM.

(/work/it_css/sw/openmm/20240726)$ conda install -c conda-forge openmm
Solving environment: done
    :
Proceed ([y]/n)? y
    :
Preparing transaction: done                                                     
Verifying transaction: done                                                     
Executing transaction: \ By downloading and using the CUDA Toolkit conda packages, you accept the terms and conditions of the CUDA End User License Agreement (EULA): https://docs.nvidia.com/cuda/eula/index.html
 
done

Assuming the workgroup does not already have a TensorFlow VALET package definition, the following text:

tensorflow:
    prefix: /work/my_workgroup/sw/tensorflow
    description: TensorFlow Python environments
    flags:
        - no-standard-paths
    actions:
        - action: source
          script:
              sh: anaconda-activate.sh
          order: failure-first
          success: 0
    versions:
        "2.2.0:gpu":
            description: 2.2.0 with GPU support
            dependencies:
                - intel-python/2020u2:python3
        "2.3.0:intel,python3.8":
            description: 2.3.0 with Python 3.8, Intel optimizations
            dependencies:
                - intel-python/2020u2:python3

would be added to ${WORKDIR}/sw/valet/tensorflow.vpkg_yaml. If that file already exists, add your new version at the same level as others:

tensorflow:
    prefix: /work/my_workgroup/sw/tensorflow
    description: TensorFlow Python environments
    flags:
        - no-standard-paths
    actions:
        - action: source
          script:
              sh: anaconda-activate.sh
          order: failure-first
          success: 0
    versions:
        "2.2.0:gpu":
            description: 2.2.0 with GPU support
            dependencies:
                - intel-python/2020u2:python3
        "2.3.0:intel,python3.8":
            description: 2.3.0 with Python 3.8, Intel optimizations
            dependencies:
                - intel-python/2020u2:python3
        "1.8.0":
            description: 1.8.0 from pkgs/main
            dependencies:
                - intel-python/2018u3:python3

Make sure you modify prefix: /work/my_workgroup/sw/tensorflow for your workgroup (e.g. If my workgroup is it_nss, then use I would use prefix: /work/it_nss/sw/tensorflow).

On Caviness after a user has used the workgroup command, VALET searches for package definitions in ${WORKDIR}/sw/valet by default. VALET also searches a ~/.valet directory (in your home directory) if it exists, so that's the best location for personal package definitions – for software you've installed in your home directory, for example.

With a properly-constructed package definition file, you can now check for your versions of TensorFlow:

[(it_nss:frey)@login00 ~]$ vpkg_versions tensorflow
 
Available versions in package (* = default version):
 
[/work/my_workgroup/sw/valet/tensorflow.vpkg_yaml]
tensorflow               TensorFlow Python environments
* 2.2.0:gpu              2.2.0 with GPU support
  2.3.0:intel,python3.8  2.3.0 with Python 3.8, Intel optimizations
 
     :

¹⁾

The steps should also work on the DARWIN HPC system, though with different package versions.

OpenMM Python Virtual Environment

Prepare Workgroup Directory

Create Virtualenv

Building OpenMM

VALET Package Definition

hpc documentation