Python
Python is an interpreted, object-oriented, high-level programming language with dynamic semantics. Its high-level built in data structures, combined with dynamic typing and dynamic binding, make it very attractive for Rapid Application Development, as well as for use as a scripting or glue language to connect existing components together.
The recommended way to use Python on UD's clusters is through virtual environments created with the Conda package manager. If you are new to Conda, please refer to the documentation on Miniforge, which is available as a VALET package. The cluster-specific Miniforge pages are also linked below in Details by cluster.
There are also some Python versions available via VALET, which you can see with the VALET command vpkg_versions python. However, Miniforge is recommended so that you can set up your own virtual environments that include any version of Python that you would like to use, along with any packages that you would like to include. Mpi4py is also another Python platform with bindings of the Message Passing Interface (MPI) standard for the Python programming language allowing any Python program to exploit multiple processors.
Running Python jobs on our clusters, in particularly, machine learning and/or multiprocessing (parallel) jobs will likely use shared memory on each node assigned to the job. Unfortunately, shared memory (/dev/shm) cannot be easily reserved or cleared as part of your job submission via the job scheduler on each cluster, as a result, you may receive the following error
It has been suggested setting the environment variable JOBLIB_TEMP_DIR=${TMPDIR} as part of your job submission or defining it in your Python code for multiprocessing something like this
from multiprocessing import process process . current_process (). _config [ 'tempdir' ] = '${TMPDIR}'
will use the local scratch storage on the node versus shared memory and avoid the error described above.