This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revisionBoth sides next revision |
software:python:python [2020-07-09 18:15] – anita | software:python:python [2021-09-30 14:14] – [Details by cluster] anita |
---|
====== Python ====== | |
| |
[[https://www.python.org/|Python]] is an interpreted, object-oriented, high-level programming language with dynamic semantics. Its high-level built in data structures, combined with dynamic typing and dynamic binding, make it very attractive for Rapid Application Development, as well as for use as a scripting or glue language to connect existing components together. | |
| |
There are various Python versions (platforms) and packages installed on each cluster. Please use the VALET command ''vpkg_list | grep python'' the cluster you are using to see all related Python software installation. [[software:anaconda:anaconda|Anaconda]] is a popular Python platform that may be more suitable based on the documentation provided for your software installation use or requirements. [[software:mpi4py:mpi4py|Mpi4py]] is also another Python platform with bindings of the Message Passing Interface (MPI) standard for the Python programming language allowing any Python program to exploit multiple processors. | |
| |
Running Python jobs on our clusters, in particularly, machine learning and/or multiprocessing (parallel) jobs will likely use shared memory on each node assigned to the job. Unfortunately, shared memory (''/dev/shm'') cannot be easily reserved or cleared as part of your job submission via the job scheduler on each cluster, as a result, you may receive the following error | |
| |
<note important>OSError: [Errno 28] No space left on device</note> | |
| |
It has been suggested setting the environment variable ''JOBLIB_TEMP_DIR=${TMPDIR}'' as part of your job submision will use the local scratch storage on the node versus shared memory and avoid the above error. | |
| |