Differences

This shows you the differences between two versions of the page.

--- software:python:python [2020-07-09 17:13] – created anita
+++ software:python:python [2024-02-20 09:48] – [Python] anita
@@ Line 1: / Line 1: @@
-==== Python ====
+====== Python ======
+[[https://www.python.org/|Python]] is an interpreted, object-oriented, high-level programming language with dynamic semantics. Its high-level built in data structures, combined with dynamic typing and dynamic binding, make it very attractive for Rapid Application Development, as well as for use as a scripting or glue language to connect existing components together.
+There are various Python versions (platforms) and packages installed on each cluster. Please use the VALET command ''vpkg_list | grep python'' on the cluster you are using to see all related Python software installations.  [[software:anaconda:anaconda|Anaconda]] is a popular Python platform that may be more suitable based on the documentation provided for your software installation use or requirements. [[software:mpi4py:mpi4py|Mpi4py]] is also another Python platform with bindings of the Message Passing Interface (MPI) standard for the Python programming language allowing any Python program to exploit multiple processors.
+Running Python jobs on our clusters, in particularly, machine learning and/or multiprocessing (parallel) jobs will likely use shared memory on each node assigned to the job.  Unfortunately, shared memory (''/dev/shm'') cannot be easily reserved or cleared as part of your job submission via the job scheduler on each cluster, as a result, you may receive the following error
+<note important>OSError: [Errno 28] No space left on device</note>
+It has been suggested setting the environment variable ''JOBLIB_TEMP_DIR=${TMPDIR}'' as part of your job submission or defining it in your Python code for multiprocessing something like this
+<code bash>
+from multiprocessing import process
+process . current_process (). _config [ 'tempdir' ] = '${TMPDIR}'
+</code>
+will use the local scratch storage on the node versus shared memory and avoid the error described above.
+===== Details by cluster =====
+  * Caviness
+      * [[software:anaconda:caviness | Anaconda]]
+      * [[software:mpi4py:caviness| Mpi4py]]
+  * DARWIN
+      * [[software:anaconda:darwin | Anaconda]]
+      * [[technical:recipes:mpi4py-in-virtualenv|Python Virtual Environments with mpi4py]]
+  * Farber
+      * [[software:anaconda:farber | Anaconda]]
+      * [[software:mpi4py:farber| Mpi4py]]