software:mpi4py:mpi4py

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		 Previous revision
software:mpi4py:mpi4py [2017-10-23 17:57] – created sraskarsoftware:mpi4py:mpi4py [2024-03-05 17:00] (current) – [Details by cluster] anita
Line 1: Line 1:
 ====== Mpi4py ====== ====== Mpi4py ======
  
-MPI for Python ([[http://pythonhosted.org//mpi4py/usrman/index.html|mpi4py]]) provides bindings of the Message Passing Interface (MPI) standard for the Python programming language, allowing any Python program to exploit multiple processors.+MPI for Python ([[http://mpi4py.scipy.org/docs/|mpi4py]]) provides bindings of the Message Passing Interface (MPI) standard for the Python programming language, allowing any Python program to exploit multiple processors.
  
 This package is constructed on top of the MPI-1/2 specifications and provides an object oriented interface which closely follows MPI-2 C++ bindings. It supports point-to-point (sends, receives) and collective (broadcasts, scatters, gathers) communications of any picklable Python object, as well as optimized communications of Python object exposing the single-segment buffer interface (NumPy arrays, builtin bytes/string/array objects) This package is constructed on top of the MPI-1/2 specifications and provides an object oriented interface which closely follows MPI-2 C++ bindings. It supports point-to-point (sends, receives) and collective (broadcasts, scatters, gathers) communications of any picklable Python object, as well as optimized communications of Python object exposing the single-segment buffer interface (NumPy arrays, builtin bytes/string/array objects)
  
-===== Sample mpi4py script =====+===== Recipes ===== 
 +If you need to build a Python virtualenv based on a collection of Python modules including mpi4py, then you will need to follow this recipe to get a properly-integrated mpi4py module versus following details by cluster.
  
-Adapted from the documentation provided by [[https://modelingguru.nasa.gov/docs/DOC-2412|NASA Modeling Guru]] consider the mpi4py script that implements the [[https://github.com/jbornschein/mpi4py-examples/blob/master/03-scatter-gather|scatter-gather procedure]]: +  * [[technical:recipes:mpi4py-in-virtualenv|Building a Python virtualenv with a properly-integrated mpi4py module]] 
- +===== Details by cluster===== 
-<code - scatter-gather.py> +  * [[software:mpi4py:caviness| Caviness]] 
-#-------------- +  * [[technical:recipes:mpi4py-in-virtualenv| DARWIN]] 
-# Loaded Modules +  * [[software:mpi4py:farber| Farber]]
-#-------------- +
-import numpy as np +
-from mpi4py import MPI +
-  +
-#------------- +
-# Communicator +
-#------------- +
-comm = MPI.COMM_WORLD +
-  +
-my_N = 4 +
-N = my_N * comm.size +
-  +
-if comm.rank == 0: +
-    A = np.arange(N, dtype=np.float64) +
-else: +
-    #Note that if I am not the root processor A is an empty array +
-    A = np.empty(N, dtype=np.float64) +
-  +
-my_A = np.empty(my_N, dtype=np.float64) +
-  +
-#------------------------- +
-# Scatter data into my_A arrays +
-#------------------------- +
-comm.Scatter( [A, MPI.DOUBLE], [my_A, MPI.DOUBLE] ) +
-  +
-if comm.rank == 0: +
-   print "After Scatter:" +
-  +
-for r in xrange(comm.size): +
-    if comm.rank == r: +
-        print "[%d] %s" % (comm.rank, my_A) +
-    comm.Barrier() +
-  +
-#------------------------- +
-# Everybody is multiplying by 2 +
-#------------------------- +
-my_A *= 2 +
-  +
-#----------------------- +
-# Allgather data into A again +
-#----------------------- +
-comm.Allgather( [my_A, MPI.DOUBLE], [A, MPI.DOUBLE] ) +
-  +
-if comm.rank == 0: +
-   print "After Allgather:" +
-  +
-for r in xrange(comm.size): +
-    if comm.rank == r: +
-        print "[%d] %s" % (comm.rank, A) +
-    comm.Barrier() +
-</code> +
- +
-===== Batch job ===== +
- +
-Any MPI job requires you to use ''mpirun'' to initiate it, and this should be done through the Grid Engine job scheduler to best utilize the resources on the cluster.  Also, if you want to run on more than 1 node (more than 24 or 48 cores depending on the node specifications), then you must initiate a batch job from the head node. Remember if you only have 1 node in your workgroup, then you would need to take advantage of the [[general/jobsched/standby|standby]] queue to be able to run a job utilizing multiple nodes. +
- +
-The best results on Mills have been found by using the //openmpi-psm.qs// template for [[clusters/mills/start#open-mpi|Open MPI]] jobs. For example, copy the template and call it ''mympi4py.qs'' for the job script using +
- +
-<code bash> +
-cp /opt/templates/gridengine/openmpi/openmpi-psm.qs mympi4py.qs +
-</code> +
- +
-and modify it for your application. Change ''NPROC'' to be number of cores you want.  If your task is floating point intensive, then you will get the best performance by specifying twice the number of cores and actually using only 1/2 of them since the FPU (Floating-Point Unit) is shared by core pairs. For example, if you need 24 cores for your job, then you would need to specify 48 cores for ''NPROC'', uncomment the options ''WANT_CPU_AFFINITY=YES'' and ''WANT_HALF_CORES_ONLY=YES'', then it will use only 24 cores by evenly loading them on the number nodes adjusted based on the PSM resources available. In this example, your job could be spread over 3 nodes using 8 cores each due to other jobs running on the nodes. However if you specify [[general/jobsched/exclusive|exclusive access]] by using ''-l exclusive=1'', then no other jobs can be running on the nodes, giving exclusive access to your job, and it would evenly load 2 nodes and use 12 cores on each.  It is important to specify a multiple of 24 when using core pairs or exclusive access.  Make sure you specify the correct VALET environment for your job. For this example, replace ''vpkg_require openmpi/1.4.4-gcc'' with +
- +
-<code bash>  +
-vpkg_require mpi4py +
-vpkg_require numpy +
-</code> +
- +
-Lastly, modify ''MY_EXE'' for your mpi4py script.  In this example, it would be +
- +
-<code> +
-MY_EXE="python scatter-gather.py" +
-</code> +
- +
-All the options for ''mpirun'' will automatically be determined based on the options you selected above for your job script. Now to run this job, from the head node, Mills, simply use +
-  +
-<code bash> +
-qsub mympi4py.qs +
-</code> +
-  +
-or +
- +
-<code bash> +
-qsub -l exclusive=1 mympi4py.qs +
-</code> +
- +
-for [[general/jobsched/exclusive|exclusive access]] of the nodes needed for your job. +
- +
-Remember if you want to specify more cores for ''NPROC'' than available in your workgroup, then you need to specify the [[general/jobsched/standby|standby]] queue ''-l standby=1'' but remember it has limited run times based on the total number of cores you are using for all of your jobs. In this example, since it is only requesting 48 cores, then it can run up to 8 hours using +
-  +
-<code bash> +
-qsub -l exclusive=1 -l standby=1 mympi4py.qs +
-</code>+
    
 ===== Additional Resources ===== ===== Additional Resources =====
  
 +  * [[https://mpi4py.readthedocs.io/en/stable/|MPI for Python Documentation]]
 +  * Cornell Virtual Workshop [[https://cvw.cac.cornell.edu/python/mpi4py|Python for High Performance: mpi4py]]
 +  * [[https://readthedocs.org/projects/mpi4py/downloads/pdf/latest/|MPI for Python - Read the Docs]]
   * [[http://www.tacc.utexas.edu/c/document_library/get_file?uuid=be16db01-57d9-4422-b5d5-17625445f351&groupId=13601|Introduction to mpi4py by TACC]]   * [[http://www.tacc.utexas.edu/c/document_library/get_file?uuid=be16db01-57d9-4422-b5d5-17625445f351&groupId=13601|Introduction to mpi4py by TACC]]
-  * [[http://jeremybejarano.zzl.org/MPIwithPython/index.html|A Python Introduction to Parallel Programming with MPI]] 
-  * Simple mpi4py examples from the [[http://mpi4py.scipy.org/docs/usrman/tutorial.html|mpi4py tutorial]] 
   * [[https://github.com/jbornschein/mpi4py-examples|More mpi4py examples]]   * [[https://github.com/jbornschein/mpi4py-examples|More mpi4py examples]]
  • software/mpi4py/mpi4py.1508795872.txt.gz
  • Last modified: 2017-10-23 17:57
  • by sraskar