hpc documentation

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Matplotlib ======

[[http://matplotlib.org/index.html|Matplotlib]] is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. matplotlib can be used in python scripts, the python and [[http://ipython.org/ipython|ipython]] shell (ala MATLAB®* or Mathematica®†), web application servers, and six graphical user interface toolkits.

Matplotlib, pylab and pyplot are all related.  Matplotlib is the whole package; pylab is a module in matplotlib that gets installed alongside [[http://matplotlib.org/api/matplotlib_configuration_api.html#module-matplotlib|matplotlib]]; and [[http://matplotlib.org/api/pyplot_api.html#module-matplotlib.pyplot|matplotlib.pyplot]] is a module in matplotlib. There is a [[http://matplotlib.org/users/pyplot_tutorial.html|pyplot tutorial]] and numerous other sites that use matplotlib for their plot examples such as [[http://www.scipy.org/Cookbook/Matplotlib/|SciPy Cookbook for Matplotlib]].

<note tip>These directions are geared towards setting up and running Matplotlib on Caviness. However these directions can easily be used on DARWIN, although references to directory paths and software versions may be different.</note>
===== Interactive job =====

All interactive jobs should be run on a compute node by setting your ''workgroup'' and using ''salloc'' and the proper X11 settings since you need to be running X-Windows to produce graphics. Also make sure you are in the directory where your input and output files will be stored.  In the example below, we are using the ''traine'' account in workgroup ''it-css'' to create the ''plot123.png'' file using Python packages Matplotlib and using the Intel version of Python since it comes with the packages we need without having to create our own Python virtual environment.

<code bash>
[traine@login01.caviness ~]$ workgroup -g it_css
[(it_css:traine)@login01.caviness ~]$ salloc --partition=it_css --x11 -N1 -n1
salloc: Granted job allocation 26918710
salloc: Waiting for resource configuration
salloc: Nodes r00n45 are ready for job
[(it_css:traine)@r00n45 ~]$ vpkg_require intel-python
Adding package `intel-python/2022u1:python3` to your environment
[(it_css:traine)@r00n45 ~]$ python
Python 3.9.10 (main, Mar  2 2022, 12:02:00)
[GCC 9.3.0] :: Intel Corporation on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import scipy
>>> import matplotlib
>>> from pylab import *
>>> plot([1,2,3])
[<matplotlib.lines.Line2D object at 0x2b9e83d2bee0>]
>>> show()
>>> quit()
[(it_css:traine)@r00n45 ~]$ ls *.png
plot123.png
</code>

After typing ''show()'' in the example above, a ''Figure'' window should appear displaying the plot. Now click on the ''Save the figure'' icon at the bottom of the ''Figure'' window. A ''Save the Figure'' window appears with the default file name
''image.png'' for the file format type Portable Network Graphics (png). In this example, we changed ''image'' to ''plot123'' and the plot file ''plot123.png'' was saved in the current working directory. If you want a different file format, use the pull-down menu at the bottom of the ''Save the Figure'' window and this will automatically change the file name extension. When you close the ''Figure'' window it returns you back to Python.

===== Batch job =====

If you need to create many plots, then you will want to use a non-interactive method by specifying the file format with ''matplotlib.use()'' and saving each plot using ''savefig()''. In the example below we are only generating one plot by specifying ''AGG'' to use Portable Network Graphics (png) file format and ''savefig('plot-cs.png')'' to save this one plot instead of using ''show()''. 

<code - plot-ex.py>
# Python program to generate plots in batch

import matplotlib
matplotlib.use("AGG")

from pylab import *

X = np.linspace(-np.pi, np.pi, 256,endpoint=True)
C,S = np.cos(X), np.sin(X)

plot(X,C)
plot(X,S)

# show()
savefig('plot-cs.png')
</code>
 
Now we can use this Python plot example file and run it using a basic serial job script.  Copy the job script template from ''/opt/shared/templates/slurm/generic/serial.qs'' and call it ''serial-plot.qs''. Modify ''serial-plot.qs'' to set the correct partition, add the correct packages with VALET ''vpkg_require intel-python'' and replace ''srun date'' with ''python plot-ex.py''. Remember ''_workgroup_'' can be used for the partition because in order to submit jobs, the workgroup must be set using the ''workgroup'' command, so whatever workgroup has been set will be substituted in for ''_workgroup''.  This allows our job scripts to be more flexible when you have more than one workgroup on the cluster.
 
<code - serial-plot.qs>
#
#        To run with priority-access to resources owned by your workgroup,
#        use the "_workgroup_" partition:
#
#SBATCH --partition=_workgroup_

#
# If you have VALET packages to load into the job environment,
# uncomment and edit the following line:
#
#vpkg_require intel/2019
vpkg_require intel-python

#
# [EDIT] Add your script statements hereafter, or execute a script or program
#        using the srun command.
#
#srun date
python plot-ex.py

</code>
 
Now we can submit our batch run by using

<code bash>
sbatch serial-plot.qs
</code>
 
This will create the plot file ''plot-cs.png'' in the current working directory where you submitted the job. 

<code bash>
[(it_css:traine)@login01.caviness matplotlib]$ ls
plot-cs.png  plot-ex.py  serial-plot.qs  slurm-26918733.out
</code>

==== View plots with Imagemagick ====

If you want to review the plots created, then you can use Imagemagick on the compute nodes via ''salloc'' with the proper X11 options and you are running X-Windows. For example, using account ''traine'' and workgroup ''it_css'', we display the ''plot-cs.png'' plot file. 
 
<code bash>
[(it_css:traine)@login01.caviness matplotlib]$ salloc --partition=it_css --x11 -N1 -n1
salloc: Granted job allocation 26918735
salloc: Waiting for resource configuration
salloc: Nodes r04n16 are ready for job
[(it_css:traine)@r04n16 matplotlib$ vpkg_require imagemagick
Adding dependency `fftw/3.3.8` to your environment
Adding dependency `ghostscript/9.23` to your environment
Adding package `imagemagick/7.0.8-3` to your environment
[(it_css:traine)@r04n16 matplotlib]$ display plot-cs.png
</code>

==== Using LaTeX in plot text ====

With some minor modifications from [[http://matplotlib.org/1.3.1/users/usetex.html| Text rendering with LaTeX - Matplotlib]], we can create a plot called ''latex_demo.png'' with labels rendered by LaTeX.

<code - latex_demo.py>
import numpy as np
import matplotlib
matplotlib.use("AGG")
import matplotlib.pyplot as plt


# Example data
t = np.arange(0.0, 1.0 + 0.01, 0.01)
s = np.cos(4 * np.pi * t) + 2

plt.rc('text', usetex=True)
plt.rc('font', family='serif')
plt.plot(t, s)

plt.xlabel(r'\textbf{time} (s)')
plt.ylabel(r'\textit{voltage} (mV)',fontsize=16)
plt.title(r"\TeX\ is Number "
          r"$\displaystyle\sum_{n=1}^\infty\frac{-e^{i\pi}}{2^n}$!",
          fontsize=16, color='gray')
# Make room for the ridiculously large title.
plt.subplots_adjust(top=0.8)

plt.savefig('latex_demo')
</code>

Using a basic serial job script like ''serial-plot.qs'' copy it and make changes below to set the partition, include the appropriate VALET packages and the command to run Python, then use ''sbatch serial-plot-latex.qs'' to submit the job. This job will create the plot file ''latex_demo.png'' in the current working directory where you submitted the job.

<code - serial-plot-latex.qs>
#
#        To run with priority-access to resources owned by your workgroup,
#        use the "_workgroup_" partition:
#
#SBATCH --partition=_workgroup_

#
# If you have VALET packages to load into the job environment,
# uncomment and edit the following line:
#
#vpkg_require intel/2019
vpkg_require intel-python texlive

#
# [EDIT] Add your script statements hereafter, or execute a script or program
#        using the srun command.
#
#srun date

python latex_demo.py
</code>