This is an old revision of the document!
Using Matlab
MATLAB® is a high-level language and interactive environment for numerical computation, visualization, and programming. Using MATLAB, you can analyze data, develop algorithms, and create models and applications. The language, tools, and built-in math functions enable you to explore multiple approaches and reach a solution faster than with spreadsheets or traditional programming languages, such as C/C++ or Java™
See Computational model for running Matlab on a shared Cluster
License information
MATLAB is product licensed by Mathworks. The MATLAB product and related toolboxes are usually installed, on campus, to use the UD license server. When MATLAB is started a license token is checked out from the license server. Each toolbox function also requires the availability of a license token to start during a MATLAB session. The UD license server currently has tokens for core MATLAB and about 40 toolboxes.
The Mathworks R2013b and newer license server is being queried every 6 minutes on each cluster for available seats licensed for Matlab, Simulink and associated toolboxes.
Project directory
MATLAB is a language for scientific computing. A MATLAB project consists of a main program, called a MATLAB script, that when executed will read data, call MATLAB functions, and write data. This data may be in the form of text files, image files, MATLAB save files (MAT-file), or any of many standard file formats, e.g., HDF5. To keep your project portable you should put your scripts, functions and data in one directory. This directory is called the Current Folder on the MATLAB desktop.
MATLAB has tools to help you make your project portable, and you should use them. For example, you should specify all files using MATLAB file construction functions.
Your preferences at
MATLAB → General → MAT-files
may have a older MAT-file save format selected as the default. You should check to make sure you can save large files (greater than 2GB) for all versions of MATLAB where you may write .mat
files. There is another advantage to using the newest file format:
When you have a MATLAB job tested and debugged in a project directory, you are ready run on a cluster.
Copy the project folder
Copy the project folder to a directory on the cluster. Use any file transfer client to copy your entire project directory.
Batch job
See cluster details for running MATLAB as a batch application since each cluster will have different job templates based on the job scheduler such as Slurm or Grid-Engine. You should have a copy of your MATLAB project directory on the cluster.
MATLAB has a new version twice a year. It is important to keep the version you use on your desktop the same as the one on the cluster. The command
vpkg_versions matlab
will show you the versions available on a cluster. Choose the one that matches the version on your desktop. We recommend you do not upgrade MATLAB in the middle of a project, unless there is a new feature or bug fix you need.
It is frequently advisable to keep your MATLAB project clean from non-MATLAB files such as the queue script file and the script output file. But you may combine them, and even use the MATLAB editor to create the script file and look at the output file. If you create the file on a PC, take care to not transfer the files as binary. See Transfer Files for the appropriate cluster.
When you have one combined directory, do not put the cd
command in the queue script; instead, change
to the project directory using cd
on the command line, before submitting your job.
Create a queue script file
You should create a queue script file to submit a batch job. Start by modifying a job template file (/opt/templates
), for example, to submit a serial job on one core of a compute node, copy the serial template.
In your copy change the commented vpkg_require
command to
require MATLAB, and then add your shell commands to the end of the file. Your copy may contain the lines:
# Add vpkg_require commands after this line: vpkg_require matlab # Now append all of your shell commands necessary to run your program # after this line: cd project_directory matlab -nodisplay -singleCompThread -r main_script
The project_directory
should have a file named main_script.m
with your script. It could have just
one line display 'Hello World
'.
For more examples, including examples that use multiple computational threads or the Parallel toolbox, see the cluster details
Submit batch job
Your shell must be in a workgroup environment
to submit any jobs.
Use the qsub
command to submit a batch job
and note the «JOBID»
that is assigned to your job. For example, if you queue script file name is matlab_first.qs
,
submit the job with:
qsub matlab_first.qs
This is the message you get if you are not in workgroup. Choose a workgroup with the workgroup
command.
It is true that a queue script file is (usually) a bash script, but it must be executed with the qsub
command instead of the sh
command. This way the grid engine commands with be processed, and the job will be run on a compute node.
Wait for job to complete
You can check on the status of you job with the qstat
command.
For example, to list the information for job «JOBID»
, type
qstat -j <<JOBID>>
For long running jobs, you could change your queue script to notify you via an e-mail message when the job is complete.
Post process job
All MATLAB output data files will be in the project directory, but the MATLAB standard output will be in the current directory, from which you submitted the job. Look for a file ending in your assigned JOBID.
Interactive job
Here are specific details for running MATLAB as an interactive application on a compute node. You should have a copy of your MATLAB project directory on the cluster.
Command-line
You should work on a compute node when in command-line MATLAB.
Your shell must be in a workgroup environment
to submit a single threaded interactive job using qlogin
.
qlogin vpkg_require matlab cd project_directory matlab -nodesktop -singleCompThread
This will start a interactive command-line session in your terminal window. When done type the quit
or exit
to terminated the MATLAB session and then exit
to terminated the qlogin session.
Desktop
You should be on a compute node before you start MATLAB. To start a MATLAB desktop (GUI mode) on a cluster, you must be running an X11 server and you must have connected using X11 tunneling.
Your shell must be in a workgroup environment
to submit a job using qlogin
.
qlogin -l exclusive=1 vpkg_require matlab cd project_directory matlab
This will start a interactive DESKTOP session on you X11 screen. When done type the quit
or exit
in the command window or just close the window. When back at the terminal bash prompt, type exit
to terminate the qlogin session.
See tips on starting Matlab in an interactive session without the desktop, including executing a script.
Compiling with MATLAB
We show the three most common ways to work with compilers when using MATLAB.
- Compiling your matlab code to run in the MCR (Matlab Compiler Runtime)
- Compiling your C or Fortran program to call MATLAB engine.
- Compiling your own function in C or Fortran to be used in a MATLAB session.
Warning: You are using gcc version '4.9.3'. The version currently supported with MEX is '4.7.x'. For a list of currently supported compilers see: http://www.mathworks.com/support/compilers/current_release.
But the compilation completes successfully.
Compiling your MATLAB
There is an example MCR project in the /opt/templates/
directory for you to copy and try. Copy on the head node and qlogin to compile with MATLAB. Once your program is compiled you can run it interactively or in batch, without needing a MATLAB license.
Copy dev-projects template
On the head node
cp -r /opt/templates/dev-projects/MCR . cd MCR
Compile with make
Now compile on the compute node by using
qlogin make
qlogin
Resulting output from the make command:
Adding package `mcr/r2014b-nojvm` to your environment make[1]: Entering directory `/home/work/it_css/traine/matlab/MCR' mcc -o maxEig -R "-nojvm,-nodesktop,-singleCompThread" -mv maxEig.m Compiler version: 5.2 (R2014b) Dependency analysis by REQUIREMENTS. Parsing file "/home/work/it_css/traine/matlab/MCR/maxEig.m" (Referenced from: "Compiler Command Line"). Deleting 0 temporary MEX authorization files. Generating file "/home/work/it_css/traine/matlab/MCR/readme.txt". Generating file "run_maxEig.sh". make[1]: Leaving directory `/home/work/it_css/traine/matlab/MCR'
Take note of the package added, and the files that are generated. You can remove these files, as they are not needed. You must add the package in your batch script or to test interactively.
test interactively
To test interactively on the same compute node.
vpkg_require mcr/r2014b-nojvm time ./maxEig 20.8
back to the head node
When done, exit the compute node.
exit
Copy array job example
cp /opt/templates/gridengine/matlab-mcr.qs . vi matlab-mcr.qs diff /opt/templates/gridengine/matlab-mcr.qs matlab-mcr.qs
The diff
output shows changes made in the vi
session:
46c46 < # -l m_mem_free=5G --- > #$ -l m_mem_free=3G 51c51 < # -t 1-4 --- > #$ -t 1-100 63c63,64 < vpkg_require mcr/r2014b-nojvm --- > vpkg_require mcr/r2015a-nojvm > let lambda="$SGE_TASK_ID-1" 79c80 < MCR_EXECUTABLE_FLAGS=("$RANDOM") --- > MCR_EXECUTABLE_FLAGS=("$lambda")
To submit a standby array job that has 100 tasks.
qsub -l standby=1 matlab-mcr.qs
Example
[(it_css:traine)@farber MCR]$ qsub -l standby=1 matlab-mcr.qs Your job-array 627074.1-100:1 ("matlab-mcr.qs") has been submitted [(it_css:traine)@farber MCR]$ date Mon Apr 11 14:56:26 EDT 2016 [(it_css:traine)@farber MCR]$ date Mon Apr 11 15:17:33 EDT 2016 [(it_css:traine)@farber MCR]$ ls -l matlab-mcr.qs.o627074.* | wc -l 100
There are 100 output files with the names matlab-mcr.qs.o627074.1 to matlab-mcr.qs.o627074.100 For example file 50:
[CGROUPS] UD Grid Engine cgroup setup commencing [CGROUPS] Setting 3221225472 bytes (vmem none bytes) on n106 (master) [CGROUPS] with 1 core = [CGROUPS] done. Adding package `mcr/r2015a-nojvm` to your environment GridEngine parameters: MCR_ROOT = /opt/shared/matlab/r2015a MCR executable = /home/work/it_css/traine/matlab/MCR/maxEig flags = 49 MCR_CACHE_ROOT = /tmp/627074.50.standby.q -- begin maxEig run -- maxe = 5.0243e+03 -- end maxEig run --
Compiling your code to use MATLAB engine
There is an simple example function fengdemo.F
coded in Fortran, you can copy and use as a starting point.
On the head node and in a workgroup shell:
vpkg_require matlab/r2015a gcc/4.9 cp $MATLABROOT/extern/examples/eng_mat/fengdemo.F . export LD_LIBRARY_PATH=$MATLABROOT/bin/glnxa64:$MATLABROOT/sys/os/glnx64:$LD_LIBRARY_PATH mex -client engine fengdemo.F
To start MATLAB on a compute node to test this new program:
qlogin vpkg_require matlab/r2015a gcc/4.9 export LD_LIBRARY_PATH=$MATLABROOT/bin/glnxa64:$MATLABROOT/sys/os/glnx64:$LD_LIBRARY_PATH ./fengdemo exit
Step one of the fengdemo should give the plot:
Step two should give the table:
MATLAB computed the following distances: time(s) distance(m) 1.00 -4.90 2.00 -19.6 3.00 -44.1 4.00 -78.4 5.00 -123. 6.00 -176. 7.00 -240. 8.00 -314. 9.00 -397. 10.0 -490.
Compiling your own MATLAB function
There is an simple example function timestwo.c
, coded in c, you can copy and use as a starting point.
On the head node and in a workgroup shell:
vpkg_require matlab/r2015a gcc/4.9 cp $MATLABROOT/extern/examples/refbook/timestwo.c . mex timestwo.c
To start MATLAB on a compute node to test this new function:
qlogin vpkg_require matlab/r2015a gcc/4.9 matlab -nodesktop timestwo(4) quit exit
You should get the answer
>> timestwo(4) ans = 8 >>
Parallel job
You can use multicore or multiprocessor MATLAB for your project. There are three ways do this using the cluster scheduler and/or MATLAB toolboxes.
Array job
Multiple, independent, single-threaded MATLAB scripts can be run on the same node, or different nodes. The only communication between the jobs is through reading and writing files. Your script should be compiled using mcc
so it can be deployed on the cluster to run in a MATLAB Runtime environment.
Multiple computational threads
MATLAB will detect you are on a multicore host and use all the cores it can to speed up its internal calculations. Use MATLAB builtin functions to take advantage of these multiple computational threads. You should scedule a computing node with exclusive access, so your job does not interfere with other jobs on the node.
Parallel computing toolbox
Using the MATLAB Parallel Computing Toolbox you can create a pool of workers and take control of the parallelism in your algorithm using MATLAB functions such as parfor
. You can run this on one node using the default local scheduler, but the real benefit comes with a separate Mathworks product called the MATLAB Distributed Computing Server