software:matlab:farber

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
software:matlab:farber [2019-08-29 14:38] – [Interactive job] anitasoftware:matlab:farber [2021-04-27 16:21] (current) – external edit 127.0.0.1
Line 2: Line 2:
  
 For use on Farber, MATLAB projects should be developed using a Desktop installation of MATLAB and then copied to Farber For use on Farber, MATLAB projects should be developed using a Desktop installation of MATLAB and then copied to Farber
-to be run in batch.  Here an extended MATLAB example is considered involving one simple MATLAB function, and two MATLAB scripts to executed this function in a loop, and to execute in parallel using the Parallel Toolbox.+to be run in batch.  Here an extended MATLAB example is considered involving one simple MATLAB function, and two MATLAB scripts to execute this function in a loop, and another to execute in parallel using the Parallel Toolbox.
  
 Details on how to run these two scripts in batch are given with the resulting output files.  There is also a Details on how to run these two scripts in batch are given with the resulting output files.  There is also a
-section with UNIX commands you can use to watch your jobs and gather [[#timing and core count]] numbers.+section with UNIX commands you can use to watch your jobs and gather [[#timings-and-core-count | timing and core count]] numbers.
 It is important to know how much memory with be needed and how many cores will be used to set your resource requirements. If you do not ask for enough memory your job will fail.  If you do not ask for enough cores, the job will take longer. It is important to know how much memory with be needed and how many cores will be used to set your resource requirements. If you do not ask for enough memory your job will fail.  If you do not ask for enough cores, the job will take longer.
  
Line 12: Line 12:
 second example shows an interactive session, which starts multiple MATLAB pool of workers to execute the function as a parallel toolbox loop, **''parfor''**. The Parallel toolbox gives a faster time to completion, but with more memory and CPU resources consumed. second example shows an interactive session, which starts multiple MATLAB pool of workers to execute the function as a parallel toolbox loop, **''parfor''**. The Parallel toolbox gives a faster time to completion, but with more memory and CPU resources consumed.
  
-You can run [[#desktop |MATLAB as a desktop (GUI)]] application on Farber. +You can run [[#desktop |MATLAB as a desktop (GUI)]] application on Farber, but is not recommended as the graphics is slow to display especially with a slower network connection. 
  
 Many MATLAB research projects fall in the the "high throughput computing" category.  One run can be done on the desktop, but it is desired complete 100s or 1000s of independent runs.  This greatly increases disk, memory and CPU requirements.  Many MATLAB research projects fall in the the "high throughput computing" category.  One run can be done on the desktop, but it is desired complete 100s or 1000s of independent runs.  This greatly increases disk, memory and CPU requirements. 
Line 165: Line 165:
 end end
 </code> </code>
-See [[maxNumCompThreads|Setting maximum number of computational threads]]</note>+See [[maxNumCompThreadsGridEngine|Setting maximum number of computational threads]]</note>
  
 <note tip> <note tip>
Line 206: Line 206:
  
 ===== Create a job script file ===== ===== Create a job script file =====
-You should create a job script file to submit a batch job. Start by modifying a job template file (''/opt/templates/gridengine''), for example, to submit a serial job on one core of a compute node, copy the serial template. +You should create a job script file to submit a batch job. Start by modifying a job template file (''/opt/shared/templates/gridengine''), for example, to submit a serial job on one core of a compute node, copy the serial template. 
 In your copy change the commented ''vpkg_require'' command to  In your copy change the commented ''vpkg_require'' command to 
 require MATLAB, and then add your shell commands to the end of the file. Your copy may contain the lines: require MATLAB, and then add your shell commands to the end of the file. Your copy may contain the lines:
Line 318: Line 318:
 ===== Compiling your MATLAB ===== ===== Compiling your MATLAB =====
  
-There is an example MCR project in the ''/opt/templates/'' directory for you to copy and try.  Copy on the head node and qlogin to compile with MATLAB.  Once your program is compiled you can run it interactively or in batch, without needing a MATLAB license.+There is an example MCR project in the ''/opt/shared/templates/'' directory for you to copy and try.  Copy on the head node and qlogin to compile with MATLAB.  Once your program is compiled you can run it interactively or in batch, without needing a MATLAB license.
  
 ==== Copy dev-projects template ==== ==== Copy dev-projects template ====
Line 324: Line 324:
 On the head node On the head node
 <code> <code>
-cp -r /opt/templates/dev-projects/MCR .+cp -r /opt/shared/templates/dev-projects/MCR .
 cd MCR cd MCR
 </code> </code>
Line 376: Line 376:
  
 <code> <code>
-cp /opt/templates/gridengine/matlab-mcr.qs .+cp /opt/shared/templates/gridengine/matlab-mcr.qs .
 vi matlab-mcr.qs vi matlab-mcr.qs
-diff /opt/templates/gridengine/matlab-mcr.qs matlab-mcr.qs +diff /opt/shared/templates/gridengine/matlab-mcr.qs matlab-mcr.qs 
 </code> </code>
 The ''diff'' output shows changes made in the ''vi'' session: The ''diff'' output shows changes made in the ''vi'' session:
Line 522: Line 522:
 </code> </code>
  
-==== Grid Engine script ====+====== Batch job serial example ======
  
 Second, write a shell script file to set the Matlab environment and start Matlab running your script file. The following script file will set the Matlab environment and run the command in the [[#matlab-script|script.m]] file: Second, write a shell script file to set the Matlab environment and start Matlab running your script file. The following script file will set the Matlab environment and run the command in the [[#matlab-script|script.m]] file:
Line 547: Line 547:
 If everyone in your group carefully set these values, multiply jobs can run concurrently on the node. If everyone in your group carefully set these values, multiply jobs can run concurrently on the node.
  
-See [[maxNumCompThreads|Setting maximum number of computational threads]]+See [[maxNumCompThreadsGridEngine|Setting maximum number of computational threads]]
  
 </note> </note>
Line 585: Line 585:
 </code> </code>
  
-<note warning>You should specify required [[#matlab-license-information-for-grid-engine|Matlab licenses for Grid-Engine]] as a resource, especially if there are limited number of license seats available for particular toolboxes.+<note warning>You should specify required [[#matlab-license-information-for-grid-engine|Matlab licenses for GridEngine]] as a resource, especially if there are limited number of license seats available for particular toolboxes.
  
 In this example you will only need a license for the base Matlab, and the parallel toolbox needs one license.  We are using the default local scheduler which will give you workers on the same node with one license. In this example you will only need a license for the base Matlab, and the parallel toolbox needs one license.  We are using the default local scheduler which will give you workers on the same node with one license.
Line 747: Line 747:
  
  
-===== Batch parallel example =====+====== Batch parallel example ======
  
 The Matlab parallel toolbox uses JVM to manage the workers and communicate while you are running.  You The Matlab parallel toolbox uses JVM to manage the workers and communicate while you are running.  You
Line 815: Line 815:
  
  
-===== Interactive example =====+====== Interactive example ======
  
 The basic steps to running a [[:software:matlab#interactive-job|MATLAB]] interactively on a compute node.  The basic steps to running a [[:software:matlab#interactive-job|MATLAB]] interactively on a compute node. 
Line 953: Line 953:
  
  
-===== MCR array job example =====+====== MCR array job example ======
  
 Most Matlab functions can be compiled using the Matlab Compiler (MCC) and then deployed to run on the compute nodes in the MATLAB Compiler Runtime (MCR).  The MCR is a prerequisite for deployment, and is installed on all the compute nodes. You must use VALET to set up the libraries you will need to run your function from the command line.  You do not need to use the shell (''.sh'' file) that the compiler creates. Most Matlab functions can be compiled using the Matlab Compiler (MCC) and then deployed to run on the compute nodes in the MATLAB Compiler Runtime (MCR).  The MCR is a prerequisite for deployment, and is installed on all the compute nodes. You must use VALET to set up the libraries you will need to run your function from the command line.  You do not need to use the shell (''.sh'' file) that the compiler creates.
Line 1034: Line 1034:
  
 The ''mcc'' command will generate a ''.sh'' file, which you can use to setup your environment and run the command.  This does not use VALET and does not have any grid engine commands in it.  We suggest you the gridengine template in the file  The ''mcc'' command will generate a ''.sh'' file, which you can use to setup your environment and run the command.  This does not use VALET and does not have any grid engine commands in it.  We suggest you the gridengine template in the file 
-  /opt/templates/gridengine/matlab-mcr.qs+  /opt/shared/templates/gridengine/matlab-mcr.qs
 or modify this simple example: or modify this simple example:
 <code> <code>
  • software/matlab/farber.1567103897.txt.gz
  • Last modified: 2019-08-29 14:38
  • by anita