software:matlab:matlab

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
software:matlab:matlab [2019-08-27 14:32] – [Batch job] anitasoftware:matlab:matlab [2021-02-24 16:37] (current) – [Details by cluster] anita
Line 8: Line 8:
 ====== License information ====== ====== License information ======
  
-MATLAB is product licensed by Mathworks.  The MATLAB product and related toolboxes are usually installed, on campus, to use the UD license server.  When MATLAB is started a license token is checked out from the license server. Each toolbox function also requires the availability of a license token to start during a MATLAB session. The UD license server currently has tokens for [[central licenses|core MATLAB and about 40 toolboxes]].+MATLAB is product licensed by Mathworks.  The MATLAB product and related toolboxes are usually installed requiring access to the UD license server and requires a **persistent network connection** as well as [[http://www.udel.edu/it/help/connecting/vpn/|UD VPN]] when accessing off-campus. When MATLAB is started a license token is checked out from the license server. Each toolbox function also requires the availability of a license token to start during a MATLAB session. The UD license server currently has tokens (number of licenses) for [[central licenses|core MATLAB and about 50 toolboxes]].
  
 The Mathworks R2013b and newer license server is being queried every 6 minutes on each cluster for available seats licensed for Matlab, Simulink and associated toolboxes. The Mathworks R2013b and newer license server is being queried every 6 minutes on each cluster for available seats licensed for Matlab, Simulink and associated toolboxes.
Line 32: Line 32:
 When you have a MATLAB job tested and debugged in a project directory, you are ready run on a cluster. When you have a MATLAB job tested and debugged in a project directory, you are ready run on a cluster.
  
-===== Copy the project folder ===== +See [[#details-by-clusters|cluster details]] for examples on running your Matlab job on a particular cluster.
- +
-Copy the project folder to a directory on the cluster. +
-Use  any [[:abstract:farber:transfer|file transfer client]] to copy your entire project directory. +
- +
-====== Batch job====== +
- +
-See [[#cluster details]] for running MATLAB as a batch application since each cluster will have different job templates based on the job scheduler such as Slurm or Grid-Engine.  You should have a copy of your MATLAB [[#project directory]] on the cluster. +
- +
-<note important>**Versions of MATLAB** +
- +
-MATLAB has a new version twice a year.  It is important to keep the version you use on your desktop the same as the +
-one on the cluster.  The command +
-<code> +
-vpkg_versions matlab +
-</code> +
-will show you the versions available on a cluster.  Choose the one that matches the version on your desktop.  We recommend you do not upgrade MATLAB in the middle of a project, unless there is a new feature or bug fix you need. +
-</note> +
- +
-<note tip>**Two directories** +
- +
-It is frequently advisable to keep your MATLAB project clean from non-MATLAB files such as the queue  +
-script file and the script output file.  But you may combine them, and even use the MATLAB editor to  +
-create the script file and look at the output file.   +
-If you create the file on a PC, take care to not transfer the files as binary. See Transfer Files for the appropriate cluster. +
- +
-When you have one combined directory, do not put the ''cd'' command in the queue script; instead, change +
-to the project directory using ''cd'' on the command line, before submitting your job. +
-</note> +
- +
- +
-===== Create a queue script file ===== +
-You should create a queue script file to submit a batch job. Start by modifying a job template file (''/opt/templates''), for example, to submit a serial job on one core of a compute node, copy the serial template.  +
-In your copy change the commented ''vpkg_require'' command to  +
-require MATLAB, and then add your shell commands to the end of the file. Your copy may contain the lines: +
-<code> +
-# Add vpkg_require commands after this line: +
-vpkg_require matlab +
- +
-# Now append all of your shell commands necessary to run your program +
-# after this line: +
-cd project_directory +
-matlab -nodisplay -singleCompThread -r main_script +
-</code> +
-The ''project_directory'' should have a file named ''main_script.m'' with your script.  It could have just +
-one line **''display 'Hello World'''**. +
- +
- +
-<note tip>**Toolbox dependencies** +
- +
-You should include toolbox dependencies in your batch script.  This documents the toolboxes you +
-need to run the MATLAB script, and it will avoid a failure, which will occur if the job starts with no [[matlab#license-information|licenses]] available. +
- +
-For example, the Bioinformatics toolbox only has one seat, and in addition it requires the Statistics and Machine Learning toolbox, as well as the core MATLAB.  So add the line: +
-<code> +
-#$ -l MLM.MATLAB=1,MLM.Statistics_Toolbox=1,MLM.Bioinformatics_Toolbox=1 +
-</code> +
- +
-</note> +
- +
-For more examples, including examples that use multiple computational threads or the Parallel toolbox, see the [[#cluster details]] +
-===== Submit batch job ===== +
-Your shell must be in a [[general:userguide:04_compute_environ?&#using-workgroup-and-directories|workgroup environment]] +
-to submit any jobs. +
-Use the ''qsub'' command to submit a [[general:jobsched/grid-engine/30_batch|batch job]] +
-and note the ''<<JOBID>>'' that is assigned to your job.  For example, if you queue script file name is ''matlab_first.qs'', +
-submit the job with: +
-<code> +
-qsub matlab_first.qs +
-</code> +
- +
-<note important>**WARNING:  Please choose a workgroup before submitting jobs** +
- +
-This is the message you get if you are not in workgroup.  Choose a workgroup with the ''workgroup'' command. +
-</note> +
- +
-<note warning>**Bash script vs queue script** +
- +
-It is true that a queue script file is (usually) a bash script, but it must be executed with the ''qsub'' command instead of the ''sh'' command.  This way the grid engine commands with be processed, and the job will be run on a compute node. +
-</note> +
- +
-===== Wait for job to complete ===== +
-You can [[general:userguide:06_runtime_environ?&#checking-job-status|check on the status]] of you job with the ''qstat'' command.  +
-For example, to list the information for job ''<<JOBID>>'', type +
-<code> +
-qstat -j <<JOBID>> +
-</code> +
- +
-For long running jobs, you could change your queue script to notify you via an e-mail message when the job is  +
-complete. +
-===== Post process job ===== +
-All MATLAB output data files will be in the project directory, but the MATLAB standard output will be in  +
-the current directory, from which you submitted the job.  Look for a file ending in your assigned JOBID. +
- +
- +
- +
- +
-  +
-====== Interactive job ====== +
- +
-Here are specific details for running MATLAB as an [[general:userguide:06_runtime_environ#submitting-interactive-jobs-qlogin|interactive application]] on a compute node.  You should have a copy of your [[#MATLAB project directory]] on the cluster. +
- +
-===== Command-line ===== +
- +
-You should work on a compute node when in command-line MATLAB. +
-Your shell must be in a [[general:userguide:04_compute_environ?&#using-workgroup-and-directories|workgroup environment]] +
-to submit a single threaded interactive job using ''qlogin''. +
- +
-<code> +
-qlogin +
-vpkg_require matlab +
-cd project_directory +
-matlab -nodesktop -singleCompThread +
-</code> +
- +
-This will start a interactive command-line session in your terminal window.  When done type the ''quit'' or ''exit'' to terminated the MATLAB session and then ''exit'' to terminated the qlogin session. +
- +
-===== Desktop ===== +
- +
-You should be on a compute node before you start MATLAB. +
-To start a MATLAB desktop (GUI mode) on a cluster, you must be running an X11 server and you must have  +
-[[general:userguide:02_access| connected using +
-X11 tunneling]]. +
- +
-Your shell must be in a [[general:userguide:04_compute_environ?&#using-workgroup-and-directories|workgroup environment]] +
-to submit a job using ''qlogin''. +
- +
-<code> +
-qlogin -l exclusive=1 +
-vpkg_require matlab +
-cd project_directory +
-matlab +
-</code> +
- +
-This will start a interactive DESKTOP session on you X11 screen.  When done type the ''quit'' or ''exit'' in the command window or just close the window. When back at the terminal bash prompt, type ''exit'' to terminate the qlogin session. +
- +
- +
-See [[:software:matlab:interactive:tips|tips on starting Matlab]] in an interactive session without the desktop, including executing a script. +
- +
-====== Compiling with MATLAB ====== +
- +
-We show the three most common ways to work with compilers when using MATLAB. +
- +
-  - Compiling your matlab code to run in the MCR (Matlab Compiler Runtime) +
-  - Compiling your C or Fortran program to call MATLAB engine. +
-  - Compiling your own function in C or Fortran to be used in a MATLAB session. +
- +
-<note>Make sure your compiler is newer than the one one required by your MATLAB version. In these examples MATLAB requires gcc 4.7 or newer. You may get the Warning: +
-<code> +
-Warning: You are using gcc version '4.9.3'. The version currently supported  +
-with MEX is '4.7.x'. For a list of currently supported compilers see:  +
-http://www.mathworks.com/support/compilers/current_release. +
-</code> +
-But the compilation completes successfully. +
-</note> +
- +
-===== Compiling your MATLAB ===== +
- +
-There is an example MCR project in the ''/opt/templates/'' directory for you to copy and try.  Copy on the head node and qlogin to compile with MATLAB.  Once your program is compiled you can run it interactively or in batch, without needing a MATLAB license. +
- +
-==== Copy dev-projects template ==== +
- +
-On the head node +
-<code> +
-cp -r /opt/templates/dev-projects/MCR . +
-cd MCR +
-</code> +
- +
-==== Compile with make ==== +
- +
-Now compile on the compute node by using +
- +
-<code> +
-qlogin +
-make +
-</code> +
- +
-<note>Remember you must be in a workgroup before using ''qlogin''  +
-</note> +
- +
-Resulting output from the make command: +
-<code> +
-Adding package `mcr/r2014b-nojvm` to your environment +
-make[1]: Entering directory `/home/work/it_css/traine/matlab/MCR' +
-mcc -o maxEig -R "-nojvm,-nodesktop,-singleCompThread" -mv maxEig.m +
-Compiler version: 5.2 (R2014b) +
-Dependency analysis by REQUIREMENTS. +
-Parsing file "/home/work/it_css/traine/matlab/MCR/maxEig.m" +
- (Referenced from: "Compiler Command Line"). +
-Deleting 0 temporary MEX authorization files. +
-Generating file "/home/work/it_css/traine/matlab/MCR/readme.txt". +
-Generating file "run_maxEig.sh". +
-make[1]: Leaving directory `/home/work/it_css/traine/matlab/MCR' +
-</code> +
-Take note of the package added, and the files that are generated.  You can remove these files, as they are not needed. +
-You must add the package in your batch script or to test interactively. +
- +
-==== test interactively ==== +
- +
-To test interactively on the same compute node. +
-<code> +
-vpkg_require mcr/r2014b-nojvm +
-time ./maxEig 20.8 +
-</code> +
-<note tip>This example is designed as a test for batch computing, and takes about 15 minutes to complete. If you +
-change the MATLAB statement dim=10000 to dim=1000, and recompile, it will take about 10 seconds</note> +
- +
-==== back to the head node ==== +
-When done, exit the compute node. +
-<code> +
-exit +
-</code> +
- +
-==== Copy array job example ==== +
- +
-<code> +
-cp /opt/templates/gridengine/matlab-mcr.qs . +
-vi matlab-mcr.qs +
-diff /opt/templates/gridengine/matlab-mcr.qs matlab-mcr.qs  +
-</code> +
-The ''diff'' output shows changes made in the ''vi'' session: +
-<code> +
-46c46 +
-< # -l m_mem_free=5G +
---- +
-> #$ -l m_mem_free=3G +
-51c51 +
-< # -t 1-4 +
---- +
-> #$ -t 1-100 +
-63c63,64 +
-< vpkg_require mcr/r2014b-nojvm +
---- +
-> vpkg_require mcr/r2015a-nojvm +
-> let lambda="$SGE_TASK_ID-1" +
-79c80 +
-< MCR_EXECUTABLE_FLAGS=("$RANDOM") +
---- +
-> MCR_EXECUTABLE_FLAGS=("$lambda") +
-</code> +
-To submit a standby array job that has 100 tasks. +
-<code> +
-qsub -l standby=1 matlab-mcr.qs +
-</code> +
- +
-Example +
-<code> +
-[(it_css:traine)@farber MCR]$ qsub -l standby=1 matlab-mcr.qs  +
-Your job-array 627074.1-100:1 ("matlab-mcr.qs") has been submitted +
-[(it_css:traine)@farber MCR]$ date +
-Mon Apr 11 14:56:26 EDT 2016 +
-[(it_css:traine)@farber MCR]$ date +
-Mon Apr 11 15:17:33 EDT 2016 +
-[(it_css:traine)@farber MCR]$ ls -l matlab-mcr.qs.o627074.* | wc -l +
-100 +
-</code> +
-There are 100 output files with the names matlab-mcr.qs.o627074.1 to matlab-mcr.qs.o627074.100 +
-For example file 50: +
-<code> +
-[CGROUPS] UD Grid Engine cgroup setup commencing +
-[CGROUPS] Setting 3221225472 bytes (vmem none bytes) on n106 (master) +
-[CGROUPS]   with 1 core =  +
-[CGROUPS] done. +
- +
-Adding package `mcr/r2015a-nojvm` to your environment +
-GridEngine parameters: +
-          MCR_ROOT = /opt/shared/matlab/r2015a +
-    MCR executable = /home/work/it_css/traine/matlab/MCR/maxEig +
-             flags = 49 +
-    MCR_CACHE_ROOT = /tmp/627074.50.standby.q +
--- begin maxEig run -- +
- +
-maxe = +
- +
-   5.0243e+03 +
- +
--- end maxEig run -- +
-</code> +
- +
- +
-[[more examples]] +
- +
- +
-===== Compiling your code to use MATLAB engine ====== +
- +
-There is an simple example function ''**fengdemo.F**'' coded in Fortran, you can copy and use as a starting point. +
- +
-On the head node and in a workgroup shell: +
- +
-<code> +
-vpkg_require matlab/r2015a gcc/4.9 +
-cp $MATLABROOT/extern/examples/eng_mat/fengdemo.F . +
-export LD_LIBRARY_PATH=$MATLABROOT/bin/glnxa64:$MATLABROOT/sys/os/glnx64:$LD_LIBRARY_PATH +
-mex -client engine fengdemo.F +
-</code> +
- +
-To start MATLAB on a compute node to test this new program: +
- +
-<code> +
-qlogin +
-vpkg_require matlab/r2015a gcc/4.9 +
-export LD_LIBRARY_PATH=$MATLABROOT/bin/glnxa64:$MATLABROOT/sys/os/glnx64:$LD_LIBRARY_PATH +
-./fengdemo +
-exit +
-</code> +
- +
-Step one of the fengdemo should give the plot: +
- +
-{{:software:figure_1.png?400|}} +
- +
-Step two should give the table: +
- +
-<code> +
- MATLAB computed the following distances: +
-   time(s)  distance(m) +
-   1.00     -4.90     +
-   2.00     -19.6     +
-   3.00     -44.1     +
-   4.00     -78.4     +
-   5.00     -123.     +
-   6.00     -176.     +
-   7.00     -240.     +
-   8.00     -314.     +
-   9.00     -397.     +
-   10.0     -490.     +
-</code> +
- +
-===== Compiling your own MATLAB function ====== +
- +
-There is an simple example function ''**timestwo.c**'', coded in c, you can copy and use as a starting point. +
- +
-On the head node and in a workgroup shell: +
- +
-<code> +
-vpkg_require matlab/r2015a gcc/4.9 +
-cp $MATLABROOT/extern/examples/refbook/timestwo.c . +
-mex timestwo.c +
-</code> +
- +
-To start MATLAB on a compute node to test this new function: +
- +
-<code> +
-qlogin +
-vpkg_require matlab/r2015a gcc/4.9 +
-matlab -nodesktop +
-timestwo(4) +
-quit +
-exit +
-</code> +
- +
-You should get the answer +
- +
-<code> +
->> timestwo(4) +
- +
-ans = +
- +
-     8 +
- +
->> +
-</code> +
- +
- +
- +
  
 ====== Parallel job ====== ====== Parallel job ======
Line 417: Line 53:
  
  
-=====Details by clusters=====+======Details by cluster====== 
 +  * [[software:matlab:caviness | Caviness]]
   * [[software:matlab:farber | Farber]]   * [[software:matlab:farber | Farber]]
   * [[software:matlab:mills| Mills]]   * [[software:matlab:mills| Mills]]
  • software/matlab/matlab.1566930720.txt.gz
  • Last modified: 2019-08-27 14:32
  • by anita