| Both sides previous revision Previous revision Next revision | Previous revisionLast revisionBoth sides next revision |
| software:abaqus:farber [2018-10-09 19:36] – [Abaqus on Farber] anita | software:abaqus:farber [2018-10-10 00:12] – [Interactive job] anita |
|---|
| ====== Abaqus on Farber ====== | ====== Abaqus on Farber ====== |
| |
| <note important>Abaqus version 6.14-1 will not work on Lustre. Please use [[:abstract:farber:filesystems:filesystems#workgroup-storage|workgroup storage]] ($WORKDIR) on Farber </note> | <note important>Abaqus version 6.14-1 will not work on Lustre. Please use [[:abstract:farber:filesystems:filesystems#workgroup-storage|workgroup storage]] ($WORKDIR) on Farber.</note> |
| |
| ===== Batch job ===== | ===== Batch job ===== |
| |
| You must define your [[general/userguide/04_compute_environ?s[]=workgroup#using-workgroup-and-directories|workgroup]] before submitting any job on the cluster. Due to the size of input and output files associated with Abaqus, it is suggested that separate project directories be created in ''$WORKDIR'' for each run. You will need to setup your environment for Abaqus including the license information by supplying a valid port and license server (''port@lm.udel.edu'') in your queue submission script. Examples of queue submission scripts are available for serial (''abaqus-serial.qs'') and parallel (''abaqus-openmp.qs'' and ''abaqus-mpi.qs'') batch jobs. Once you have your queue submission file setup, you simply use ''qsub abaqus-serial.qs'' to submit a serial batch job, ''qsub abaqus-openmp.qs'' to submit a parallel OpenMP batch job, or ''qsub abaqus-mpi.qs'' to submit a parallel generic MPI batch job. | You must define your [[abstract:farber:app_dev:compute_env|workgroup]] before submitting any job on the cluster. Due to the size of input and output files associated with Abaqus, it is suggested that separate project directories be created in ''$WORKDIR'' for each run. You will need to setup your environment for Abaqus including the license information by supplying a valid port and license server (''port@lm.udel.edu'') in your queue submission script. Examples of queue submission scripts are available for serial (''abaqus-serial.qs'') and parallel (''abaqus-openmp.qs'' and ''abaqus-mpi.qs'') batch jobs. Once you have your queue submission file setup, you simply use ''qsub abaqus-serial.qs'' to submit a serial batch job, ''qsub abaqus-openmp.qs'' to submit a parallel OpenMP batch job, or ''qsub abaqus-mpi.qs'' to submit a parallel generic MPI batch job. |
| |
| ==== Serial ==== | ==== Serial ==== |
| </code> | </code> |
| |
| <note important>A local script ''abaqus_mp_setup verbose'' will automatically setup your Abaqus runtime environment by defining options according to the queue submission script. In this example, since it is a serial job, the keyword ''cpus=1'' is defined and no parallel environment. See section [[abaqus#parallel | Parallel]] for defining a parallel runtime environment. The keyword ''interactive'' should always be used in queue submission scripts to run Abaqus, otherwise the job will be backgrounded and the queuing system will think the job has finished before it really has. </note> | <note important>A local script ''abaqus_mp_setup verbose'' will automatically setup your Abaqus runtime environment by defining options according to the queue submission script. In this example, since it is a serial job, the keyword ''cpus=1'' is defined and no parallel environment. See section [[:software:abaqus:farber#parallel | Parallel]] for defining a parallel runtime environment. The keyword ''interactive'' should always be used in queue submission scripts to run Abaqus, otherwise the job will be backgrounded and the queuing system will think the job has finished before it really has. </note> |
| |
| |
| </code> | </code> |
| |
| <note important>A local script ''abaqus_mp_setup verbose'' will automatically setup your Abaqus runtime environment by defining options according to the queue submission script. In this example, since it is a parallel job based on threads, the keyword ''cpus=8'' is defined and 'mp_mode=MP' for the parallel environment. See section [[abaqus#serial | Serial]] for defining a serial runtime environment. The keyword ''interactive'' should always be used in queue submissions scripts to run Abaqus, otherwise the job will be backgrounded and the queuing system will think the job has finished before it really has. </note> | <note important>A local script ''abaqus_mp_setup verbose'' will automatically setup your Abaqus runtime environment by defining options according to the queue submission script. In this example, since it is a parallel job based on threads, the keyword ''cpus=8'' is defined and 'mp_mode=MP' for the parallel environment. See section [[:software:abaqus:farber#serial | Serial]] for defining a serial runtime environment. The keyword ''interactive'' should always be used in queue submissions scripts to run Abaqus, otherwise the job will be backgrounded and the queuing system will think the job has finished before it really has. </note> |
| |
| |
| </code> | </code> |
| |
| Equally as important is to specify the [[clusters/mills/runapps?s[]=mem&s[]=free#qsub-s-resource-management-options|memory resource requirements]] using ''mem_free'' and ''ram_free'' in your job script. This prevents your job from being assigned to a node unless it has enough memory available on the node before it starts and it reserves this amount of memory during the run to prevent another job from being assigned to it. If we are specifying multiple cpus in a parallel environment such threads or MPI, then we divide the amount of memory needed by the number of cpus. For example, if we need 48GB using 8 cpus, then we would specify | Equally as important is to specify the [[:abstract:farber:runjobs:schedule_jobs#resource-management-options-on-farber|memory resource requirements]] using ''mem_free'' and ''m_mem_free'' in your job script. This prevents your job from being assigned to a node unless it has enough memory available on the node before it starts and it reserves this amount of memory during the run to prevent another job from being assigned to it. If we are specifying multiple cpus in a parallel environment such threads or MPI, then we divide the amount of memory needed by the number of cpus. For example, if we need 48GB using 8 cpus, then we would specify |
| |
| <code> | <code> |
| #$ -l pe threads 8 | #$ -l pe threads 8 |
| #$ -l mem_free=6G | #$ -l mem_free=6G |
| #$ -l ram_free=6G | #$ -l m_mem_free=6G |
| </code> | </code> |
| |
| ===== Interactive job ===== | ===== Interactive job ===== |
| |
| All interactive jobs should be run on a compute node by setting your [[general/userguide/04_compute_environ?s[]=workgroup#using-workgroup-and-directories|workgroup]] and using ''qlogin''. You need to be running [[general:training#connecting|X-Windows]] and you should make sure you are in the directory where your Abaqus input and output files will be stored. These files can be very large, so you should use your ''$WORKDIR'' filesystem. Once on the compute node, you will automatically be in the same working directory, but you will need to setup your environment to run Abaqus including the license information by supplying a valid port and license server (''port@ls.udel.edu''). In the example below, we are running ABAQUS CAE on the ''traine'' account in workgroup ''it-css'' using license server ''port@ls.udel.edu'': | All interactive jobs should be run on a compute node by setting your [[:abstract:farber:app_dev:compute_env|workgroup]] and using ''qlogin''. You need to be running [[training:ssh:ssh|X-Windows]] and you should make sure you are in the directory where your Abaqus input and output files will be stored. These files can be very large, so you should use your ''$WORKDIR'' filesystem. Once on the compute node, you will automatically be in the same working directory, but you will need to setup your environment to run Abaqus including the license information by supplying a valid port and license server (''port@ls.udel.edu''). In the example below, we are running ABAQUS CAE on the ''traine'' account in workgroup ''it-css'' using license server ''port@ls.udel.edu'': |
| |
| <code bash> | <code bash> |
| [(it_css:traine)@mills ABAQUS]$ cd $WORKDIR/traine/ABAQUS | [(it_css:traine)@farber ABAQUS]$ cd $WORKDIR/traine/ABAQUS |
| [(it_css:traine)@mills ABAQUS]$ qlogin | [(it_css:traine)@farber ABAQUS]$ qlogin |
| Your job 108587 ("QLOGIN") has been submitted | Your job 108587 ("QLOGIN") has been submitted |
| waiting for interactive job to be scheduled ... | waiting for interactive job to be scheduled ... |
| Your interactive job 108587 has been successfully scheduled. | Your interactive job 108587 has been successfully scheduled. |
| Establishing /opt/shared/GridEngine/local/qlogin_ssh session to host n016 ... | Establishing /opt/shared/GridEngine/local/qlogin_ssh session to host n016 ... |
| Last login: Tue Nov 27 14:13:55 2012 from mills.mills.hpc.udel.edu | Last login: Tue Nov 27 14:13:55 2012 from farber.farber.hpc.udel.edu |
| [traine@n016 ABAQUS]$ pwd | [traine@n016 ABAQUS]$ pwd |
| /lustre/work/it_css/traine/ABAQUS | /home/work/it_css/traine/ABAQUS |
| [traine@n016 ABAQUS]$ vpkg_require abaqus | [traine@n016 ABAQUS]$ vpkg_require abaqus |
| Adding dependency `x11/RHEL6.1` to your environment | Adding dependency `x11/RHEL6.1` to your environment |
| | |
| <code bash> | <code bash> |
| [(it_css:traine)@mills ABAQUS]$ qlogin -pe threads 8 | [(it_css:traine)@farber ABAQUS]$ qlogin -pe threads 8 |
| Your job 161231 ("QLOGIN") has been submitted | Your job 161231 ("QLOGIN") has been submitted |
| waiting for interactive job to be scheduled ... | waiting for interactive job to be scheduled ... |
| Your interactive job 161231 has been successfully scheduled. | Your interactive job 161231 has been successfully scheduled. |
| Establishing /opt/shared/OpenGridScheduler/local/qlogin_ssh session to host n015 ... | Establishing /opt/shared/OpenGridScheduler/local/qlogin_ssh session to host n015 ... |
| Last login: Fri Jan 25 13:40:04 2013 from mills.mills.hpc.udel.edu | Last login: Fri Jan 25 13:40:04 2013 from farber.farber.hpc.udel.edu |
| [traine@n016 ABAQUS]$ | [traine@n016 ABAQUS]$ |
| </code> | </code> |