Both sides previous revision Previous revision Next revision | Previous revision Next revisionBoth sides next revision |
abstract:farber:runjobs:schedule_jobs [2018-10-09 12:10] – [Command options for qsub] anita | abstract:farber:runjobs:schedule_jobs [2019-02-19 10:35] – [Parallel environments] anita |
---|
| [$JOB_NAME].pe[$JOB_ID] | Parallel job **error** filename (Usually empty) | | | [$JOB_NAME].pe[$JOB_ID] | Parallel job **error** filename (Usually empty) | |
| |
==== More command-line options for qsub ==== | ==== More options for qsub ==== |
| |
The most commonly used **qsub** options fall into two categories: //operational// and //resource-management//. The operational options deal with naming the output files, mail notification of the processing steps, sequencing of a series of jobs, and establishing the UNIX environment. The resource-management options deal with the specific system resources you desire or need, such as parallel programming environments, number of processor cores, maximum CPU time, and virtual memory needed. | The most commonly used **qsub** options fall into two categories: //operational// and //resource-management//. The operational options deal with naming the output files, mail notification of the processing steps, sequencing of a series of jobs, and establishing the UNIX environment. The resource-management options deal with the specific system resources you desire or need, such as parallel programming environments, number of processor cores, maximum CPU time, and virtual memory needed. |
</code> | </code> |
| |
This will submit 30 jobs to the queue, with the SGE_TASK_ID variable set for use in the ''myjobs.qs'' script (an [[general:userguide:06_runtime_environ?&#array-jobs|array job]].) | This will submit 30 jobs to the queue, with the SGE_TASK_ID variable set for use in the ''myjobs.qs'' script (an [[:abstract:farber:runjobs:schedule_jobs#array-jobs|array job]].) |
The ''mem_free'' resource will cause Grid Engine to find a node (or wait until one is available) with 20 Gbytes of memory free. | The ''mem_free'' resource will cause Grid Engine to find a node (or wait until one is available) with 20 Gbytes of memory free. |
The ''m_mem_free'' resource will tell Grid Engine to not schedule a job on a node unless the specified amount of memory is available to consume on that node. | The ''m_mem_free'' resource will tell Grid Engine to not schedule a job on a node unless the specified amount of memory is available to consume on that node. |
=== The threads parallel environment === | === The threads parallel environment === |
| |
Jobs such as those having openMP directives use the **//threads//** parallel environment, an implementation of the [[:general:userguide:05_development#programming-models|shared-memory programming model]]. These SMP jobs can only use the cores on a **single** node. | Jobs such as those having openMP directives use the **//threads//** parallel environment, an implementation of the [[:abstract:farber:app_dev:prog_env#programming-models|shared-memory programming model]]. These SMP jobs can only use the cores on a **single** node. |
| |
For example, if your group only owns nodes with 24 cores, then your ''–pe threads'' request may only ask for 24 or fewer slots. Use Grid Engine's **qconf** command to determine the names and characteristics of the queues and compute nodes available to your investing-entity group on a cluster. | For example, if your group only owns nodes with 24 cores, then your ''–pe threads'' request may only ask for 24 or fewer slots. Use Grid Engine's **qconf** command to determine the names and characteristics of the queues and compute nodes available to your investing-entity group on a cluster. |
==== Array jobs ==== | ==== Array jobs ==== |
| |
An [[:general:jobsched:grid-engine:35_parallelism#array-jobs|array job]] essentially runs the same job by generating a new repeated task many times. Each time, the environment variable **SGE_TASK_ID** is set to a sequence number by Grid Engine and its value provides input to the job submission script. | An array job essentially runs the same job by generating a new repeated task many times. Each time, the environment variable **SGE_TASK_ID** is set to a sequence number by Grid Engine and its value provides input to the job submission script. |
| |
<note tip> | <note tip> |