| Both sides previous revision Previous revision Next revision | Previous revision |
| abstract:farber:runjobs:job_status [2018-07-16 15:16] – anita | abstract:farber:runjobs:job_status [2021-10-13 16:49] (current) – [Per-Group QStat and QHost] anita |
|---|
| ====== Viewing Job Status Information on Farber ====== | ====== Managing Jobs on Farber ====== |
| |
| Once a user has been able to submit jobs to the queue -- interactive or batch -- the user will from time to time want to know what those jobs are doing. Is the job waiting in a queue for resources to become available, or is it executing? How long has the job been executing? How much CPU time or memory has the job consumed? Users can query Grid Engine for job information using the ''qstat'' command. The ''qstat'' command has a variety of command line options available to customize and filter what information it displays; discussing all of them is beyond the scope of this document. Please see the ''qstat'' man page for a detailed description of all options. | Once a user has been able to submit jobs to the queue -- interactive or batch -- the user will from time to time want to know what those jobs are doing. Is the job waiting in a queue for resources to become available, or is it executing? How long has the job been executing? How much CPU time or memory has the job consumed? Users can query Grid Engine for job information using the ''qstat'' command. The ''qstat'' command has a variety of command line options available to customize and filter what information it displays; discussing all of them is beyond the scope of this document. Please see the ''qstat'' man page for a detailed description of all options. |
| The ''qstatgrp'' command by default summarizes usage of all queues to which the user has access given his/her current working group. Adding the ''-j'' flag summarizes the jobs executing in those queues rather than summarizing the queues themselves. | The ''qstatgrp'' command by default summarizes usage of all queues to which the user has access given his/her current working group. Adding the ''-j'' flag summarizes the jobs executing in those queues rather than summarizing the queues themselves. |
| |
| The ''qhostgrp'' command by default summarizes usage of all hosts to which the user has access given his/her current working group. Adding the ''-j'' flag summarizes the jobs (including [[general/jobsched/standby|standby]]) executing on those hosts rather than summarizing the hosts themselves. | The ''qhostgrp'' command by default summarizes usage of all hosts to which the user has access given his/her current working group. Adding the ''-j'' flag summarizes the jobs (including [[abstract:farber:runjobs:queues#farber-standby-queues|standby]]) executing on those hosts rather than summarizing the hosts themselves. |
| |
| Both ''qstatgrp'' and ''qhostgrp'' accept a ''-g ''<<''group name''>> option to limit to an arbitrary group (and not just the user's current working group). | Both ''qstatgrp'' and ''qhostgrp'' accept a ''-g ''<<''group name''>> option to limit to an arbitrary group (and not just the user's current working group). |
| **Details by cluster** | **Details by cluster** |
| |
| * [[clusters:farber:runapps#resource-management-options|farber]] | * [[abstract:farber:runjobs:schedule_jobs#resource-management-options-on-farber|Farber]] |
| * [[clusters:farber:runapps#resource-management-options|Farber]] | |
| |
| ===== Managing Jobs ===== | ===== Managing Jobs ===== |
| </code> | </code> |
| |
| Use **qalter** to change the attributes of the pending job such as reducing the number of slots requested to be within the workgroup ''it_css'' nodes or change the resources specified to the [[general:jobsched:standby|standby queue]] so the job could run. For example, let's change the number of slots requested to 48 instead of 144 by using | Use **qalter** to change the attributes of the pending job such as reducing the number of slots requested to be within the workgroup ''it_css'' nodes or change the resources specified to the [[:abstract:farber:runjobs:queues#farber-standby-queues|standby queue]] so the job could run. For example, let's change the number of slots requested to 48 instead of 144 by using |
| |
| <code base> | <code base> |
| | ''qhostgrp'' | ''qhostgrp'' displays the current status of your owner-group's nodes | | | ''qhostgrp'' | ''qhostgrp'' displays the current status of your owner-group's nodes | |
| | ''qhostgrp –g'' <<//investing_entity//>> | ''qhostgrp -g it_css'' displays the current status of the nodes owned by the\\ //it_css// investing-entity. | | | ''qhostgrp –g'' <<//investing_entity//>> | ''qhostgrp -g it_css'' displays the current status of the nodes owned by the\\ //it_css// investing-entity. | |
| | ''qhostgrp -j -g'' <<//investing_entity//>> | ''qhostgrp –j -g it_css'' shows all jobs running (including [[general/jobsched/standby|standby]] and spillover) in the owner-group nodes for the //it_css// investing-entity. | | | ''qhostgrp -j -g'' <<//investing_entity//>> | ''qhostgrp –j -g it_css'' shows all jobs running (including [[:abstract:farber:runjobs:queues#farber-standby-queues|standby]] and spillover) in the owner-group nodes for the //it_css// investing-entity. | |
| |
| ==== Checking overall usage of resource quotas ==== | ==== Checking overall usage of resource quotas ==== |
| |
| Resource quotas are used to help control the standby and spillover queues. Each user has a quota based on the limits set by the [[general/jobsched/standby|standby]] queue specifications for each cluster, and each workgroup has a per_workgroup quota based on the number of slots purchased by the research group. | Resource quotas are used to help control the standby and spillover queues. Each user has a quota based on the limits set by the [[:abstract:farber:runjobs:queues#farber-standby-queues|standby]] queue specifications for each cluster, and each workgroup has a per_workgroup quota based on the number of slots purchased by the research group. |
| |
| ^ Command ^ Illustrative example ^ | ^ Command ^ Illustrative example ^ |
| | ''qquota -u'' <<//username//>> ''| grep standby'' | ''qquota -g traine | grep standby'' displays the current usage of slots by user\\ //traine// in the standby resources. | | | ''qquota -u'' <<//username//>> ''| grep standby'' | ''qquota -u traine | grep standby'' displays the current usage of slots by user\\ //traine// in the standby resources. | |
| | ''qquota -u \* | grep'' <<//investing_entity//>> | ''qquota -u \* | grep it_css'' displays the current usage of slots being used by all\\ members of the //it_css// investing-entity, the per_workgroup quota. | | | ''qquota -u \* | grep'' <<//investing_entity//>> | ''qquota -u \* | grep it_css'' displays the current usage of slots being used by all\\ members of the //it_css// investing-entity, the per_workgroup quota. | |
| |