| Both sides previous revision Previous revision Next revision | Previous revision |
| abstract:darwin:runjobs:accounting [2021-07-26 17:11] – [sproject projects] anita | abstract:darwin:runjobs:accounting [2024-02-29 13:14] (current) – [Job Accounting on DARWIN] anita |
|---|
| |
| **Moral of the story:** Request only the resources needed for your job. Over or under requesting resources results in wasting your allocation credits for everyone in your project/workgroup.</note> | **Moral of the story:** Request only the resources needed for your job. Over or under requesting resources results in wasting your allocation credits for everyone in your project/workgroup.</note> |
| | |
| | <note important>**Interactive jobs:** An interactive job is billed the SU associated with the full wall time of its execution, not just for CPU time accrued through its duration. For example, if you leave an interactive job running for 2 hours and execute code for 2 minutes, your allocation will be billed for 2 hours of time, not 2 minutes. Please review the SU associated for each type of resource requested ([[abstract:darwin:runjobs:accounting#compute-allocations|compute]], [[abstract:darwin:runjobs:accounting#gpu-allocations|gpu]]) and the associated SUs billed per hour.</note> |
| | |
| | //If you need to have SU to dollar conversions based on your DARWIN allocation for grant proposals or reports, please submit a [[https://services.udel.edu/TDClient/32/Portal/Requests/TicketRequests/NewForm?ID=D5ZRIgFlfLw_|Research Computing High Performance Computing (HPC) Clusters Help Request]] and complete the form including DARWIN and indicated you are requesting SU to dollar conversions in the details in the description field. |
| | // |
| ===== Compute Allocations ===== | ===== Compute Allocations ===== |
| |
| |
| Note that these are estimates based on nominal memory. Actual charges are based on available memory which will be lower than nominal memory due to the memory requirements for the OS and system daemons. | Note that these are estimates based on nominal memory. Actual charges are based on available memory which will be lower than nominal memory due to the memory requirements for the OS and system daemons. |
| | |
| | ===== The idle partition ===== |
| | |
| | Jobs that execute in the [[abstract:darwin:runjobs:queues#the-idle-partition|idle partition]] do not result in charges against your allocation(s). If your jobs can support [[abstract:darwin:runjobs:schedule_jobs#handling-system-signals-aka-checkpointing|checkpointing]], the idle partition will enable you to continue your research even if you exhaust your allocation(s). However, jobs submitted to the other partitions which do get charged against allocations **will take priority** and may cause ''idle'' partition jobs to be **preempted**. |
| | |
| | <note>Since jobs in the ''idle'' partition do not result in charges you will not see them in the output of the ''sproject'' command documented below. You can still use [[abstract:darwin:runjobs:job_status#checking-job-status|standard Slurm commands to check the status]] of those jobs.</note> |
| |
| ====== Checking Allocation Usage ====== | ====== Checking Allocation Usage ====== |
| ==== sproject allocations ==== | ==== sproject allocations ==== |
| |
| The ''allocations'' subcommand shows information for all projects/workgroups allocations on DARWIN. To see only your workgroup's allocations, use the ''-g <//workgroup//>'' option as show below as an example for workgroup ''it_css''. | The ''allocations'' subcommand shows information for resource allocations granted to projects/workgroups on DARWIN to which you are a member. To see a specific workgroup's allocations, use the ''-g <//workgroup//>'' option as in this example for workgroup ''it_css'': |
| |
| <code> | <code> |
| </code> | </code> |
| |
| The ''%%--%%detail'' flag will show additional information reflecting the credits, pending + running job charges, debits, and balance of each allocation: | The ''%%--%%detail'' flag will show additional information reflecting the credits, running + completed job charges, debits, and balance of each allocation: |
| |
| <code> | <code> |
| ==== sproject projects ==== | ==== sproject projects ==== |
| |
| The ''projects'' subcommand shows information for projects/workgroups on DARWWIN such as the project id, group id, name, and creation date. To see only your workgroup's allocations, use the ''-g <//workgroup//>'' option as show below as an example for workgroup ''it_css''. | The ''projects'' subcommand shows information (such as the project id, group id, name, and creation date) for projects/workgroups on DARWIN to which you are a member. To see a specific project/workgroup, use the ''-g <//workgroup//>'' option as in this example for workgroup ''it_css'': |
| |
| <code> | <code> |
| ---------- ------- -------- ---------- ------------------------- | ---------- ------- -------- ---------- ------------------------- |
| 2 it_css 1002 it_css 2021-07-12 14:51:57-04:00 | 2 it_css 1002 it_css 2021-07-12 14:51:57-04:00 |
| <code> | </code> |
| |
| Adding the ''%%--%%detail'' flag will also show each allocation associated with the project. | Adding the ''%%--%%detail'' flag will also show each allocation associated with the project. |
| ==== sproject jobs ==== | ==== sproject jobs ==== |
| |
| The ''jobs'' subcommand shows information on all jobs run against all projects/workgroups such as the Slurm job id, owner, and amount charged. Various options are available for sorting and filtering, use ''sproject jobs -h'' for complete details. To see only your workgroup's jobs, use the ''-g <//workgroup//>'' option as show below as an example for workgroup it_css. | The ''jobs'' subcommand shows information (such as the Slurm job id, owner, and amount charged) for individual jobs billed against resource allocations for projects/workgroups on DARWIN to which you are a member. Various options are available for sorting and filtering, use ''sproject jobs -h'' for complete details. To see jobs associated with a specific project/workgroup, use the ''-g <//workgroup//>'' option as in this example for workgroup ''it_css'': |
| |
| | <code> |
| | $ sproject jobs -g it_css |
| | Activity id Alloc id Alloc descr Job id Owner Status Amount Creation date |
| | ----------- -------- ------------ ------ -------- --------- ------ ------------------------- |
| | 25000 43 it_css::cpu 80000 traine executing -3966 2021-08-10 18:27:59-04:00 |
| | 25001 43 it_css::cpu 80001 traine executing -3966 2021-08-10 18:52:11-04:00 |
| | 25002 43 it_css::cpu 80002 trainf executing -4200 2021-08-11 13:06:26-04:00 |
| | 25003 43 it_css::cpu 80003 traine executing -1200 2021-08-11 16:07:42-04:00 |
| | </code> |
| |
| | Jobs that complete execution will be displayed with a status of ''completed'' and the actual billable amount used by the job. At the top and bottom of each hour, completed jobs are //resolved// into per-user debits and disappear from the ''jobs'' listing (see the **sproject allocations** section above for the display of resource allocation credits, debits, and pre-debits). |
| |
| ==== sproject failures ==== | ==== sproject failures ==== |
| |
| The ''failures'' subcommand shows the job id and error message for all jobs that failed due to allocation limits. Various options are available for sorting and filtering, use ''sprojects failures -h'' for complete details. To see only your workgroup's job failures, use the ''-g <//workgroup//>'' option as show below as an example for workgroup ''it_css''. | The ''failures'' subcommand shows information (such as the Slurm job id, owner, and amount charged) for all jobs that failed to execute due to insufficient allocation balance on resource allocations for projects/workgroups on DARWIN to which you are a member. Various options are available for sorting and filtering, use ''sproject failures -h'' for complete details. To see failures associated with jobs run as a specific project/workgroup, use the ''-g <//workgroup//>'' option as in this example for workgroup ''it_css'': |
| |
| <code> | <code> |
| </code> | </code> |
| |
| ===== XSEDE Allocations ====== | ===== ACCESS (XSEDE) Allocations ====== |
| |
| For XSEDE allocations on DARWIN, you may use the [[https://portal.xsede.org/my-xsede|XSEDE user portal]] to check allocation usage, however keep in mind using the [[abstract:darwin:runjobs:accounting#sproject|sproject]] command available on DARWIN will provide the most up-to-date allocation usage information since the XSEDE Portal will only be updated nightly. | For ACCESS allocations on DARWIN, you may use the [[https://allocations.access-ci.org/allocations/summary|ACCESS Allocations portal]] to check allocation usage, however keep in mind using the [[abstract:darwin:runjobs:accounting#sproject|sproject]] command available on DARWIN will provide the most up-to-date allocation usage information since the ACCESS Allocations Portal will only be updated nightly. |
| |
| ===== Storage Allocations ===== | ===== Storage Allocations ===== |