Revisions to Slurm Configuration v1.0.7 on DARWIN

This document summarizes alterations to the Slurm job scheduler configuration on the DARWIN cluster.

See this document discussing swap limits in Slurm jobs.

• The dynamic_swap_limits SPANK plugin will be compiled and installed
• The plugstack.conf configuration file will be modified to require the dynamic_swap_limits plugin

The following aspects of the system construction will be observed in the configuration of the plugin:

• In all cases, the active swap present on the node will be used as the max_swap value
• Most nodes consist of 64 cores; 1/64 = 1.5625%, so by default jobs will be granted 1% of max_swap per CPU
• The V100 nodes consist of 48 cores; 1/48 = 2.0833%, so by default jobs will be granted 1.5% of max_swap per CPU
• The Mi100 node consists of 128 cores; 1/128 = 0.78125%, so by default jobs will be granted 0.25% of max_swap per CPU
• Jobs running under the extended-mem partition on the extended memory node will have no swap limits enforced (jobs are scheduled user-exclusive on this node, so any failures will not impact other users' jobs)

These details produce the following configuration string:

partition(extended-mem)=none,host(r2v[00-02])=1.5%/cpu,host(r0m01)=0.25%/cpu,default()=1%/cpu

No downtime is expected. The slurmd daemon must be restarted on all compute nodes, but currently-executing jobs/job steps should not be affected (they will reconnect to the new slurmd as necessary to communicate job status, etc.). The slurmctld daemons do not use the SPANK plugin, thus they do not need to be restarted.