Both sides previous revision Previous revision Next revision | Previous revision |
abstract:darwin:filesystems:filesystems [2021-04-28 11:55] – [Workgroup storage] anita | abstract:darwin:filesystems:filesystems [2021-10-14 11:19] (current) – [Workgroup storage] anita |
---|
===== Home filesystem ===== | ===== Home filesystem ===== |
| |
The 65 TB filesystem uses 3 TB enterprise class SATA drives in a triple-parity RAID configuration for high reliability and availability. The filesystem is accessible to the head node via 10 Gbit/s Ethernet and to the compute nodes via 1 Gbit/s Ethernet. | The 13.5 TiB filesystem uses 960 GiB enterprise class SSD drives in a triple-parity RAID configuration for high reliability and availability. The filesystem is accessible to all nodes via IPoIB on the 100 Gbit/s InfiniBand network. |
| |
==== Home storage ==== | ==== Home storage ==== |
| |
Each user has 20 GB of disk storage reserved for personal use on the home file system. Users' home directories are in /home (e.g., ''/home/1005''), and the directory name is put in the environment variable ''$HOME'' at login. | Each user has 20 GB of disk storage reserved for personal use on the home filesystem. Users' home directories are in /home (e.g., ''/home/1005''), and the directory name is put in the environment variable ''$HOME'' at login. |
===== High-performance Lustre filesystem ===== | ===== High-performance Lustre filesystem ===== |
| |
==== Workgroup storage ==== | ==== Workgroup storage ==== |
| |
Allocation workgroup storage is available on a [[abstract:darwin:filesystems:lustre|high-performance Lustre-based filesystem]] having almost 1.1 PB of usable space. Users should have a basic understanding of the concepts of [[abstract:darwin:filesystems:lustre|Lustre]] to take full advantage of this filesystem. | Allocation workgroup storage is available on a [[abstract:darwin:filesystems:lustre|high-performance Lustre-based filesystem]] having almost 1.1 PB of usable space. Users should have a basic understanding of the concepts of [[abstract:darwin:filesystems:lustre|Lustre]] to take full advantage of this filesystem. The default stripe count is set to 1 and the default striping is a single stripe distributed across all available OSTs on Lustre. See [[https://www.nas.nasa.gov/hecc/support/kb/lustre-best-practices_226.html|Lustre Best Practices]] from Nasa. |
| |
Each allocation will have at least 1 TiB of shared ([[abstract:darwin:app_dev:compute_env#using-workgroup-and-directories|workgroup]]) storage in the ''/lustre/'' directory identified by the <<//allocation_workgroup//>> (e.g., ''/lustre/it_css'') accessbile by all users in the allocation workgroup, and is referred to as your workgroup directory (''$WORKDIR''), if the allocation workgroup has been set. | Each allocation will have at least 1 TiB of shared ([[abstract:darwin:app_dev:compute_env#using-workgroup-and-directories|workgroup]]) storage in the ''/lustre/'' directory identified by the <<//allocation_workgroup//>> (e.g., ''/lustre/it_css'') accessbile by all users in the allocation workgroup, and is referred to as your workgroup directory (''$WORKDIR''), if the allocation workgroup has been set. |
Each allocation will also have a ''/lustre/<<//workgroup//>>/sw'' directory to allow users to install software to be shared for the allocation workgroup. It can be referred to as ''$WORKDIR_SW'', if the allocation workgroup has been set. In addition a ''/lustre/<<//workgroup//>>/sw/valet'') directory is also provided to store VALET package files to shared for the allocation workgroup. | Each allocation will also have a ''/lustre/<<//workgroup//>>/sw'' directory to allow users to install software to be shared for the allocation workgroup. It can be referred to as ''$WORKDIR_SW'', if the allocation workgroup has been set. In addition a ''/lustre/<<//workgroup//>>/sw/valet'') directory is also provided to store VALET package files to shared for the allocation workgroup. |
| |
Please see [[abstract:darwin:app_dev:compute_env#using-workgroup-and-directories|allocation workgroup]] for complete details on environment variables. | Please see [[abstract:darwin:app_dev:compute_env#using-workgroup-and-directories|workgroup]] for complete details on environment variables. |
| |
**Note**: A full filesystem inhibits use for everyone preventing jobs from running. | **Note**: A full filesystem inhibits use for everyone preventing jobs from running. |
==== Workgroup ==== | ==== Workgroup ==== |
| |
All of Lustre is available as allocation workgroup storage. To check Lustre usage for all users, use ''df -h /lustre''. | All of Lustre is available for allocation workgroup storage. To check Lustre usage for all users, use ''df -h /lustre''. |
| |
The example below shows 25 TB is in use out of 954 TB of usable Lustre storage. | The example below shows 25 TB is in use out of 954 TB of usable Lustre storage. |
==== Home and Workgroup ==== | ==== Home and Workgroup ==== |
| |
<note important>While all filesystems on the DARWIN cluster utilize hardware redundancies to protect data, there is **no** backup or replication and **no** recovery available for the home or Lustre filesystems. | <note important>While all filesystems on the DARWIN cluster utilize hardware redundancies to protect data, there is **no** backup or replication and **no** recovery available for the home or Lustre filesystems. All backups are the responsibility of the user. DARWIN's systems administrators are not liable for any lost data. |
</note> | </note> |
| |