abstract:darwin:filesystems:filesystems

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
abstract:darwin:filesystems:filesystems [2021-04-28 12:40] – [Home filesystem] pdwabstract:darwin:filesystems:filesystems [2021-10-14 11:19] (current) – [Workgroup storage] anita
Line 7: Line 7:
 ==== Home storage ==== ==== Home storage ====
  
-Each user has 20 GB of disk storage reserved for personal use on the home file system. Users' home directories are in /home  (e.g., ''/home/1005''), and the directory name is put in the environment variable ''$HOME'' at login.+Each user has 20 GB of disk storage reserved for personal use on the home filesystem. Users' home directories are in /home  (e.g., ''/home/1005''), and the directory name is put in the environment variable ''$HOME'' at login.
 ===== High-performance Lustre filesystem ===== ===== High-performance Lustre filesystem =====
  
Line 13: Line 13:
 ==== Workgroup storage ==== ==== Workgroup storage ====
  
-Allocation workgroup storage is available on a [[abstract:darwin:filesystems:lustre|high-performance Lustre-based filesystem]] having almost 1.1 PB of usable space. Users should have a basic understanding of the concepts of [[abstract:darwin:filesystems:lustre|Lustre]] to take full advantage of this filesystem.+Allocation workgroup storage is available on a [[abstract:darwin:filesystems:lustre|high-performance Lustre-based filesystem]] having almost 1.1 PB of usable space. Users should have a basic understanding of the concepts of [[abstract:darwin:filesystems:lustre|Lustre]] to take full advantage of this filesystem. The default stripe count is set to 1 and the default striping is a single stripe distributed across all available OSTs on Lustre. See [[https://www.nas.nasa.gov/hecc/support/kb/lustre-best-practices_226.html|Lustre Best Practices]] from Nasa.
  
 Each allocation will have at least 1 TiB of shared ([[abstract:darwin:app_dev:compute_env#using-workgroup-and-directories|workgroup]]) storage in the ''/lustre/'' directory identified by the <<//allocation_workgroup//>> (e.g., ''/lustre/it_css'') accessbile by all users in the allocation workgroup, and is referred to as your workgroup directory (''$WORKDIR''), if the allocation workgroup has been set.  Each allocation will have at least 1 TiB of shared ([[abstract:darwin:app_dev:compute_env#using-workgroup-and-directories|workgroup]]) storage in the ''/lustre/'' directory identified by the <<//allocation_workgroup//>> (e.g., ''/lustre/it_css'') accessbile by all users in the allocation workgroup, and is referred to as your workgroup directory (''$WORKDIR''), if the allocation workgroup has been set. 
Line 21: Line 21:
 Each allocation will also have a ''/lustre/<<//workgroup//>>/sw'' directory to allow users to install software to be shared for the allocation workgroup. It can be referred to as ''$WORKDIR_SW'', if the allocation workgroup has been set. In addition a ''/lustre/<<//workgroup//>>/sw/valet'') directory is also provided to store VALET package files to shared for the allocation workgroup.  Each allocation will also have a ''/lustre/<<//workgroup//>>/sw'' directory to allow users to install software to be shared for the allocation workgroup. It can be referred to as ''$WORKDIR_SW'', if the allocation workgroup has been set. In addition a ''/lustre/<<//workgroup//>>/sw/valet'') directory is also provided to store VALET package files to shared for the allocation workgroup. 
  
-Please see [[abstract:darwin:app_dev:compute_env#using-workgroup-and-directories|allocation workgroup]] for complete details on environment variables.+Please see [[abstract:darwin:app_dev:compute_env#using-workgroup-and-directories|workgroup]] for complete details on environment variables.
  
 **Note**: A full filesystem inhibits use for everyone preventing jobs from running. **Note**: A full filesystem inhibits use for everyone preventing jobs from running.
Line 63: Line 63:
 ==== Workgroup ==== ==== Workgroup ====
  
-All of Lustre is available as allocation workgroup storage.  To check Lustre usage for all users, use ''df -h /lustre''.+All of Lustre is available for allocation workgroup storage.  To check Lustre usage for all users, use ''df -h /lustre''.
  
 The example below shows 25 TB is in use out of 954 TB of usable Lustre storage. The example below shows 25 TB is in use out of 954 TB of usable Lustre storage.
Line 109: Line 109:
 ==== Home and Workgroup ==== ==== Home and Workgroup ====
  
-<note important>While all filesystems on the DARWIN cluster utilize hardware redundancies to protect data, there is **no** backup or replication and **no** recovery available for the home or Lustre filesystems.+<note important>While all filesystems on the DARWIN cluster utilize hardware redundancies to protect data, there is **no** backup or replication and **no** recovery available for the home or Lustre filesystems. All backups are the responsibility of the user. DARWIN's systems administrators are not liable for any lost data
 </note> </note>
  
  • abstract/darwin/filesystems/filesystems.1619628004.txt.gz
  • Last modified: 2021-04-28 12:40
  • by pdw