hpc documentation - technical:generic:mills-ongoing-file-access

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Access to Mills File Systems ======

On July 8, 2019, the Mills cluster took another step toward its eventual shutdown (complete retirement on January 1, 2020(([[https://cpb-us-w2.wpmucdn.com/sites.udel.edu/dist/2/334/files/2016/01/MillsEnd-of-LifePlanandPolicies-3-1jp8lqd.pdf|End-of-life policies]] with respect to hardware failure will continue to apply: failure of critical components may lead to loss of some functionality or total shutdown of the cluster. Barring such events, Mills will continue to function in this state until January 1, 2020 when its storage and head node will be decommissioned.
))):

  * all compute nodes were taken offline
  * the head node was rebuilt with a newer OS

From this time forward, the Mills head node (''mills.hpc.udel.edu'') will serve the single purpose of allowing continued access to the following file systems:

  * ''/home'' -- user home directories
  * ''/lustre/work'' -- workgroup storage
  * ''/archive'' -- additional workgroup storage (file archival, etc.)

Users can still ''ssh'' to the head node to use a shell to interact with these filesystems.  In addition, transfers of files //off the cluster// are permissible using:

  * ''sftp''
  * ''scp''
  * ''rsync''
  * ''rclone''

Users are asked to **not** upload new data to ''mills.hpc.udel.edu'' or attempt to execute any computational programs.

<WRAP center round important 60%>
The same resource-restriction mechanisms utilized on the Farber head node are now present on the Mills head node to prevent users from monopolizing its CPU and memory.  
</WRAP>

===== Access from other Clusters =====

For workgroups or users looking to move data off Mills and onto Farber or Caviness, the ''/home'' and ''/lustre/work'' file systems have been made available on those clusters' head/login nodes.  For example, a simple ''cp'' can be used to copy data //from// a user's Mills home directory //to// the user's Caviness home directory:

<code>
[frey@login00 ~]$ mkdir ~/from-mills

[frey@login00 ~]$ cd $HOME_MILLS

[frey@login00 1001]$ pwd
/mnt/mills/home/1001

[frey@login00 1001]$ ls -l comsol.qs 
-rw-r--r-- 1 frey everyone 3985 Jul  3  2018 comsol.qs

[frey@login00 1001]$ cp comsol.qs ~/from-mills
</code>

All Mills file systems are mounted read-only, so users **will not** be able to copy data in the other direction.

==== $HOME_MILLS ====

Every user on Farber and Caviness will have the environment variable ''HOME_MILLS'' present in their environment at login.  The variable contains the directory at which his/her Mills home directory will be present.

Users who did not have a Mills account will see the following error message when trying to access the directory (where #### is the user's Unix uid number):
<code>
-bash: cd: /mnt/mills/home/####: No such file or directory
</code>

==== $WORKDIR_MILLS ====

When entering a workgroup with the ''workgroup'' command on Farber and Caviness, the ''WORKDIR_MILLS'' environment variable will be set.  The variable contains the directory at which the Mills workgroup directory will be present.

Workgroups that did not have a presence on Mills will see the following error message when trying to access the directory:
<code>
-bash: cd: /mnt/mills/lustre/work/«workgroup»: No such file or directory
</code>

==== /archive ====

Additionally, the ''/archive'' file systems have been made available on Farber and Caviness under ''/mnt/mills/archive/«workgroup»''.

Workgroups that did not have a presence on Mills will see the following error message when trying to access the directory:
<code>
-bash: cd: /mnt/mills/archive/«workgroup»: No such file or directory
</code>