hpc documentation

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Procedural installation of ArcGIS 10.1 ======

 --- //Jeffrey T Frey 2013-06-19 15:46//

This document outlines the process by which ArcGIS 10.1 Server for Linux was setup in such a way that users of an HPC cluster could make use of the Python shell embedded within the product to run GIS scripts.  Normally installation details would not be documented in this fashion, but the process was difficult enough to warrant fine-grain detail.

<note important>Portions of this document include or allude to intellectual property of Esri and its licensors and are used herein under license.  Copyright (c) 2013 Esri and its licensors.  All rights reserved.</note>
===== Shared Install =====

First and foremost it was desirable to have a single shared copy of the product available to all users.  This is the normal mode in which software managed by IT gets installed on a cluster:

  * a directory is created under ''/opt/shared'' for the product, e.g. ''/opt/shared/arcgis''
  * individual versions or variants of the product installed in subdirectories therein, e.g. ''/opt/shared/arcgis/10.1''
  * ''PATH'', ''LD_LIBRARY_PATH'', and similar environment modifications necessary per-version or per-variant are modeled in a VALET configuration file for the product

So I installed ArcGIS 10.1 Server for Linux to the directory ''/opt/shared/arcgis/10.1''; their installer automatically forces further extension of the directory depth to ''/opt/shared/arcgis/10.1/arcgis/server'' but otherwise everything was present and if I ran the ''arcgis/server/tools/python'' script I got a Python shell.

The second another support person attempted to use the software, though, an error message about ownership was produced.  This stems from the ''arcgis/server/tools/python'' script, which will not execute unless the owner of the installed files matches the identity of the user executing that command:

<code bash>
  :
installer=`stat -c %U $installDir`
  :
if [ "$installer" != "$LOGNAME" ]; then
   echo -n "Python can only be started by the install owner ($installer).  "
   echo "This script must be run by the installation owner ($installer)"
   exit
fi
  :
</code>

I tried simply removing this check, but deeper within the rest of the software there are similar checks.

ArcGIS 10.1 Server for Linux is not strictly speaking a Linux software product.  Rather, it is a Windows product run within a WINE environment on Linux.  It is necessary to run as the user who installed the software because there is a writable WINE disk image buried inside ''arcgis/server/framework'' and the environment cannot function properly unless WINE can write to that disk image.  Since my first attempt left it owned and writable only by me, disabling the identity check in the ''python'' script only delayed failure.

==== Local WINE Instances ====

The next logical step was to attempt to make a copy of the WINE disk image (just over 1 MB) for each user, keeping the rest of the ArcGIS 10.1 installation in the shared location under ''/opt/shared/arcgis''.  After careful analysis of the shell setup files that ArcGIS installs, a launcher script was written that:

  - duplicated the WINE disk image in a user's ''/home'' directory if not present
  - set the environment variables that were necessary to direct WINE to use that directory rather than the original
  - launched the Python shell within the WINE environment

This actually worked great -- until we tried ''import arcpy'' in the Python shell.  The shell would crash each and every time at exactly the same point in the import.  Enabling some tracing I eventually found that a binary Python module was the problem, and it was likely that either moving the WINE disk image out of place or (again) the end user's not being the user at time of install was the cause.  So a shared install of the software appeared to be infeasible.

The only useful product from this stage of the work was a custom startup script, similar to the ''arcgis/server/tools/python'' script included in the package.

===== Orphaned xvfb =====

Another aspect of the ArcGIS 10.1 Server for Linux product that came to light was that it absolutely requires an X11 server to be present.  Part of the extensive startup work done by the ArcGIS shell scripts prior to launching WINE is starting an embedded ''xvfb'' (X11 virtual frame buffer) executable to act as the display for WINE if no display is present.

Unfortunately, that aspect of the package was not written very well:

  * the script sets up statically-named state files when starting ''xvfb'', none of which contain the pid of that process
  * the script walks a small range of X11 display numbers, using the first that is not already in use
  * the ''xvfb'' process is detached from the current shell and ends up orphaned after you exit the Python shell

At the very least, for end users to effectively run the ArcGIS 10.1 Python shell it is necessary to exit from that shell to produce a graceful shutdown of all processes that were started.  Otherwise, the Nth startup of the Python shell would fail because (N-1) orphaned ''xvfb'' processes would still be running on the node.

This problem was remedied by:

  - the ''init_Xvfb.sh'' script provided by Esri was patched to emit a .pid and .port file containing the process id of the launched ''xvfb'' and the X11 display number it used, respectively; the files are named via a pattern that includes the pid of the originating process (since ''init_Xvfb.sh'' is sourced by the startup script, not executed)
  - the startup script was modified to include a signal handler that shuts down the ''xvfb'' that was started (by finding its pid via the .pid file that was produced)

A diff patch was created to effect the changes to the ''init_Xvfb.sh'' script.

===== Per-User Installs =====

Thus the idea of a shared install of ArcGIS 10.1 Server for Linux was abandoned.  Each user wishing to run the product must make his/her own install; to aid in this process, a copy of the installation files were made available and a helper script was written to perform the installation and all necessary patching (e.g. for ''init_Xvfb.sh'').

  - The temporary directory (which will contain the unpacked installer pieces) is set to an appropriate location (''IATEMPDIR'' environment variable)
  - Java is required for the installer; OpenJDK is configured using [[:software:valet:valet|VALET]]
  - The ''ArcGIS_Server_Ent_Linux_101_129046/ArcGISServer/Setup'' script is executed:
    - ''%%--%%mode silent'' : No GUI or interaction is desired
    - ''%%--%%directory <install dir>'' : the script's only argument is the directory into which ''arcgis/server'' should be installed
    - ''%%--%%license-agreement Yes'' : by running the install helper the user has implicitly consented :-)
    - ''%%--%%authorization-file <provisioning file>'' : license information provided by CS&S
  - Any processes started from the install directory are killed -- this gets rid of the license manager that the installer automatically launches
  - The ''init_Xvfb.sh'' script is patched to add the .pid and .port file emission code
  - The launch script is written to the install directory.  It includes all of the ''xvfb'' automata, etc.

If the install helper completes successfully, the user would execute the ArcGIS 10.1 Server for Linux Python shell using the command

<code bash>
[user@cluster ~]$ <install dir>/arcgis-10.1-shell
Adding package `x11/RHEL6.1` to your environment
Python 2.7.2 (default, Jun 12 2011, 14:24:46) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 
</code>

===== Summary =====

Creating a single shared installation of ArcGIS 10.1 Server for Linux to be used in standalone fashion proved impossible due to the structure of the runtime environment.  Even using a per-user copy of the WINE disk image did not yield a workable partially-shared configuration.  Instead, each user must make his/her own installation of ArcGIS 10.1 Server for Linux.  A script that requires a single argument -- the path into which to install -- was written to avoid end users' having to follow the complex procedure, which includes patching of the Xvfb startup procedures used when the shell lacks an X11 display.

Here is an example installation session produced running {{:software:arcgis:install-arcgis-10.1.txt|/opt/shared/arcgis/bin/install-arcgis-10.1}} on the Mills cluster:

<code>
[frey@n011 ~]$ /opt/shared/arcgis/bin/install-arcgis-10.1 /archive/it_nss/sw/ArcGIS
Adding dependency `x11/RHEL6.1` to your environment
Adding package `openjdk/1.6.0` to your environment
[ArcGIS 10.1 for Server Installation Details]
UI Mode..................silent
Agreed to Esri License...yes
Authorization File......./opt/shared/arcgis/data/10.1/XXXXXXX.prvc
Installation Directory.../archive/it_nss/sw/ArcGIS/arcgis/server

Starting installation of ArcGIS 10.1 for Server...
...ArcGIS 10.1 for Server installation is complete.

You will be able to access ArcGIS Server Manager by navigating to http://n011:6080/arcgis/manager.

Killing 4 process(es) started by the installer...

Patching the init_Xvfb.sh script...

Adding ArcGIS shell startup script.  You can run the Python shell with the command

  /archive/it_nss/sw/ArcGIS/arcgis-10.1-shell

[frey@n011 ~]$ /archive/it_nss/sw/ArcGIS/arcgis-10.1-shell
Adding package `x11/RHEL6.1` to your environment
Python 2.7.2 (default, Jun 12 2011, 14:24:46) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 
</code>

A base64-encoded copy of the Xvfb diff patch is embedded within the install script.  The diff patch itself is {{:software:arcgis:xvfb.patch|available for perusal}}.