==== Running your OpenFOAM application on Farber ====

Using the ''openmpi-ib.qs'' job script template found in ''/opt/shared/templates/gridengine/openmpi'' provides the best performance for a parallel MPI run for OpenFOAM applications.  Once you have copied the template for your job script (e.g. ''qsub_sowfa.qs''), then edit the following for your OpenFOAM application:

  - Modify NPROC to select the number of slots (cores).
  - Modify the memory specification ''-l m_mem_free'' according to the instructions if you want to have more than 1GB of memory per slot (core).
  - Modify the email notification ''-m'' and ''-M'' according to the instructions if you want email to be sent at the end of the job, if the job is aborted, rescheduled, or suspended.
  - Setup your environment with your newly built OpenFOAM application by replacing the ''vpkg_require openmpi/1.4.4-gcc'' with your VALET package name (e.g. ''sowfa/2.0-OF2.1-gcc'')
  - Setup the MPI executable and options (arguments) for your OpenFOAM application (e.g. ''ABLSolver -parallel > log.ABLSolver.3.${JOB_ID} 2>err.ABLSolver.3'')

Now submit your job script using

<code>
qsub qsub_sowfa.qs
</code>

<note important> There is a 1.5.x Grid Engine integration error when **only** using OpenFOAM 2.1.1 compiled with ''openmpi-1.5.3'' which causes the following warning when initiating an MPI run 

<code> mca: base: component_find: unable to open /opt/shared/openfoam/2.1.
1-gcc/ThirdParty-2.1.1/platforms/linux64Gcc/openmpi-1.5.3/lib/openmpi/mca_plm_rs
hd: /opt/shared/openfoam/2.1.1-gcc/ThirdParty-2.1.1/platforms/linux64Gcc/openmpi
-1.5.3/lib/openmpi/mca_plm_rshd.so: cannot open shared object file: Permission d
enied (ignored)</code>

This is a warning and not an error. Please ignore.</note>