Differences

This shows you the differences between two versions of the page.

--- software:tensorflow:caviness [2019-06-26 20:05] – anita
+++ software:tensorflow:caviness [2021-04-27 16:21] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
 ====== Tensorflow on Caviness ======
-TensorFlow must be used as a container.  You write your Python code either somewhere in your home directory ($HOME) or in the workgroup directory ($WORKDIR).  You should speak to other group members to understand how you should make use of the workgroup directory, e.g. create a directory for yourself, etc.
+TensorFlow must be used as a container, so the versions of TensorFlow installed on Caviness are provided as containers and dependent on Singularity.
+<code bash>
+$ vpkg_versions tensorflow
+Available versions in package (* = default version):
+[/opt/shared/valet/2.1/etc/tensorflow.vpkg_yaml]
+tensorflow        an end-to-end open source machine learning platform
+.12.0          release 1.12.0
+.12.0:gpu      release 1.12.0-gpu (uses CUDA toolkit 9.0)
+.12.0:gpu,py3  release 1.12.0-gpu-py3 (uses CUDA toolkit 9.0)
+.12.0:py3      release 1.12.0-py3
+* 1.13.1          release 1.13.1
+.13.1:gpu      release 1.13.1-gpu (uses CUDA toolkit 10.0)
+.13.1:gpu,py3  release 1.13.1-gpu-py3 (uses CUDA toolkit 10.0)
+.13.1:py3      release 1.13.1-py3
+</code>
+You write your Python code either somewhere in your home directory ($HOME) or in the workgroup directory ($WORKDIR).  You should speak to other group members to understand how you should make use of the workgroup directory, e.g. create a directory for yourself, etc.
 Remember you must specify your **[[abstract:caviness:app_dev:compute_env#using-workgroup-and-directories|workgroup]]** to define your cluster group or //investing-entity// compute nodes before submitting any job, and this includes starting an interactive session or submitting a batch job.
@@ Line 14: / Line 33: @@
 </code>
-For example, say your TensorFlow Python script is called ''tf-script.py'', then you should copy this file or create it the ''tf-test-001'' directory, then copy the tensorflow.qs job script template:
+For example, say your TensorFlow Python script is called ''tf-script.py'', then you should copy this file or create it in the ''tf-test-001'' directory, then copy the tensorflow.qs job script template:
 <code bash>
-$ cp /opt/shared/tensorflow/tensorflow.qs .
+$ cp /opt/shared/templates/slurm/applications/tensorflow.qs .
 </code>
-You will need to modify the copy of ''tensorflow.qs'' accordingly (''%%-%%-cpus-per-task=2'' to however many CPU cores you need, 1 - 36; ''%%-%%-mem-per-cpu=1024M'' to alter max memory limit; ''%%-%%-job-name=tensorflow'', etc.).  The template has extensive documentation that should assist you in customizing it for the job.  Last but not least, the last line should be changed to match your Python script name and for this example, it it would be ''tf-script.py'':
+You will need to modify the copy of ''tensorflow.qs'' accordingly such as ''%%-%%-cpus-per-task=2'' to however many CPU cores you need, 1 - 36; ''%%-%%-mem-per-cpu=1024M'' to alter max memory limit; ''%%-%%-job-name=tensorflow'', etc.  The template has extensive documentation that should assist you in customizing it for the job.  Last but not least, you need to specify the version of Tensorflow you want via VALET, and then the last line should be changed to match your Python script name and for this example, so for this example it would be ''tf-script.py'':
 <code bash>
+...
+#
+# Add a TensorFlow container to the environment:
+#
+vpkg_require tensorflow/1.12.0:py3
 #
 # Execute our TensorFlow Python script:
 #
-python tf-script.py
+python3 tf-script.py
 </code>
-Finally, submit the job using the "sbatch" command:
+Finally, submit the job using the ''sbatch'' command:
 <code bash>