software:tensorflow:caviness

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
software:tensorflow:caviness [2026-04-02 15:17] – [GPU support] Complete section mbottosoftware:tensorflow:caviness [2026-04-02 15:33] (current) – [CPU only] Complete section mbotto
Line 177: Line 177:
 ==== CPU only ==== ==== CPU only ====
  
 +The ''conda search tensorflow'' command can be used to locate the specific version you wish to install. Some examples are shown (with many ommitted):
 +
 +<code bash>
 +[(my_workgroup:user)@login00.caviness ~]$ conda search tensorflow
 +Loading channels: done
 +# Name                       Version           Build  Channel
 +tensorflow                    2.10.0 cpu_py310hd1aba9c_0  conda-forge
 +tensorflow                    2.10.0 cpu_py37h08536eb_0  conda-forge
 +   :
 +tensorflow                    2.19.1 cpu_py312h69ecde4_52  conda-forge
 +tensorflow                    2.19.1 cpu_py312h69ecde4_53  conda-forge
 +tensorflow                    2.19.1 cpu_py312h69ecde4_54  conda-forge
 +tensorflow                    2.19.1 cuda128py310h40b8f1e_200  conda-forge
 +tensorflow                    2.19.1 cuda128py310h40b8f1e_201  conda-forge
 +tensorflow                    2.19.1 cuda128py310h40b8f1e_203  conda-forge
 +   :
 +</code>
 +
 +Note that the build tag provides the distinction between variants built on top of specific devices or libraries. For example, the third ''2.19.1'' package above is built for CPUs with Python 3.12 and translates to the qualified Conda package name ''tensorflow[version=2.19.1,build=cpu_py312h69ecde4_54]''. We will use this version for the sake of this example, but make sure to choose the version that is most relevant for you when you follow along with these steps.
 +
 +All versions of the TensorFlow virtualenv will be stored in the common base directory, ''${WORKDIR_SW}/tensorflow-conda''; each virtualenv must have a unique name that will become the VALET version. In this tutorial, the latest version of TensorFlow built for CPUs will be installed using the tag ''cpu'' on the version:
 +
 +<code bash>
 +[(my_workgroup:user)@login00.caviness ~]$ vpkg_id2path --version-id=2.19.1:cpu
 +2.19.1-cpu
 +</code>
 +
 +The virtualenv is created using the ''%%--%%prefix'' option to direct the installation to the desired directory:
 +
 +<code bash>
 +[(my_workgroup:user)@login00.caviness ~]$ conda create --prefix=${WORKDIR_SW}/tensorflow-conda/2.19.1-cpu 'tensorflow[version=2.19.1,build=cpu_py312h69ecde4_54]'
 +   :
 +Preparing transaction: done
 +Verifying transaction: done
 +Executing transaction: done
 +#
 +# To activate this environment, use
 +#
 +#     $ conda activate /work/my_workgroup/sw/tensorflow-conda/2.19.1-cpu
 +#
 +# To deactivate an active environment, use
 +#
 +#     $ conda deactivate
 +</code>
 +
 +=== VALET package definition ===
 +
 +Assuming the workgroup does //not// already have a TensorFlow VALET package definition, the following YAML config can be modified (e.g. alter the ''prefix'' path) and added to the file ''${WORKDIR_SW}/valet/tensorflow-conda.vpkg_yaml'':
 +
 +<code yaml>
 +tensorflow-conda:
 +    prefix: /work/my_workgroup/sw/tensorflow-conda
 +    description: TensorFlow Python environments
 +    url: "https://www.tensorflow.org"
 +    
 +    flags:
 +        - no-standard-paths
 +
 +    versions:
 +        "2.19.1:cpu":
 +            description: 2.19.1, cpu_py312h69ecde4_54 build
 +            dependencies:
 +                - miniforge
 +            actions:
 +                - action: source
 +                  script:
 +                      sh: miniforge-activate.sh
 +                  success: 0 
 +</code>
 +
 +If the ''${WORKDIR_SW}/valet/tensorflow-conda.vpkg_yaml'' file already exists, add the new version at the same level as others (under the ''versions'' key):
 +
 +<code>
 +               :
 +        "2.19.1:cpu":
 +            description: 2.19.1, cpu_py312h69ecde4_54 build
 +            dependencies:
 +                - miniforge
 +            actions:
 +                - action: source
 +                  script:
 +                      sh: miniforge-activate.sh
 +                  success: 0
 +                  
 +        "2.19.1:gpu":
 +            description: 2.19.1, cuda129py312ha3fd0c4_252 build
 +               :
 +</code>
 +
 +With a properly-constructed package definition file, you can now check for your versions of TensorFlow:
 +
 +<code bash>
 +[(my_workgroup:user)@login00.caviness ~]$ vpkg_versions tensorflow-conda
 +
 +Available versions in package (* = default version):                                                                    
 +                                                                                                                        
 +[/work/my_workgroup/sw/valet/tensorflow-conda.vpkg_yaml]
 +tensorflow-conda      TensorFlow Python environments
 +* 2.19.1:cpu          2.19.1, cpu_py312h69ecde4_54 build
 +    :
 +</code>
 +
 +=== Job scripts ===
 +
 +Any job scripts designed to run scripts using this virtualenv should include something like the following toward its end:
 +
 +<code>
 +   :
 +   
 +#
 +# Setup TensorFlow virtualenv:
 +#
 +vpkg_require tensorflow-conda/2.19.1:cpu
 +
 +#
 +# Run a Python script in that virtualenv:
 +#
 +python3 my_tf_work.py
 +rc=$?
 +
 +#
 +# Do cleanup work, etc....
 +#
 +
 +#
 +# Exit with whatever exit code our Python script handed back:
 +#
 +exit $rc
 +</code>
 ===== Container images ===== ===== Container images =====
  
  • software/tensorflow/caviness.txt
  • Last modified: 2026-04-02 15:33
  • by mbotto