Next revision | Previous revision Next revisionBoth sides next revision |
technical:recipes:keras-in-virtualenv [2020-02-04 12:22] – created frey | technical:recipes:keras-in-virtualenv [2020-02-04 12:37] – frey |
---|
[(my_workgroup:user)@login01 ~]$ vpkg_require intel-python/2019u5:python3 | [(my_workgroup:user)@login01 ~]$ vpkg_require intel-python/2019u5:python3 |
Adding package `intel-python/2019u5:python3` to your environment | Adding package `intel-python/2019u5:python3` to your environment |
(root) [(it_nss:frey)@login01 ~]$ | (root) [(my_workgroup:user)@login01 ~]$ |
</code> | </code> |
| |
| |
<code bash> | <code bash> |
(root) [(it_nss:frey)@login01 keras]$ conda create --prefix=${WORKDIR}/sw/keras/20200204-sklearn keras numpy pip | (root) [(my_workgroup:user)@login01 ~]$ conda create --prefix=${WORKDIR}/sw/keras/20200204-sklearn keras numpy pip |
</code> | </code> |
| |
</code> | </code> |
| |
Answer "y" to create the virtualenv. If successful, text will be displayed that mentions source'ing a file to activate the virtualenv: refrain from doing that, and instead use VALET to manage the Keras virtualenv instances. | Answer "y" to create the virtualenv. If successful, text will be displayed that mentions source'ing a file to activate the virtualenv: refrain from doing that, and instead use VALET to manage the Keras virtualenv instances. Rollback the ''intel-python'' environment changes before proceeding: |
| |
| <code bash> |
| (root) [(my_workgroup:user)@login01 ~]$ vpkg_rollback |
| [(my_workgroup:user)@login01 ~]$ |
| </code> |
| |
| Notice the ''(root)'' has disappeared from the prompt, indicating that the baseline virtualenv has been deactivated. |
===== VALET Package Definition ===== | ===== VALET Package Definition ===== |
| |
</file> | </file> |
| |
would be added to ''${WORKDIR}/sw/valet/keras.vpkg_yaml''. If the file already exists, add your new version at the same level as others: | would be added to ''${WORKDIR}/sw/valet/keras.vpkg_yaml''. If that file already exists, add your new version at the same level as others: |
| |
<file keras.vpkg_yaml> | <file keras.vpkg_yaml> |
</file> | </file> |
| |
<note tip>On Caviness, VALET consults ''${WORKDIR}/sw/valet'' by default after the user has used the ''workgroup'' command.</note> | <note tip>On Caviness after a user has used the ''workgroup'' command, VALET searches for package definitions in ''${WORKDIR}/sw/valet'' by default.</note> |
| |
| ===== Install SKLearn ===== |
| |
| The SKLearn package is not present in the conda online repositories, but it can be installed using ''pip.'' First, activate the new Keras virtualenv: |
| |
| <code bash> |
| [(my_workgroup:user)@login01 ~]$ vpkg_require keras/20200204:sklearn |
| Adding dependency `intel-python/2019u5:python3` to your environment |
| Adding package `keras/20200204:sklearn` to your environment |
| (/work/my_workgroup/sw/keras/20200204-sklearn) [(my_workgroup:user)@login01 keras]$ which pip |
| /work/my_workgroup/sw/keras/20200204-sklearn/bin/pip |
| </code> |
| |
| A prefix has reappeared on the prompt — the path at which the new Keras virtualenv was created — and the ''pip'' command refers to an executable within that directory tree. The virtualenv is ready to have SKLearn installed. |
| |
| <code bash> |
| (/work/my_workgroup/sw/keras/20200204-sklearn) [(my_workgroup:user)@login01 ~]$ pip install sklearn |
| Collecting sklearn |
| Downloading .. |
| : |
| Successfully built sklearn |
| Installing collected packages: joblib, scikit-learn, sklearn |
| Successfully installed joblib-0.14.1 scikit-learn-0.22.1 sklearn-0.0 |
| (/work/my_workgroup/sw/keras/20200204-sklearn) [(my_workgroup:user)@login01 ~]$ python3 -c "import sklearn;print(sklearn.__version__)" |
| 0.22.1 |
| </code> |
| |
| The Keras environment with SKLearn is now ready for use. |
| |
| ===== Job Scripts ===== |
| |
| Any job scripts you submit that want to run scripts using this virtualenv should include something like the following: |
| |
| <code> |
| # |
| # Setup Keras virtualenv: |
| # |
| vpkg_require keras/20200204:sklearn |
| |
| # |
| # Run a Python script in that virtualenv: |
| # |
| python3 my_keras_work.py |
| rc=$? |
| |
| # |
| # Do cleanup work, etc.... |
| # |
| |
| # |
| # Exit with whatever exit code our Python script handed back: |
| # |
| exit $rc |
| </code> |