hpc documentation

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Miniforge on DARWIN ======

Miniforge is available as a VALET package on DARWIN. To check available versions, use:

<code>
$ vpkg_versions miniforge

Available versions in package (* = default version):

[/opt/shared/valet/2.1/etc/miniforge.vpkg_yaml]
miniforge    Tthe preferred conda-forge installer and includes conda, mamba, and their dependencies
* 25.11.0-1  Miniforge py312 25.11.0-1 for Linux (x86_64)
</code>

To load Miniforge, use:

<code>
$ vpkg_require miniforge/25.11.0-1
Adding package `miniforge/25.11.0-1` to your environment
</code>

===== Creating Python virtual environments with Conda =====

Once you have loaded Miniforge, you can create a new Conda environment with the ''conda create'' command, passing in the packages you want to include in the environment. We also strongly recommend using the ''%%--prefix%%'' option to specify a path for your environment in your [[abstract:darwin:filesystems:filesystems#workgroup-storage|workgroup storage]] to avoid filling up your home storage. It may be convenient for you to use the ''WORKDIR_SW'' environment variable, which is set automatically when you start a shell in your workgroup.

For example, to create an environment in your workgroup storage that includes the latest version of Python detected by Conda:

<code bash>
[(my_workgroup:user)@login00.darwin ~]$ conda create --prefix=${WORKDIR_SW}/python-env python
</code>

<note important>
Make sure to list all of the packages you need in the ''conda create'' command. Conda recommends that you list all the packages you want to install at once (rather than in separate ''conda install'' commands after creating the environment) to avoid dependency conflicts.
</note>

<note important>
By default, Conda environments created by ''%%--%%name'' or ''-n'' are in your home directory. However, the limited home directory quotas (20 GB) can be filled up quickly. It is recommended to create environments by specifying path ''%%--prefix%%'' in your work directory instead of ''$HOME'' to avoid quota issues.
</note>

After creating the Conda environment, activate it with:

<code bash>
[(my_workgroup:user)@login00.darwin ~]$ conda activate ${WORKDIR_SW}/python-env
</code>

To list packages in the currently active environment, use:

<code bash>
(/lustre/my_workgroup/sw/python-env) [(my_workgroup:user)@login00.darwin ~]$ conda list
# packages in environment at /lustre/my_workgroup/sw/python-env:
#
# Name                     Version          Build                 Channel
_openmp_mutex              4.5              20_gnu                conda-forge
bzip2                      1.0.8            hda65f42_9            conda-forge
ca-certificates            2026.2.25        hbd8a1cb_0            conda-forge
icu                        78.3             h33c6efd_0            conda-forge
ld_impl_linux-64           2.45.1           default_hbd61a6d_102  conda-forge
libexpat                   2.7.4            hecca717_0            conda-forge
libffi                     3.5.2            h3435931_0            conda-forge
libgcc                     15.2.0           he0feb66_18           conda-forge
libgomp                    15.2.0           he0feb66_18           conda-forge
liblzma                    5.8.2            hb03c661_0            conda-forge
libmpdec                   4.0.0            hb03c661_1            conda-forge
libsqlite                  3.52.0           hf4e2dac_0            conda-forge
libstdcxx                  15.2.0           h934c35e_18           conda-forge
libuuid                    2.41.3           h5347b49_0            conda-forge
libzlib                    1.3.2            h25fd6f3_2            conda-forge
ncurses                    6.5              h2d0b736_3            conda-forge
openssl                    3.6.1            h35e630c_1            conda-forge
pip                        26.0.1           pyh145f28c_0          conda-forge
python                     3.14.3           h32b2ec7_101_cp314    conda-forge
python_abi                 3.14             8_cp314               conda-forge
readline                   8.3              h853b02a_0            conda-forge
tk                         8.6.13           noxft_h366c992_103    conda-forge
tzdata                     2025c            hc9c84f9_1            conda-forge
zstd                       1.5.7            hb78ec9c_6            conda-forge
</code>

To deactivate the environment:

<code bash>
(/lustre/my_workgroup/sw/python-env) [(my_workgroup:user)@login00.darwin ~]$ conda deactivate
[(my_workgroup:user)@login00.darwin ~]$
</code>

To remove the environment:

<code bash>
[(my_workgroup:user)@login00.darwin ~]$ conda remove --prefix=${WORKDIR_SW}/python-env --all
</code>

To remove the unused packages and caches:

<code bash>
[(my_workgroup:user)@login00.darwin ~]$ conda clean --all
</code>

===== Migrating Environments =====

While an environment is active, you can export a list of the packages installed in the environment that can be used to recreate the environment elsewhere. For example, with the Python environment from the previous section active:

<code bash>
[(my_workgroup:user)@login00.darwin ~]$ conda activate ${WORKDIR_SW}/python-env
(/lustre/my_workgroup/sw/python-env) [(my_workgroup:user)@login00.darwin ~]$ conda export --format=environment-yaml --file=python_env.yaml
</code>

Then, you can recreate the same environment with:
<code bash>
(/lustre/my_workgroup/sw/python-env) [(my_workgroup:user)@login00.darwin ~]$ conda deactivate
[(my_workgroup:user)@login00.darwin ~]$ conda create --prefix=${WORKDIR_SW}/python-env-copy --file=python_env.yaml
</code>