| Both sides previous revision Previous revision Next revision | Previous revision |
| abstract:caviness:app_dev:compute_env [2020-10-14 13:40] – [Using VALET and your UNIX environment] anita | abstract:caviness:app_dev:compute_env [2024-07-22 13:20] (current) – [Startup and logout scripts] anita |
|---|
| |
| Environment variables store dynamic system values that affect the user environment. For example, the ''PATH'' environment variable tells the operating system where to look for executables. Many UNIX commands and tools, such as the compilers, debuggers, profilers, editors, and applications with graphical user interfaces, often look at environment variables for information they need to function. The man pages for these programs typically have an ENVIRONMENT VARIABLES section with a list of variable names which tells how the program uses the values. | Environment variables store dynamic system values that affect the user environment. For example, the ''PATH'' environment variable tells the operating system where to look for executables. Many UNIX commands and tools, such as the compilers, debuggers, profilers, editors, and applications with graphical user interfaces, often look at environment variables for information they need to function. The man pages for these programs typically have an ENVIRONMENT VARIABLES section with a list of variable names which tells how the program uses the values. |
| | |
| | This is why we encourage users to use [[abstract:caviness:app_dev:compute_env#using-valet-and-your-unix-environment|VALET]] to modify your environment versus explicitly setting environment variables. |
| |
| In bash, a variable must be exported to be used as an environment variable. By convention, environment variables are all uppercase. You can display a list of currently set environment variables by typing | In bash, a variable must be exported to be used as an environment variable. By convention, environment variables are all uppercase. You can display a list of currently set environment variables by typing |
| All UNIX systems set up a default environment and provide users with the ability to execute additional UNIX commands to alter the environment. These commands are automatically sourced (executed) by your shell and define the normal and environmental variables, command aliases, and functions you need. Additionally, there is a final system-wide startup file that automatically makes global environment changes that IT sets for all users. | All UNIX systems set up a default environment and provide users with the ability to execute additional UNIX commands to alter the environment. These commands are automatically sourced (executed) by your shell and define the normal and environmental variables, command aliases, and functions you need. Additionally, there is a final system-wide startup file that automatically makes global environment changes that IT sets for all users. |
| |
| You can modify the default environment by adding lines __at the end__ of the ''~/.bash_profile'' file and the ''~/.bashrc'' file. These modifications affect shells started on the login node __and__ the compute nodes. | You can modify the default environment by adding lines __at the end__ of the ''~/.bash_profile'' file and the ''~/.bashrc'' file. These modifications affect shells started on the login node __and__ the compute nodes. In general we recommend that you __should not__ modify these files especially when software documentation refers to changing the ''PATH'' environment variable, instead use VALET to load the software. |
| |
| The ''~/.bash_profile'' file's commands are executed once at login. Add commands to this file to set your login environment and to run startup programs. | * The ''~/.bash_profile'' file's commands are executed once at login. Add commands to this file to set your login environment and to run startup programs. |
| | * The ''~/.bashrc'' file's commands are executed by each new shell you start (spawn). Add lines to this file to create aliases and bash functions. Commands such as **xterm** and **[[abstract:caviness:app_dev:compute_env#using-workgroup-and-directories|workgroup]]** automatically start a new shell and execute commands in the ''~/.bashrc'' file. The **salloc** command starts a shell on a compute node and will execute the ''~/.bashrc'' file from your home directory, but it does not execute the commands in the ''~/.bash_profile'' file. |
| |
| The ''~/.bashrc'' file's commands are executed by each new shell you start (spawn). Add lines to this file to create aliases and bash functions. Commands such as **xterm** and **[[abstract:caviness:app_dev:compute_env#using-workgroup-and-directories|workgroup]]** automatically start a new shell and execute commands in the ''~/.bashrc'' file. The **salloc** command starts a shell on a compute node and will execute the ''~/.bashrc'' file from your home directory, but it does not execute the | You may modify the IT-supplied ''~/.bash_udit'' file to be able to use several IT-supplied aliases (commands) and environment settings related to your [[abstract:caviness:app_dev:compute_env#using-workgroup-and-directories|workgroup]] and [[abstract:caviness:app_dev:compute_env#using-workgroup-and-directories|work directory]] . Edit ''.bash_udit'' and follow the directions in the file to activate these options. This is the ONLY way you should set your default workgroup at login. __DO NOT__ add the workgroup command to your ''.bashrc'' or ''.bash_profile'' as this will likely prevent you from logging in and will cause file transfer programs like WinSCP, sftp or Fetch to break. |
| commands in the ''~/.bash_profile'' file. | |
| | |
| You may modify the IT-supplied ''~/.bash_udit'' file to be able to use several IT-supplied aliases (commands) and environment settings related to your [[abstract:caviness:app_dev:compute_env#using-workgroup-and-directories|workgroup]] and [[abstract:caviness:app_dev:compute_env#using-workgroup-and-directories|work directory]] . Edit ''.bash_udit'' and follow the directions in the file to activate these options. | |
| |
| Exiting the login session or typing the **logout** command executes your ''~/.bash_logout'' file and terminates your session. Add commands to ''~/.bash_logout'' that you want to execute at logout. | Exiting the login session or typing the **logout** command executes your ''~/.bash_logout'' file and terminates your session. Add commands to ''~/.bash_logout'' that you want to execute at logout. |
| |
| <code bash> | <code bash> |
| cp /etc/skel/.bash* $HOME | cp /opt/shared/templates/homedir/.bash* $HOME |
| </code> | </code> |
| <note tip>**Where to put startup commands:** You can put bash commands in either ''~/.bashrc'' or ''~/.bash_profile''. Here are general suggestions: | <note tip>**Where to put startup commands:** You can put bash commands in either ''~/.bashrc'' or ''~/.bash_profile''. Again we do not recommend modifying these files unless you really know what you are doing. Here are general suggestions: |
| * Even if you have favorite commands from other systems, start by using the supplied files. Only modify ''.bash_udit'' for customization. | * Even if you have favorite commands from other systems, start by using the supplied files and **only** modify ''.bash_udit'' for customization. |
| * Add essential commands that you fully understand, and keep it simple. Quoting rules can be complicated. | * Add essential commands that you fully understand, and keep it simple. Quoting rules can be complicated. |
| * Do not depend on the order of command execution. Do not assume your environment, set in ''.bash_profile'', will be available when the commands in ''.bashrc'' are executed. | * Do not depend on the order of command execution. Do not assume your environment, set in ''.bash_profile'', will be available when the commands in ''.bashrc'' are executed. |
| * Do not include commands that spawn new shells, such as ''workgroup''. | * Do not include commands that spawn new shells, such as ''workgroup''. |
| * Be very careful of commands that may produce output. If you must, only execute them after a test to make sure there is a terminal to receive the output. Keep in mind using any commands that produce output may break other applications like file transfer (sftp, scp, WINscp). | * Be very careful of commands that may produce output. If you must, only execute them after a test to make sure there is a terminal to receive the output. Keep in mind using any commands that produce output may break other applications like file transfer (sftp, scp, WinSCP, FileZilla, etc.). |
| * Keep a session open on the cluster, so when you make a change that prevents you from logging on you can reverse the last change, or copy the ''/etc/skel'' file to start over. | * Do not include VALET commands as they produce output and will be a part of every job submitted which could cause conflicts with other applications you are trying to run in your job script. |
| | * Keep a session open on the cluster, so when you make a change that prevents you from logging on you can reverse the last change, or copy the original files from ''/opt/shared/templates/homedir/'' to start over. |
| </note> | </note> |
| |
| The table below shows the basic informational commands for VALET. In subsequent sections, VALET commands are illustrated in the contexts of application development (e.g., compiling, using libraries) and running IT-installed applications. | The table below shows the basic informational commands for VALET. In subsequent sections, VALET commands are illustrated in the contexts of application development (e.g., compiling, using libraries) and running IT-installed applications. |
| |
| ^ Command ^ Function ^ | ^ Command ^ Function ^ |
| | ''vpkg_help'' | VALET help. | | | ''vpkg_help'' | VALET help. | |
| | ''vpkg_list'' | List the packages that have VALET configuration files. | | | ''vpkg_list'' | List the packages that have VALET configuration files. | |
| | ''vpkg_versions ''<<//pkgid//>> | List versions available for a single package. | | | ''vpkg_versions ''<<//pkgid//>> | List versions available for a single package. | |
| | ''vpkg_info ''<<//pkgid//>> | Show information for a single package (or package version). | | | ''vpkg_info ''<<//pkgid//>> | Show information for a single package (or package version). | |
| | ''vpkg_require ''<<//pkgid//>> | Configure environment for one or more VALET packages. | | | ''vpkg_require ''<<//pkgid//>> | Configure environment for one or more VALET packages. | |
| | ''vpkg_devrequire ''<<//pkgid//>> | Configure environment for one or more VALET packages including software development variables such as ''CPPFLAGS'' and ''LDFLAGS''. | | | ''vpkg_devrequire ''<<//pkgid//>> | Configure environment for one or more VALET packages including software development variables such as ''CPPFLAGS'' and ''LDFLAGS''. | |
| | ''vpkg_rollback ''<<//# or all//>> | Each time VALET changes the environment, it makes a snapshot of your environment to which it can return.\\ ''vpkg_rollback'' attempts to restore the UNIX environment to its previous state. | | | ''vpkg_rollback ''<<//# or all//>> | Each time VALET changes the environment, it makes a snapshot of your environment to which it can return.\\ ''vpkg_rollback'' attempts to restore the UNIX environment to its previous state. You can specify a number (''#'') to revert one or more prior changes to the environment or ''all'' to remove all changes. | |
| | | You can specify a number (''#'') to revert one or more prior changes to the environment or ''all'' to remove all changes. | | | ''vpkg_history'' | List the versioned packages that have been added to the environment. | |
| | ''vpkg_history'' | List the versioned packages that have been added to the environment. | | | ''man valet'' | Complete documentation of VALET commands. | |
| | ''man valet'' | Complete documentation of VALET commands. | | |
| |
| ==== Having a personal Work area apart from the Home directory? ==== | ==== Having a personal Work directory apart from the common workgroup directory ==== |
| |
| Users can maintain their own work directories ($WORKDIR) apart from the common work directory of the workgroup. It is also possible to make Valet software to look into this user level work directory. Users can now add ~/.workdirs.yaml to override what gets set on WORKDIR for workgroups to which they are members. Detailed explanation follows. | Users can maintain their own work directories (''$WORKDIR'') apart from the common work directory of the workgroup. It is also possible to make VALET software to look into this user level work directory. Users can now add ''~/.workdirs.yaml'' to override what gets set for ''$WORKDIR'' for workgroups to which they are members. Detailed explanation follows. |
| |
| <code bash> | <code bash> |
| [traine@login00 ~]$ workdir -g it_css | [traine@login00 ~]$ workdir -g it_css |
| /work/it_nss | /work/it_css |
| |
| [traine@login00 ~]$ echo "it_css: /work/it_css/users/traine" > ~/.workdirs.yaml | [traine@login00 ~]$ echo "it_css: /work/it_css/users/traine" > ~/.workdirs.yaml |
| </code> | </code> |
| |
| From this point, valet looks at this directory for possible valet packages and also the $WORKDIR after changing to workgroup will point to the path overriden in workdirs.yaml file. | From this point, valet looks at this directory for possible valet packages and also the ''$WORKDIR'' after changing to workgroup will point to the path overridden in ''workdirs.yaml'' file. |
| |
| <code bash> | <code bash> |
| </code> | </code> |
| |
| To revert back to the defaults : | To revert back to the defaults: |
| |
| <code bash> | <code bash> |
| [traine@login00 ~]$ rm ~/.workdirs.yaml | [traine@login00 ~]$ rm ~/.workdirs.yaml |
| |
| traine@login00 ~]$ workgroup -g it_nss | traine@login00 ~]$ workgroup -g it_css |
| |
| [(it_css:traine)@login00 ~]$ echo $WORKDIR | [(it_css:traine)@login00 ~]$ echo $WORKDIR |