Appropriate use of RCI Clusters as a VSCode Backend
Microsoft's Visual Studio Code (VSCode) is a free source code editor that supports various programming languages and offers a wealth of extensions. The Remote-SSH extension enhances the VSCode's capabilities by allowing developers to work on the remote execution of code. This is intended to allow the developer to edit/run/debug in the (eventual) target environment or access hardware not present in the local host (e.g., GPUs).
There are several issues with this when the target environment is an HPC cluster:
- The remote infrastructure (shell and Python scripts) is left running indefinitely.
- Users who become reliant on VSCode as a production environment will inevitably have their work killed (since it will be executed outside of job scheduling).
- Any specialized hardware (like GPUs, large RAM) is not present in login nodes.
The clusters currently in production at UD have multiple login nodes balanced via round-robin DNS resolution of a single hostname. With regard to the first point above, the local VSCode application will "remember" the session it previously created and attempt to reconnect to it. But since repeated DNS resolution of the hostname may not produce the same IP address, that session may not always be found — and a new one will be created while the old one persists on a different login node. We often see users accumulate many of these orphaned VSCode remote sessions on our login nodes.
An attempt at proxying VSCode remote sessions through the cluster job scheduler seemed like a worthy project. If the VSCode app's shell commands were forwarded to an interactive job running on a compute node, then the remote execution infrastructure would:
- have access to specialized hardware by virtue of the parameters associated with the interactive job.
- not consume significant CPU resources on the login node.
- be automatically terminated when the interactive job is completed.
vscode-shell-proxy
has been developed to achieve the above goals. It requires each user to configure VSCode Remote-SSH extension in the application to use vscode-shell-proxy
as the command to connect. When configured properly, our tests show using vscode-shell-proxy
eliminates the issues stated above for Caviness and DARWIN.