Visualization Portal

The ACCRE Visualization Portal is a graphical interface to the ACCRE cluster powered by Open OnDemand. This interface allows users to access all cluster resources conveniently through modern web browsers, and provides the ability to create virtual desktop jobs that create a temporary desktop environment within the cluster on a compute node for interactive work and plotting data.

The Visualization Portal is considered a "best effort" service as it is the result of an employee-driven initiative that does not receive specific funding or staffing from Vanderbilt University or any grant, and is offered free of charge. We believe that this system is generally suitable for production usage, but our time to further develop the system to meet the needs of our clients is limited. Any user feedback is appreciated, so please open a helpdesk ticket if you encounter difficulties or have suggestions. If our time allows, we will endeavor to improve the service to meet the needs of ACCRE users, but it may not be feasible to entertain all feature requests given the limitations of our support.

Accessing the Portal

The portal is accessed through any modern, up-to-date web browser at viz.accre.vu. This address will redirect you to keycloak.accre.vu where you can enter your VUNet ID and password to access your account, and redirect you back to viz.accre.vu.

Do not attempt to log into the portal from untrusted machines or browsers that are not fully up to date. Do not under any circumstances enter your password into a site claiming to be related to ACCRE portal authentication that is not keycloak.accre.vu or does not offer a valid certificate.

Always remember to promptly sign out when you have completed your session or are away from your desk. Signing out of the portal will not affect any running jobs or interactive desktops – these will still be running when you sign back in.

The File Explorer

Clicking on the “Home Directory” item in the “files” menu will take you to the file explorer application. This application allows you to browse your directory tree, and securely upload and download files to your workstation. You can view text files as well as standard image formats such as JPG and PNG in the browser window.

The file explorer will also allow you to edit files with the ACE text editor. This modern editor provides syntax highlighting for most languages, automatic indentation, and a number of other features designed with software development in mind.

Gateway Shell Access

Under the “clusters” menu the “ACCRE Shell Access” item will open a tab with a typical ssh login to a gateway node, only through your web browser rather than through any software installed on your workstation.

Interactive Applications

Interactive Applications are the primary feature of the Visualization Portal at ACCRE. These allow you to run Linux Desktops, Jupyter Notebooks, or RStudio sessions through your web browser using resources available to your user via Slurm. These features are run as normal Slurm jobs but are submitted through a graphical interface where you choose the resources that you need.

These applications work best when run under the interactive or interactive_gpu partitions as these partitions are designed to allow groups immediate access to resources without having to wait in the queue. However, it is also possible to run these applications in the batch or batch_gpu partitions. If you need to access to resources in the batch or batch_gpu partition, then we recommend requesting your application job 24 hours in advance if possible and requesting anywhere from a few days to a week.

Currently there are three primary applications detailed below: The ACCRE Desktop, The ACCRE JupyterLab Server, and the ACCRE RStudio Server. These have been rewritten for the current ACCRE environment based on the Rocky 9 Linux distribution. However, in order to continue service for existing workflows, we continue to make several legacy applications available as well.

ACCRE Desktops

Under the “Interactive Apps” menu the “ACCRE Desktop” item will open a tab where you can request a temporary desktop on an ACCRE compute node to do interactive, graphical work with applications such as MATLAB or Python Matplotlib. The way this works is that you only request the resources you need in terms of CPU cores, memory, and wall-clock time, just as a regular batch job. The specific resources to be requested will vary depending on the partition selected, see the SLURM documentation for details about each partition. The resource request is processed just like any normal job, so it may have to wait in the queue until resources are ready. Once ready, you will see an entry for your desktop if you click the “My Interactive Sessions” link in the top navigation bar. There will be a link to open the desktop in a new tab and another sharable link to access the desktop in view-only mode, so that you can screen share with collaborators.

The desktop will remain active when you close your browser tab or log out from the portal. It will only be deleted when you request deletion from the portal or press the “log out” button from within the desktop, or when time expires.

When you create a desktop, you set the initial screen resolution. You can click the “reset resolution” button will set the size to that of your browser window. If you would prefer for the desktop resolution to rescale to your display, you can fix this from within the desktop. Click on the NoVNC tab that is partially hidden on the far left of the desktop window to bring it out. Then click on the gear icon to access settings. Change the “Scaling Mode” to “Remote Resizing” to make the desktop screen resize to your browser window. Note that resizing the desktop may cause some software packages such as MATLAB to report errors, which is why this is turned off by default. If your desktop is larger than your current browser window, you may move around the desktop by clicking on the hand icon in the NoVNC tab and dragging the desktop around within your browser window.

JupyterLab Servers

Under the “Interactive Apps” menu the “ACCRE JupyterLab” item will open a tab where you can request a JupyterLab Notebook Interface on an ACCRE compute node to run notebooks with Python, Julai, or R kernels. You must also specify the working directory for your notebook, which may be your home directory or another directory that you specify. As with the interactive desktop, the way this works is that you only request the resources you need in terms of CPU cores, memory, and wall-clock time, just as a regular batch job. If the batch or batch_gpu partition is specified, then the job may have to wait in the queue until resources are ready. Once ready, you will see an entry for your JupyterLab server if you click the “My Interactive Sessions” link in the top navigation bar. There will be a link to open a JupyterLab session in a new tab.

The notebook server and any running kernels will remain active when you close your browser tab or log out from the portal. It will only be deleted when you request deletion from the portal or when time expires.

Implementation Details

By default this application will use python modules from the Digital Research Alliance of Canada (alliancecan.ca) software stack. These modules will include a reasonably up to date version of python, support for kernels in other commonly used languages, and the now informally defined "SciPy Stack" collection which includes commonly used python tools such as numpy, pandas, and matplotlib.

The modules used are collected into a bundle and given the name ACCREJupyterLab and version so ACCRE users can easily reproduce the environment of the application in their own batch jobs or other python scripts used on the ACCRE cluster. Different versions of the bundle will be made available as new packages are added to the software stack in the future. We generally recommend using the latest available version unless there is a compatibility or other issue forcing you to use an older bundle. You can see the exact modules and versions of the modules that comprise the bundle with the module show command after setting up the software stack in your shell environment, for example module show ACCREJupyterLab/2025a will show the module names and versions of the modules that comprise the 2025a version.

This application works by installing a temporary python virtual environment on the local disk drive of the node that your job runs on as documented in the ACCRE Python Documentation Page. You can request additional python packages to be installed in this python environment by providing a list of packages or by giving the path to a python requirements text file. All packages to be installed in the temporary environment must be available in the Digital Research Alliance of Canada python "WheelHouse". Note that the creation of this temporary virtual environment may add a minute or two delay to the start of your server but will then generally improve performance over the life of your session.

Converting your Notebook to a Batch Job

Adapting your notebook code to a python script is the responsibility of the user and the necessary changes may vary depending on how Jupyter is used in your notebook. This section does not detail how to create a stand-alone python script from a notebook, but does detail how to re-create an identical python environment as was used in the portal app.

For this example, we will assume that the ACCRE user bob has been converted a jupyter notebook into a script bobs_analyzer.py which needs to then run in a job array of 50 jobs on the batch partition. For each job, the script should be called using an integer argument such as bobs_analyzer.py 3 where the integer used should go from 1 to 50. Each job in the array will require 5 hours, 4 physical CPU cores, 24GiB of system memory, and will also require the redis package to be installed into the virtual environment as it is imported in the python analysis script. Furthermore, when Bob ran his JupyterLab server through the portal, he selected the ACCREJupyterLab/2025a bundle.

Bob's batch submission script to perform his job array might then look like this:

#!/bin/bash
 
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH --time=5:00:00
#SBATCH --mem=24G
#SBATCH --account=bob_lab
#SBATCH --array=1-50

setup_accre_software_stack
module load ACCREJupyterLab/2025a

source setup_accre_runtime_dir
echo "Using temporary directory ${ACCRE_RUNTIME_DIR}"

python -m venv ${ACCRE_RUNTIME_DIR}/venv
source ${ACCRE_RUNTIME_DIR}/venv/bin/activate

pip install --no-index --upgrade pip
pip install --no-index -r redis

./python bobs_analyzer.py ${SLURM_ARRAY_TASK_ID}

Advanced Usage

For many users the default setup of using the provided module bundles from the software stack as well as optional packages from the "Wheelhouse" should be sufficient for their interactive analysis work. However, there may be special software or precise versions needed in some cases that is not available. To accommodate such workflows, the JupyterLab app has normally hidden advanced features allowing users to bring their own environments.

If you need your own environment, our recommendation whenever possible is to build an Apptainer (formerly Singularity) container with your required environment. This has the advantage of fully isolating your environment from the one provided by the software stack and the ACCRE cluster environment, and allows a researcher to easily migrate their environment to other clusters at other institutions. If you select the option to use a Apptainer container, you must provide a path to your container image file in a location accessible to the compute node running your job. Your container must be capable of running the jupyter lab command via apptainer exec, or the JupyterLab server app will not function with your container. When using a container, the software stack will not be loaded. All required software must be installed inside your container.

Using an existing python virtual environment is also supported. For more information on the use of virtual environments and Python on the ACCRE cluster, see the Python Documentation Page. When you use an existing virtual environment, that environment must have been created with the same version of the ACCREJupyterLab module bundle loaded as you select on the portal application. All modules will be loaded before the virtual environment is activated. You must indicate the absolute path to your virtual environment, and your virtual environment must have jupyter installed and allow the jupyter lab command to be run. If you have an alternative module collection that you wish to use instead of the ACCREJupyterLab bundle, you can configure the app to use your module collection instead. If your python virtual environment was created outside of the software stack, perhaps using an installation with pyenv or uv, then you may elect to not load the software stack at all. Anaconda virtual environments are not supported.

RStudio Servers

The RStudio app uses an R and RStudio installation that is separate from the software stack and provided via an Apptainer container. While ACCRE tries to keep a reasonably new version of R and RStudio available, we may not always have the latest release or all the packages that certain workflows may require.

ACCRE is currently working on support for users to bring their own container to the app with the software they require, and this description will be updated once that feature is ready.

Please Note: RStudio is an interactive development environment (IDE), not a batch processing engine. It is designed for writing quick checks and debugging code, not for executing long-running or memory-intensive jobs. Also, RStudio requires a constant "heartbeat" between the UI and the R engine to show progress/state of the code execution. If the engine hangs due to high CPU or swap usage, or if the UI looses connection with the engine because of browser session timeouts, the Rstudio UI can become unresponsive. Thus, it is recommended to the users to use RStudio to develop quick checks and tests, on small data samples. For large datasets, or long running jobs, please submit your code as a slurm job using Rscript. Please read more about submitting slurm job with R

Legacy Applications

Several "Legacy" Applications are available to support workflows that were in use before the current reorganization of the Visualization Portal Interactive Applications. These are provided "as-is" for the foreseeable future so that users may continue to use the same workflows they are used to previously. Note that these will not allow access to the interactive or interactive_gpu partitions. We do not recommend the use of these for new analysis workflows.