R9 Software Stack
The R9 cluster now supports software provided by the Digital Research Alliance of Canada (the Alliance). These software packages, distributed using the CVMFS file system, will be available to all users. For the software modules not provided by the Alliance, we will maintain and manage our own software list to complement the Alliance's offerings.
Key Notes
- Software Stack Not Loaded by Default : The software stack on R9 is not automatically loaded into your environment. To load it, use the command:
setup_accre_software_stack
Refer to the Using the New Software Stack section for more details.
- Choice Between Software Stacks:
- Users can choose between the Alliance-based stack (new) or may choose not to load a software stack and use their own tools on top of the standard Rocky 9 Linux environment.
- In the future, additional software stacks from other providers may be made available
- The use of the legacy CentOS 7 software stack may be accessed through a provided container, see legacy software stack section below.
Using the New Software Stack
Setting Up the Software Stack
The setup_accre_software_stack command configures the environment based on the ACCRE_SOFTWARE_STACK environment variable. accre_cc is the default and currently the only supported option for ACCRE_SOFTWARE_STACK variable. The variable will allow for us to make multiple software stacks easily available.
Autoloading the stack at login
Depending on the shell you use, to automatically load the software stack at login, you can add the setup command to your rc file. The .bashrc file is a hidden configuration script in your home directory. It runs automatically whenever you log in using the Bash shell.
Example .bashrc snippet:
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
...
# accre_cc is the default; other option is accre_legacy
export ACCRE_SOFTWARE_STACK=accre_cc
# Load ACCRE software stack
setup_accre_software_stack
Loading Software
Lmod is a powerful tool for managing software modules in shared high-performance computing environments. The process for loading software from the new stack remains unchanged from the old stack. For detailed instructions on using Lmod commands, please visit the existing Software Stack page.
Notes
- Python, R, and Perl Packages: We no longer accept requests for installing individual Python, R, or Perl packages. Users are responsible to manage their own installations. Refer to the relevant section for guidance:
- Singularity is Now Apptainer : Containers, including Docker containers, can now be run using the apptainer command.
- OS-Related Packages : Many OS-related packages (e.g., Git) are included in the StdEnv environment. These packages are available to use without having to use
module loadcommand.
Legacy Software Stack
We will no longer maintain our old CentOS 7-based software stack.
Running Legacy Software in a Container
We provide an Apptainer container with a CentOS 7 environment for running legacy software. The container can be accessed from /panfs/accrepfs.vampire/accre/common/singularity/lmod_c7.sif directory. Make sure you bind /cvmfs and /accre directory to the container. As running apptainer run will pass the environment variables in the session to the container, make sure you have a fresh session and have not loaded our software stack with setup_accre_software_stack command.
# Example apptainer command. apptainer run --bind /cvmfs:/cvmfs:ro,/accre:/accre:ro /panfs/accrepfs.vampire/accre/common/singularity/lmod_c7.sif /bin/bash
Additionally, for convenience, we have added a legacy_load module to the new stack. This module creates an alias in your environment, forwarding arguments directly to the Apptainer container.
[~]$ module spider legacy_load
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
legacy_load: legacy_load/1.0
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
This module can be loaded directly: module load legacy_load/1.0
Help:
legacy_load (Run software from legacy CentOS 7 stack in containerized environment)
[~]$ module load legacy_load ------------------------------------------------------------ This helper alias command assists with running software from our legacy CentOS 7 stack under an apptainer container. For greater flexibility, you can use the apptainer commands yourself with the container located at /panfs/accrepfs.vampire/accre/common/singularity/lmod_c7.sif. Usage : legacy_load "<commands to pass to the container>" Example : legacy_load "module load ffmpeg; ffmpeg -i input.mp4 -q:a 0 -map a output.mp3" ------------------------------------------------------------
[~]$ legacy_load "module load GCCcore/.11.3.0 FFmpeg;ffmpeg"
ffmpeg version 5.0.1 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 11.3.0 (GCC)
configuration: --prefix=/accre/arch/easybuild/software/Compiler/GCCcore/11.3.0/FFmpeg/5.0.1 --enable-pic --enable-shared --enable-gpl --enable-version3 --enable-nonfree --cc=gcc --cxx=g++ --enable-libx264 --enable-libx265 --enable-libmp3lame --enable-libfreetype --enable-fontconfig --enable-libfribidi --enable-sdl2
libavutil 57. 17.100 / 57. 17.100
libavcodec 59. 18.100 / 59. 18.100
libavformat 59. 16.100 / 59. 16.100
libavdevice 59. 4.100 / 59. 4.100
libavfilter 8. 24.100 / 8. 24.100
libswscale 6. 4.100 / 6. 4.100
libswresample 4. 3.100 / 4. 3.100
libpostproc 56. 3.100 / 56. 3.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...
Using the Legacy Stack on R9
Set the ACCRE_SOFTWARE_STACK environment variable to accre_legacy to use the legacy software stack from old cluster. Since those software were compiled under CentOS7 environment, there are no guarantees that the software will work properly with Rocky 9 Linux. Therefore, we strongly advise against using this unless using the container option is not feasible for your work.
export ACCRE_SOFTWARE_STACK=accre_legacy setup_accre_software_stack
Python
Although you will get a Python interpreter from our base OS package without loading the software stack, it is generally not the one you should use, especially if you need to install external Python packages. You should load the appropriate version of Python through the module command. When in doubt, use the latest available version of Python in the software stack.
For convenience, in addition to the base Python module, the SciPy package is also available as a module. The scipy-stack module includes:
- NumPy
- SciPy
- Matplotlib
- IPython
- pandas
- SymPy
- Nose
The pip command can install packages from a variety of sources, including PyPI and prebuilt distribution packages called Python wheels. Alliance provides Python wheels for a number of packages in their CVMFS repository. You can run the following command to get the list of available wheels:
avail_wheels
Using wheels provided by Alliance can substantially reduce installation time. You just have to add the --no-index option to pip to prevent installation from PyPI and install only from Alliance's wheelhouse. If you want to ignore prebuilt packages entirely, you can use the --no-binary option in pip.
pip install numpy --no-index
R
To install packages from CRAN in an interactive R session, use the install.packages function.
We recommend loading a GCC module before installing any R packages since most R packages have dependencies on it.
module load gcc/9.3.0 r/4.0.2
Perl
Perl packages can be installed using the Comprehensive Perl Archive Network (CPAN). You can use the cpan tool in the software stack to install packages in your home directory.
We recommend loading a GCC module before installing any Perl packages through CPAN:
module load gcc/9.3.0
To install a Perl package, enter the CPAN shell and use the install command:
$ cpan $ cpan[1]> install Chess
Apptainer/Singularity
In the C7 environment, common directories such as /data, /panfs, and /home are automatically bind-mounted into Singularity containers by default. However, in the R9 environment, this automatic binding is not enabled. Users must manually specify bind mounts using the APPTAINER_BINDPATH environment variable or the --bind flag when launching a container.
1. Using the APPTAINER_BINDPATH Environment Variable
You can predefine bind mounts by setting the APPTAINER_BINDPATH variable before running the container:
export APPTAINER_BINDPATH="/panfs,/data"
This method ensures that the specified directories are consistently mounted for all subsequent container executions within the session.
2. Using the --bind Flag
You can also specify bind mounts directly when starting an Apptainer container:
apptainer shell --bind=/panfs:/panfs,/data:/data container.sif
This method is useful when you want to define bind mounts on a per-command basis.
