R9 Software Stack

From ACCRE Wiki

Revision as of 04:39, 10 January 2025 by Dhakas1 (talk | contribs) (→‎Notes)

The R9 cluster now supports software provided by the Digital Research Alliance of Canada (the Alliance). These software packages, distributed using the CVMFS file system, will be available to all users. For the software modules not provided by the Alliance, we will maintain and manage our own software list to complement the Alliance's offerings.

Key Notes

  • Software Stack Not Loaded by Default : The software stack on R9 is not automatically loaded into your environment. To load it, use the command:
setup_accre_software_stack

Refer to the Using the New Software Stack section for more details.

  • Choice Between Software Stacks:
    • Users can choose between the Alliance-based stack (new) and the legacy CentOS 7-based stack.
    • We do not guarantee compatibility of all CentOS 7-based software with the new R9 operating system.

Using the New Software Stack

Setting Up the Software Stack

The setup_accre_software_stack command configures the environment based on the ACCRE_SOFTWARE_STACK environment variable. Here are the available options for the variable ACCRE_SOFTWARE_STACK:

  • accre_cc (default): Loads the new Alliance-based software stack.
  • accre_legacy: Loads the legacy CentOS 7-based stack.

To switch to the legacy stack, set the variable to accre_legacy before running the setup command: 

export ACCRE_SOFTWARE_STACK=accre_legacy  
setup_accre_software_stack

Autoloading the stack at login

Depending on the shell you use, to automatically load the software stack at login, you can add the setup command to your rc file. The .bashrc file is a hidden configuration script in your home directory. It runs automatically whenever you log in using the Bash shell.

Example .bashrc snippet: 

# .bashrc  
# Source global definitions  
if [ -f /etc/bashrc ]; then  
    . /etc/bashrc  
fi 
 
...

# accre_cc is the default; other option is accre_legacy
ACCRE_SOFTWARE_STACK=accre_cc
# Load ACCRE software stack  
setup_accre_software_stack

Loading Software

Lmod is a powerful tool for managing software modules in shared high-performance computing environments. The process for loading software from the new stack remains unchanged from the old stack. For detailed instructions on using Lmod commands, please visit the Software Stack page.

Notes

  • Python, R, and Perl Packages: We no longer accept requests for installing individual Python, R, or Perl packages. Users are responsible to manage their own installations. Refer to the relevant pages for guidance:
  • Singularity is Now Apptainer : Containers, including Docker containers, can now be run using the apptainer command.
  • OS-Related Packages : Many OS-related packages (e.g., Git) are included in the StdEnv environment. These packages are available to use without having to use module load command.

Legacy Software Stack

We will no longer maintain our old CentOS 7-based software stack. All new software requests will be installed exclusively on the new stack. If you notice software missing in the new stack, submit a software installation request from PI Dashboard. We'll review and try to include it in the stack promptly. In the meantime, you have two options for accessing legacy software:

Using the Legacy Stack on R9

Set the ACCRE_SOFTWARE_STACK environment variable to "accre_legacy" to continue using the legacy software stack. Since those software were compiled under CentOS7 environment, there are no guarantees that the software will work properly with Rocky 9 Linux.

Running Legacy Software in a Container

We provide an Apptainer container with a CentOS 7 environment for running legacy software. The container can be accessed from /panfs/accrepfs.vampire/accre/common/singularity/lmod_c7.sif directory. Make sure you bind /cvmfs and /accre directory to the container. 

# Example apptainer command.
apptainer run --bind /cvmfs:/cvmfs:ro,/accre:/accre:ro /panfs/accrepfs.vampire/accre/common/singularity/lmod_c7.sif /bin/bash

Additionally, for convenience, we have added a legacy_load module to the new stack. This module creates an alias in your environment, forwarding arguments directly to the Apptainer container. 

[~]$ module spider legacy_load

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  legacy_load: legacy_load/1.0
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    This module can be loaded directly: module load legacy_load/1.0

    Help:
      legacy_load (Run software from legacy CentOS 7 stack in containerized environment)

[~]$ module load legacy_load
------------------------------------------------------------
This helper alias command assists with running software from our legacy CentOS 7 stack under an apptainer container.
For greater flexibility, you can use the apptainer commands yourself with the container located at /panfs/accrepfs.vampire/accre/common/singularity/lmod_c7.sif.
Usage : legacy_load "<commands to pass to the container>"
Example : legacy_load "module load ffmpeg; ffmpeg -i input.mp4 -q:a 0 -map a output.mp3"
------------------------------------------------------------

[~]$ legacy_load "module load GCCcore/.11.3.0 FFmpeg;ffmpeg"
ffmpeg version 5.0.1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 11.3.0 (GCC)
  configuration: --prefix=/accre/arch/easybuild/software/Compiler/GCCcore/11.3.0/FFmpeg/5.0.1 --enable-pic --enable-shared --enable-gpl --enable-version3 --enable-nonfree --cc=gcc --cxx=g++ --enable-libx264 --enable-libx265 --enable-libmp3lame --enable-libfreetype --enable-fontconfig --enable-libfribidi --enable-sdl2
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...
Retrieved from "https://help.accre.vanderbilt.edu/index.php?title=R9_Software_Stack&oldid=826"