Overview of Software Stack

The software stack is a suit of software ACCRE builds for the users to use. In general the ACCRE has two types of software stack: binary software stack and compiler based software stack. The binary software stack typically includes the stand-alone software which does not need to compile for installation, and the compiler based software stack contains the software that is compiled for installation from a given version of compiler (for example, GCC 10.2). Usually we use the compiler and a year tag to name the compiler based software stack.

Our current active compiler based software stack includes the followings:

GCC 2016b (based on GCC/5.4.0-2.26) and Intel 2016b (based on Intel/2016.3.210);
GCC 2017b (based on GCC/6.4.0-2.28) and Intel 2017b (based on Intel/2017.4.196);
GCC 2019a (based on GCC/8.2.0) and Intel 2019a (based on Intel/2019.1.144);
GCC 2020a (based on GCC/10.2.0);
GCC 2022a (still in progress, based on GCC/11.3.0)

Usually we upgrade software stack every two years, and since the 2020a software stack we only provide Intel compiler and corresponding libraries for user to use, we do not compile the software based on the Intel compiler and libraries. However if you think you really need it and you have a good reason for the request, please open a helpdesk ticket with us so we can build a specific version for you.

Sometimes to build a software also need additional cluster libraries such as MPI. Therefore, based on the compiler, we grouped some additional cluster libraries to form a general compiler tool set for the user to use. These additional cluster libraries include the MPI library (openMPI for GCC and IntelMPI for Intel compiler), blas and lapack libraries (openBLAS and ScaLAPACK for GCC and MKL for Intel), and other math libraries such as fftw. For the compiler tool set which are based on GCC compilers they are called foss tool set. The available foss tool set include the followings:

foss 2016b;
foss 2017b;
foss 2019a;
foss 2020a;
foss 2022a

Before foss 2022a, all of foss toolchain are hidden module so you need to load it with a dot sign:

module load foss/.2020a

For the GPU nodes, we also add in cuda into the foss tool set so on GPU we have fosscuda tool set. For all of above foss versions they also have corresponding fosscuda tool set on GPU nodes.

Software Stack on Compute Nodes

All output looks like this:
*** caught illegal operation ***
address 0x2b02bd5762bc, cause 'illegal operand'

because you compiled the R library on newer CPU but tried to run it on the older CPU. Therefore during the the running the CPU can not recognize the newer instruction set hence the program got problem.

In this case, the simplest way to resolve it is to recompile all of your R libraries on our gateway, since they are with the oldest CPU in our cluster. Or you can try the following:

#SBATCH --constraint=haswell,skylake

This means the compute node can be either haswell or skylake CPUs as listed above. This will exclude the running of sandy bridge CPU in the cluster.

LMOD

For loading the software from the stack ACCRE provides the user of Lmod. Lmod is a tool for managing software modules within a shared high-performance computing environment (e.g. the ACCRE cluster). Lmod is designed to intelligently manage, negotiate, and enforce the complex dependencies between software and libraries in a HPC environment. This should lead to better usability of administrator-installed software modules and prevent conflicts between lower-level libraries that many modules depend on.

For instructions on basic Lmod usage, please start with the excellent and concise User Guide for Lmod: [1].

Normal modules and hidden Modules

Searching for available modules


[root@gw344 ~]# ml spider R

-----------------------------------------------------------------------------------------------------------------------------------------
  R:
-----------------------------------------------------------------------------------------------------------------------------------------
    Description:
      R is a free software environment for statistical computing and graphics.

     Versions:
        R/3.3.3
        R/3.4.3
        R/3.6.0
        R/4.0.5
        R/4.2.1
     Other possible modules matches:
        APR  APR-util  Amber  Archive-Zip  Armadillo  Arrow  Aspera-CLI  BerkeleyGW  Bio-SearchIO-hmmer  BioPerl  Brotli  CNVnator  ...

-----------------------------------------------------------------------------------------------------------------------------------------
  To find other possible module matches execute:

      $ module -r spider '.*R.*'

-----------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "R" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider R/4.2.1
-----------------------------------------------------------------------------------------------------------------------------------------

Sometimes if the module is hidden, you need to add additional option to search it; for example XZ is a commonly used library and we install it as a hidden module:


[root@gw344 ~]# module spider XZ
Lmod has detected the following error:  Unable to find: "XZ".

If you don't understand the warning or error, contact the helpdesk at www.vanderbilt.edu/accre/support/helpdesk

Because it's a hidden module the normal search way will fail. To search the XZ library, you need an additional option:

[root@gw344 ~]# module --show-hidden spider XZ

-----------------------------------------------------------------------------------------------------------------------------------------
  XZ:
-----------------------------------------------------------------------------------------------------------------------------------------
    Description:
      xz: XZ utilities

     Versions:
        XZ/.5.2.2
        XZ/.5.2.3
        XZ/.5.2.4
        XZ/.5.2.5

-----------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "XZ" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider XZ/.5.2.5
-----------------------------------------------------------------------------------------------------------------------------------------

Unloading specific modules

Modules collections

FAQ