Using Software Modules in an HPC Cluster

Introduction

In HPC clusters, it is common to use an Environment Modules system to provide access to compilers, libraries, and scientific applications without installing them directly in the user's personal environment.

A module modifies the shell environment variables in a controlled manner. Depending on the software, it may update, for example:

• PATH
• LD_LIBRARY_PATH
• CPATH
• LIBRARY_PATH
• MANPATH
• software-specific variables

This makes it possible to have multiple versions of the same software installed simultaneously and to explicitly choose which version to use.

The module Command

Module management is mainly performed through the command:

module

To display the help:

module help

or:

module --help

The available commands may vary slightly depending on the implementation installed on the cluster, for example Lmod or Environment Modules.

Displaying Available Modules

To display the available software modules:

module avail

The output may contain, for example:

gcc/11.3.0
openmpi/4.1.5
python/3.11
cuda/12.0
matlab/R2023b

The versions actually available depend on the software installed on the cluster.

Loading a Module: module load

To make a software package available, use:

module load module_name

Example:

module load gcc/11.3.0

To load multiple modules:

module load gcc/11.3.0
module load openmpi/4.1.5

or, if supported by the configuration:

module load gcc/11.3.0 openmpi/4.1.5

After loading, the module modifies the required environment variables so that commands, libraries, and other software components are accessible.

Checking Loaded Modules: module list

To check which modules are active in the current session:

module list

This is particularly useful before compiling a program or running a Slurm job because it allows you to verify the software environment currently in use.

Removing a Module: module unload

To remove a single module from the environment:

module unload module_name

Example:

module unload gcc/11.3.0

The system removes the environment changes associated with the module.

Completely Cleaning the Environment: module purge

To remove all currently loaded modules:

module purge

A useful practice in HPC scripts is to start from a clean environment:

module purge
module load gcc/11.3.0
module load openmpi/4.1.5

This way, the script explicitly declares the required software modules and reduces its dependency on the user's interactive environment.

Searching for a Module

On many installations, module avail can be filtered, for example:

module avail gcc

or:

module avail python

With Lmod, the following command is also normally available:

module spider module_name

Example:

module spider cuda

module spider is particularly useful with hierarchical module systems because it can also display software that becomes available only after other modules have been loaded.

Displaying What a Module Changes: module show

Before loading a module, you can inspect its configuration:

module show module_name

Example:

module show gcc/11.3.0

Depending on the system, the following command may also be available:

module display module_name

These commands allow you to see the changes that the modulefile applies to the environment.

module use and MODULEPATH

Modulefiles are searched for in the paths defined by the MODULEPATH environment variable.

To display it:

echo $MODULEPATH

The command:

module use /path/to/modules

adds another directory to the path where the system searches for modulefiles.

Example:

module use $HOME/modules
module avail

This can be useful for using personal modulefiles or module collections stored in additional directories.

To remove a path previously added using module use, if supported by the implementation:

module unuse /path/to/modules

Modules and Environment Variables

A modulefile is not simply an alias used to launch an application. Its purpose is to modify the shell environment.

For example, before loading software:

which gcc

After loading it:

module load gcc/11.3.0
which gcc
gcc --version

To check the environment variables:

echo $PATH
echo $LD_LIBRARY_PATH
echo $MODULEPATH

module show allows you to understand which variables are modified by a specific module.

Changing a Software Version

If multiple versions are available:

gcc/10.4.0
gcc/11.3.0
gcc/12.2.0

you can unload the active version and load another one:

module unload gcc/11.3.0
module load gcc/12.2.0

Some systems also support commands such as module switch or module swap:

module swap gcc/11.3.0 gcc/12.2.0

The available syntax can be checked with:

module help

Modules and Dependencies

In an HPC cluster, modules may need to be loaded in an order consistent with software dependencies.

A typical example is:

module purge
module load gcc/11.3.0
module load openmpi/4.1.5

An MPI library may have been compiled using a specific compiler. Using incompatible combinations can cause compilation, linking, or runtime errors.

It is therefore good practice to check:

module list

before compiling or running an application.

Using Modules in a Slurm Script

Modules should be loaded directly in the batch script, making the software environment required by the job explicit.

Example:

#!/bin/bash
#SBATCH --job-name=test
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --time=00:30:00

module purge
module load gcc/11.3.0
module load openmpi/4.1.5

module list

srun ./program

The important principle is that the script should specify the environment required by the program in a reproducible way.

Python Example

#!/bin/bash
#SBATCH --job-name=python-test
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --time=00:30:00

module purge
module load python/3.11

python --version
python program.py

The versions shown in these examples are illustrative and should be replaced with those actually available on the cluster, which can be checked using module avail.

Recommended Procedure

A practical sequence for working with modules is:

1. Clean the Environment

module purge

2. Search for the Software

module avail

or:

module avail software_name

3. Load the Desired Version

module load software_name/version

4. Check the Active Modules

module list

5. Check the Executable

which software_name

6. Check the Version

software_name --version

7. Run the Program

Run the program interactively or through a Slurm script, according to the cluster rules.

Main Commands

Best Practices in an HPC Environment

1. Use module purge at the beginning of batch scripts when a controlled environment is required.
2. Specify the module version when reproducibility is important.
3. Use module list to check the environment before running an application.
4. Include module load commands directly in Slurm scripts instead of relying on the configuration of the login session.
5. Use module show to check how a module modifies the environment.
6. Avoid arbitrarily combining compilers, MPI implementations, and libraries built with different toolchains.
7. Use module use only when additional modulefile repositories need to be added.
8. In case of errors, start again with module purge, load only the required modules, and check the environment again using module list.

Complete Example

# Remove any previously loaded modules
module purge

# Display the available versions
module avail gcc
module avail openmpi

# Load the compiler and MPI
module load gcc/11.3.0
module load openmpi/4.1.5

# Check the environment
module list
which gcc
which mpirun

# Check the versions
gcc --version
mpirun --version

# Run the application through Slurm
srun ./mpi_program

This approach makes the software environment used by the HPC job explicit and easier to reproduce.