Slurm partitions on the ARIES cluster

Overview

The ARIES cluster uses Slurm to manage and schedule computing resources.

The cluster resources are organized into several Slurm partitions, each characterized by specific nodes, time limits, priorities, QoS configurations, and access restrictions.

The current status of the partitions can be displayed with:

sinfo

The currently visible partitions are:

• debug
• ice4hpc
• ulow
• high
• low
• user-debug
• unsafe_part
• grumo_pnrr
• faf_pnrr
• ecosister

The partition marked with * in the sinfo output is the default partition.

In the provided configuration:

ulow*

therefore indicates that ulow is the default partition.

Partition summary

Values that are not explicitly present in the provided configuration are indicated with - or described as unspecified.

Useful information

ice4hpc

Limits

Default time:

04:00:00

Maximum time:

3-00:00:00

which corresponds to 3 days.

GPUs

This partition includes gnode01 and uses the following QoS:

gpus

ulow

ulow is the default ARIES partition.

Maximum time

14-00:00:00

which corresponds to 14 days.

Access

The configuration contains:

AllowAccounts=ALL

high

Maximum time

1-12:00:00

which corresponds to 1 day and 12 hours.

QoS

The following QoS is associated with the partition:

QOS=high

low

The low partition uses the same groups of nodes as high and ulow.

Maximum time

3-00:00:00

which corresponds to 3 days.

QoS

The configuration specifies:

QOS=low
AllowQos=low,normal,high

The QoS entries specified in AllowQos are therefore:

• low
• normal
• high

user-debug

The user-debug partition is configured with a very short maximum execution time:

MaxTime=00:30:00

Maximum time

00:30:00

which corresponds to 30 minutes.

Dedicated partitions

unsafe_part

The unsafe_part partition uses:

cnode[45-48]

The allowed QoS is:

unsafe_qos

grumo_pnrr

The grumo_pnrr partition is reserved for users belonging to the GRUMO group and uses:

cnode[25-44]

Authorized accounts

lab
grumo_pnrr

QoS

AllowQos=nolimits

Maximum time

15-00:00:00

which corresponds to 15 days.

faf_pnrr

The faf_pnrr partition is reserved for users belonging to the FAF group and uses:

cnode[45-48]

Authorized accounts

faf_pnrr

QoS

AllowQos=nolimits

Maximum time

UNLIMITED

ecosister

The ecosister partition is reserved for the Ecosister group and uses:

fnode[09-12]

Authorized accounts

lab
ecosister

QoS

AllowQos=nolimits

Maximum time

10-00:00:00

which corresponds to 10 days.

Meaning of the main parameters

PartitionName

The name used by Slurm to identify the partition.

Example:

PartitionName=high

Default

Specifies whether the partition is the default partition.

Default=YES

In the provided ARIES configuration, this value is assigned to:

ulow

If a job is submitted without specifying a partition, sinfo identifies ulow with *:

ulow*

Nodes

Defines the nodes belonging to the partition.

For example:

Nodes=cnode[01-24],fnode[01-08]

includes the following node ranges:

cnode01 ... cnode24
fnode01 ... fnode08

MaxTime

Defines the maximum execution time configured for a job in the partition.

Example:

MaxTime=3-0

corresponds to:

3 days

While:

MaxTime=1-12

corresponds to:

1 day and 12 hours

DefaultTime

Defines the execution time assigned when the partition configuration provides a default value.

For example, ice4hpc contains:

DefaultTime=4:00:00

AllowAccounts

Specifies the Slurm accounts allowed to use the partition.

For example:

AllowAccounts=lab,grumo_pnrr

is present in the grumo_pnrr configuration.

The following value:

AllowAccounts=ALL

is instead present in general-purpose partitions such as ulow, high, low, user-debug, and unsafe_part.

AllowQos

Lists the QoS entries allowed by the partition configuration.

For example:

AllowQos=low,normal,high

is configured for the low partition.

The dedicated partitions grumo_pnrr, faf_pnrr, and ecosister specify:

AllowQos=nolimits

QOS

Defines the QoS associated with the partition in the provided configuration.

For example:

PartitionName=high
QOS=high

and:

PartitionName=low
QOS=low

For ice4hpc:

QOS=gpus
AllowQos=gpus

PriorityTier

Specifies the priority tier configured for the partition.

The provided data includes, among others:

high       PriorityTier=2000
user-debug PriorityTier=1000
ice4hpc    PriorityTier=100
ulow       PriorityTier=20
low        PriorityTier=10

PriorityJobFactor

This is the PriorityJobFactor value configured for the partition.

In the provided data:

user-debug  = 2000
unsafe_part = 2000
high        = 1000
ice4hpc     = 800
low         = 100
ulow        = 0

Current partition status

The status of the nodes belonging to a partition can be checked with:

sinfo

The following states appear in the provided output:

For example, for the default ulow partition, the provided output shows nodes in the following states:

mix
alloc

while for unsafe_part it shows:

idle

Using a partition

A partition can be selected in a Slurm job script using:

#SBATCH --partition=<partition_name>

For example:

#!/bin/bash -l
#SBATCH --job-name=test
#SBATCH --partition=low
#SBATCH --time=01:00:00

./program

To use a specific partition and QoS together:

#SBATCH --partition=low
#SBATCH --qos=normal

The selected combination must comply with the QOS, AllowQos, AllowAccounts, and other limits configured for the partition and for the user's Slurm association.

Useful commands

Display the partitions

sinfo

Display detailed partition information

scontrol show partition

Display a specific partition

scontrol show partition high

Specify a partition with sbatch

sbatch --partition=low job.sh

Alternatively, specify it inside the job script:

#SBATCH --partition=low