What we offer - HPC

Page / - last modified by tonee on Mon, 06/30/2025, 2:15 PM

HPC

last modified by tonee

on Mon, 06/30/2025, 2:15 PM

Table of Contents

Default Allocation for HPC service

The COARE's HPC consists of a cluster of compute and storage servers to allow high-speed and resource-intensive computations and processing of large datasets.

The system architecture for COARE HPC service is detailed below:

The current capacity of COARE HPC is summarized below:

CPU	30 Tflops
GPU	75 Tflops

The HPC service uses SLURM as its batch scheduler. The cluster is divided into 4 partitions: Debug, Batch, Serial, and GPU. Below are the specifications per partition:

Debug (2 Nodes)

44 cores, 88 threads

528GB RAM

Batch (32 Nodes)

44 cores, 88 threads

528GB RAM

Serial (2 Nodes)

44 cores, 88 threads

528GB RAM

GPU (6 Nodes)

12 cores, 24 threads

1056GB (1TB) RAM

NVIDIA Tesla P40

The home directory (/home) is COARE's network filesystem using GlusterFS and is built to serve as the user’s home directory. Users’ scripts input data are stored here.

The scratch directories (/scratch1, /scratch2, and /scratch3) are COARE's parallel filesystem using LustreFS. These are built to handle user’s I/O heavy workloads. The output of running jobs including the intermediary files are stored here.

NOTE: As part of the efforts to upgrade COARE's current infrastructure, the COARE Team has been implementing the saliksik cluster since Q2 of 2020, which comprises the next generation of HPC-based CPUs and GPUs of the COARE. For more information on saliksik, visit this Wiki page.

Default Allocation for HPC service

COARE provides a default allocation for each user to ensure the fair and equitable use of the COARE Services. For more information, please read the COARE's Acceptable Use Policy (AUP).

The implementation of the saliksik cluster also required the adjustment of the default allocation provided for each COARE HPC user, which is summarized in the table below:

CPU	86 logical cores
Network filesystem (/home)	100 GB usable
Parallel filesystem (Scratch directories: /scratch1, /scratch2, and /scratch3)	Total of 3TB across scratch directories
GPU	1 GPU
Max running job	30 jobs
Max submit job	45 jobs
Job waiting time	No guarantee; depends on the status of the queue and the availability of the requested resource/s
Job walltime limit	Debug	Maximum of 1 day allowable runtime
Batch	Maximum of 7 days allowable runtime
Serial	Maximum of 14 days allowable runtime
GPU	Maximum of 3 days allowable runtime

Any requests for allocation increase will be subject to the COARE Team's evaluation and approval.

Thank you for rating this Wiki page.

You may choose to provide a more detailed feedback for this page using the form below or by contacting us here. Otherwise, click Close.

Name

Comment

Close

Submit