What we offer - HPC 

Page / - last modified by tonee on Mon, 06/30/2025, 2:15 PM 

HPC 
last modified by tonee
on Mon, 06/30/2025, 2:15 PM

Table of Contents
Default Allocation for HPC service

 
The COARE's HPC consists of a cluster of compute and storage servers to allow high-speed and resource-intensive computations and processing of large datasets.
The system architecture for COARE HPC service is detailed below:
The current capacity of COARE HPC is summarized below:
 
CPU	30 Tflops	
GPU	75 Tflops	
The HPC service uses SLURM as its batch scheduler. The cluster is divided into 4 partitions: Debug, Batch, Serial, and GPU. Below are the specifications per partition:
Debug (2 Nodes)
•	44 cores, 88 threads
•	528GB RAM 
 
Batch (32 Nodes)
•	44 cores, 88 threads
•	528GB RAM
 
Serial (2 Nodes) 
•	44 cores, 88 threads
•	528GB RAM
               
GPU (6 Nodes) 
•	12 cores, 24 threads
•	1056GB (1TB) RAM
•	NVIDIA Tesla P40
 
The home directory (/home) is COARE's network filesystem using GlusterFS and is built to serve as the user’s home directory. Users’ scripts input data are stored here.
The scratch directories (/scratch1, /scratch2, and /scratch3) are COARE's parallel filesystem using LustreFS. These are built to handle user’s I/O heavy workloads. The output of running jobs including the intermediary files are stored here.
NOTE:  As part of the efforts to upgrade COARE's current infrastructure, the COARE Team has been implementing the saliksik cluster since Q2 of 2020, which comprises the next generation of HPC-based CPUs and GPUs of the COARE. For more information on saliksik, visit this Wiki page.
Default Allocation for HPC service
COARE provides a default allocation for each user to ensure the fair and equitable use of the COARE Services. For more information, please read the COARE's Acceptable Use Policy (AUP).
The implementation of the saliksik cluster also required the adjustment of the default allocation provided for each COARE HPC user, which is summarized in the table below:
CPU	86 logical cores	
Network filesystem (/home)	100 GB usable	
Parallel filesystem
(Scratch directories:
/scratch1, /scratch2, and /scratch3)	
Total of 3TB across scratch directories	
GPU	1 GPU	
Max running job	30 jobs	
Max submit job	45 jobs	
Job waiting time	No guarantee;  depends on the status of the queue and the availability of the requested resource/s	
Job walltime limit	Debug	Maximum of 1 day allowable runtime	
	Batch 	Maximum of 7 days allowable runtime	
	Serial	Maximum of 14 days allowable runtime	
	GPU	Maximum of 3 days allowable runtime	
Any requests for allocation increase will be subject to the COARE Team's evaluation and approval.

Thank you for rating this Wiki page.
You may choose to provide a more detailed feedback for this page using the form below or by contacting us here. Otherwise, click Close. 
Name 
 
Comment 

         

           

               Close
               Submit