Current System Configuration
Summary
Please note, the CSF3 is in active development with new hardware being added several times a year.
Total CSF CPU cores: 19,632
- Standard Intel nodes total #cores: 3,000
- Standard AMD nodes total #cores: 10,248
- High memory Intel nodes total #cores: 736
- GPU host Intel and AMD nodes total #cores: 1552
- HPC Pool Intel nodes total #cores: 4,096
Total CSF Nvidia GPUs: 152
- 68 x Nvidia v100 GPUs (Volta, 16GB GPU RAM). More info…
- 64 x Nvidia A100 (80GB) GPUs (Ampere, 80GB GPU RAM). More info…
- 8 x Nvidia A100 (40GB) GPUs (Ampere, 80GB GPU RAM). More info…
- 12 x Nvidia L40S (40GB) GPUs (Ada Lovelace 48GB GPU RAM). More info…
Total lustre storage (scratch): 1.2PB
Login nodes: 2 physical machines, each with 32 (hyper-threaded) cores, 256GB of memory
OS on all CSF nodes: Redhat-based system
OS on all HPC Pool nodes: Redhat-based system
CSF CPU nodes
Standard nodes
504 Haswell cores: 21 nodes of 2×12-core Intel Xeon E5-2690 v3 @ 2.60GHz + 128GB RAM
1120 Broadwell cores: 40 nodes of 2×14-core Intel Xeon E5-2680 v4 @ 2.40GHz + 128GB RAM + 56Gb/s (4X FDR) mlx4 Mellanox InfiniBand
1376 Skylake cores: 43 nodes of 2×16-core Intel Xeon Gold 6130 CPU @ 2.10GHz + 192GB RAM + 100Gb/s (4X EDR) mlx5 Mellanox InfiniBand
10248 AMD Genoa cores: 61 nodes of 2×84-core AMD EPYC Genoa 9634 CPU @ 2.25GHz + 1.5TB RAM + 100GbE network.
High memory nodes
160 Haswell cores: 10 nodes of 2×8-core Intel E5-2640 v3 @ 2.60GHz + 512GB RAM
96 Skylake cores: 3 nodes of 2×16-core Intel Xeon Gold 6130 CPU @ 2.10GHz + 1.5TB RAM
128 Cascade Lake cores: 4 nodes of 2×16-core Intel Xeon Gold 5218 CPU @ 2.30GHz + 1.5TB RAM
320 Icelake cores: 10 nodes of 2×16-core Intel Xeon Gold CPU + 2TB RAM
32 Icelake cores: 1 node of 2×16-core Intel Xeon Gold CPU + 4TB RAM
GPU nodes
GPUs
68 v100 GPUs: 17 nodes of 4 x Nvidia v100-SXM2-16GB (Volta) GPUs, 16GB GPU global mem, 5120 CUDA Cores + NVLink. Intel Skylake and Cascadelake host CPUs.
64 A100 GPUs: 16 nodes of 4 x Nvidia HGX A100-SXM4-80GB (Ampere) GPUs, 80GB GPU global mem, 6912 CUDA Cores + NVLink. AMD EPYC Milan host CPUs.
8 A100 GPUs: 2 nodes of Nvidia HGX A100-SXM4-80GB (Ampere) GPUs 40GB GPU global mem, 6912 CUDA Cores + NVLink + InfiniBand. AMD EPYC Milan host CPUs.
12 L40S GPUs: 3 nodes of Nvidia L40S-SXM4-48GB (Ada Lovelace) GPUs 48GB GPU global mem, 18176 CUDA Cores. Intel Sapphire Rapids host CPUs.
GPU host nodes
416 Skylake cores: 13 nodes of 2×16-core Intel Xeon Gold 6130 CPU @ 2.10GHz + 192GB RAM + 100Gb/sec (4X EDR) mlx5 Mellanox InfiniBand + 1.6TB Samsung Non-Volatile memory (NVMe) SSD Controller 172Xa
128 Cascade Lake cores: 4 nodes of 2×16-core Intel Xeon Gold 5218 CPU @ 2.30GHz + 192GB RAM + 100Gb/sec (4X EDR) mlx5 Mellanox InfiniBand + 1.6TB Samsung Non-Volatile memory (NVMe) SSD Controller 172Xa
384 AMD EPYC Milan cores: 8 nodes of 2×24-core AMD 7413 “Milan” CPU @ 2.65GHz + 512GB RAM + 100Gb/sec (6X HDR) mlx5 Mellanox InfiniBand + 1.6TB Intel Non Volatile memory (NVMe) SSD Controller
96 AMD EPYC Milan cores: 2 nodes of 2×24-core AMD 7413 “Milan” CPU @ 2.65GHz + 512GB RAM + 100Gb/sec (6X HDR) mlx5 Mellanox InfiniBand + 1.5TB Intel Non Volatile memory (NVMe) SSD Controller as /tmp
144 Intel Sapphire Rapids cores: 2 x 24-core Intel Xeon(R) Gold 6442Y CPU @ 2.6GHz + 28TB SSD as /tmp
The HPC Pool
This is a distinct pool of resources with a different access procedure to the CSF. Please consult The HPC Pool documentation for further information.
4096 Skylake cores: 128 nodes of 2×16-core Intel Xeon Gold 6130 CPU @ 2.10GHz + 512GB RAM + 100Gb/s (4X EDR) mlx5 Mellanox InfiniBand