Sherlock is driven by contributions from individual PIs and groups, and as such, is constantly evolving. The technical specifications outlined here are subject to change, and may not be an accurate representation of the current cluster configuration. The numbers provided on this page are as of June 2017.
In a nutshell#
Sherlock features more than 1,000 compute nodes, 18,000+ CPU cores, 120TB of total memory, 400+ GPUs, for a total computing power of more than 1 Petaflop. That would rank it in the Top500 list of the most powerful supercomputers in the world.
A parallel high-performance filesystem of more than 3 PB, delivering over 20GB/s of sustained I/O bandwidth, provides scratch storage for more than 2,300 users, and 400 PI groups.
The Sherlock cluster has been initiated in January 2014 with a base of freely available computing resources and the accompanying networking and storage infrastructure. It has since been expanded with additions from multiple PI groups to reach the capacity of its Infinband network in December 2016.
A new Infiniband fabric has been installed in early 2017, as the foundation for Sherlock 2.0. The existing nodes will join that new cluster in the second half of 2017, at which point both clusters will be merged.
|data transfer node||1||dedicated bandwidth for large data transfers|
|compute nodes||120||16 cores1, 64 GB RAM, 100 GB local SSD|
|bigmem nodes||2||32 cores2, 1.5 TB RAM, 13TB of local storage|
|GPU nodes||5||16 cores1, 64 GB RAM, 200 GB local SSD, 8 GPUs
NVIDIA Tesla K20Xm, K80, or GeForce GTX TITAN Black
|owners nodes||716||various CPU/memory configs, bigmem and GPU nodes|
|interconnect||2:1 oversubscribed FDR Infiniband fabric (56 GB/s)|
|operating system||CentOS 6.x|
|compute nodes||60||20 cores3, 128 GB RAM, 200 GB local SSD|
|bigmem nodes||2||56 cores4, 3.0 TB RAM, 200 GB local SSD
32 cores5, 512 GB RAM, 200GB local SSD
|GPU nodes||2||20 cores3, 128 GB RAM, 200 GB local SSD, 4 GPUs
NVIDIA Tesla P100-PCIE-16GB, Tesla P40
|owners nodes||160||various CPU/memory configs, bigmem and GPU nodes|
|interconnect||2:1 oversubscribed EDR Infiniband fabric (100 GB/s)|
|operating system||CentOS 7.x|
For more information about storage options on Sherlock, please refer to the Storage section of the documentation.
Storage components are common to both clusters, meaning users can find the same files and directories from both Sherlock 1.0 and Sherlock 2.0 nodes.
- Highly-available NFS filesystem for user and group home directories (with hourly snapshots and off-site replication)
- High-performance Lustre scratch filesystem (3 PB, 20+GB/s sustained I/O bandwidth, 18 I/O servers, 1,080 disks)
- Direct access to SRCC's Oak long-term research data storage system