Technical specifications#

Note

Sherlock is driven by contributions from individual PIs and groups, and as such, is constantly evolving. The technical specifications outlined here are subject to change, and may not be an accurate representation of the current cluster configuration. The numbers provided on this page are as of June 2017.

In a nutshell#

Sherlock features more than 1,000 compute nodes, 18,000+ CPU cores, 120TB of total memory, 400+ GPUs, for a total computing power of more than 1 Petaflop. That would rank it in the Top500 list of the most powerful supercomputers in the world.

A parallel high-performance filesystem of more than 3 PB, delivering over 20GB/s of sustained I/O bandwidth, provides scratch storage for more than 2,300 users, and 400 PI groups.

Computing#

The Sherlock cluster has been initiated in January 2014 with a base of freely available computing resources and the accompanying networking and storage infrastructure. It has since been expanded with additions from multiple PI groups to reach the capacity of its Infinband network in December 2016.

Sherlock 2.0

A new Infiniband fabric has been installed in early 2017, as the foundation for Sherlock 2.0. The existing nodes will join that new cluster in the second half of 2017, at which point both clusters will be merged.

Sherlock 1.0#

Type Qty Details
login nodes 4 sherlock.stanford.edu (load-balanced)
data transfer node 1 dedicated bandwidth for large data transfers
compute nodes 120 16 cores1, 64 GB RAM, 100 GB local SSD
bigmem nodes 2 32 cores2, 1.5 TB RAM, 13TB of local storage
GPU nodes 5 16 cores1, 64 GB RAM, 200 GB local SSD, 8 GPUs
NVIDIA Tesla K20Xm, K80, or GeForce GTX TITAN Black
owners nodes 716 various CPU/memory configs, bigmem and GPU nodes
interconnect 2:1 oversubscribed FDR Infiniband fabric (56 GB/s)
operating system CentOS 6.x

Sherlock 2.0#

Type Qty Details
login nodes 2 login.sherlock.stanford.edu (load-balanced)
compute nodes 60 20 cores3, 128 GB RAM, 200 GB local SSD
bigmem nodes 2 56 cores4, 3.0 TB RAM, 200 GB local SSD
32 cores5, 512 GB RAM, 200GB local SSD
GPU nodes 2 20 cores3, 128 GB RAM, 200 GB local SSD, 4 GPUs
NVIDIA Tesla P100-PCIE-16GB, Tesla P40
owners nodes 160 various CPU/memory configs, bigmem and GPU nodes
interconnect 2:1 oversubscribed EDR Infiniband fabric (100 GB/s)
operating system CentOS 7.x

Storage#

More information

For more information about storage options on Sherlock, please refer to the Storage section of the documentation.

Storage components are common to both clusters, meaning users can find the same files and directories from both Sherlock 1.0 and Sherlock 2.0 nodes.

  • Highly-available NFS filesystem for user and group home directories (with hourly snapshots and off-site replication)
  • High-performance Lustre scratch filesystem (3 PB, 20+GB/s sustained I/O bandwidth, 18 I/O servers, 1,080 disks)
  • Direct access to SRCC's Oak long-term research data storage system

  1. two-socket Intel® Xeon® E5-2650v2 processors (8-core Ivy-Bridge, 2.60 GHz) 

  2. four-socket Intel® Xeon® E5-4640 processors (8-core Sandy-Bridge, 2.40 GHz) 

  3. two-socket Intel® Xeon® E5-2640v4 processors (10-core Broadwell, 2.40 GHz) 

  4. four-socket Intel® Xeon® E5-4650v4 processors (14-core Broadwell, 2.20 GHz) 

  5. two-socket Intel® Xeon® E5-2697Av4 processors (16-core Broadwell, 2.60 GHz)