The open archive for STFC research publications

Full Record Details

Persistent URL http://purl.org/net/epubs/work/40650679
Record Status Checked
Record Id 40650679
Title JASMIN – On the road to high performance software defined object stores
Abstract The JASMIN “Super Data Cluster”– the UK’s Environmental Data Analysis Platform – Has recently completed its £8M Phase IV upgrade, taking its storage capacity to ~45PetaBytes of high performance storage as well as doubling its cloud and batch computing capabilities. In the heart of this talk we will discuss the innovative new storage and networking technologies that, are being deployed for the first time in an HPC context and how these will be used to improve our user experience and flexibility. The storage technologies deployed for the first time in JASMIN – QuoByte, Caringo and PURE – mark the start of our road map towards high performance Object Storage over the next few years, pushing the boundaries of where Object storage will be used. In the talk I will discuss rationale for and the results of the comparative proof of concept benchmarking exercises of several software defined storage technologies, in our search for a route from high performance parallel file systems, towards high performance object storage of ideally similar performance. I will then discuss the JASMIN 4 all Ethernet network deployment and issues, of an unusual 5 tier routed CLOS “data centre” network with 17Terabits/sec edge bandwidth. I will wrap up with some of the early benchmarking results of the file and object storage, showing in excess of 220Gbytes/sec file and approaching this for S3 object performance. At least one of these new technologies – the 31.5PBytes of QuoByte software defined storage is the largest capacity deployment in the world. This is also the first Caringo deployment as an HPC, as opposed to capacity, object store tier.
Keywords JASMIN , networking , high performance computing , HPC , storage
Funding Information NERC
Related Research Object(s):
Licence Information:
Language English (EN)
Type Details URI(s) Local file(s) Year
Presentation Presented at Computing Insight UK 2018 (CIUK 2018), Manchester, England, 12-13 Nov 2018. Jonathan-Churchill-CIUK2018.pdf 2018