Click [slideshow] to begin presentation.

 

Slide Navigation






[[ Slide navigation:

Forwards:right arrow, space-bar or enter key
Reverse:left arrow

]]    




 

Introductions

The UoM CIR Ecosystem


Simon Hood

its-ri-team@manchester.ac.uk

Research Infrastructure Coordinator, IT Services

CIR:

  • Computationally-Intensive Research

RI Team:

  • Pen Richardson, George Leaver




 

Agenda

The UoM CIR Ecosystem

Resources/Facilities — much more than the CSF now:
  • Compute
    • batch,
    • interactive and
    • emerging technology;
  • research data storage — aka Isilon
    • common filesystems;
  • dedicated, fast, secure network (RDN);
  • virtual machines for researchers.
The Strategy:
  • A complete environment for Computationally-Intensive Research.




 

Brief History

A Brief History ➞ Now!

IT Services-run computational research infrastructure:

  • what there was;
  • what there is now.




 

MACE CIR History

MACE01, Redqueen and RQ2

MACE01 and RQ2
  • MACE01: 100 * two-socket, single core machines!
  • RGF: MACE01 ➞ RQ2
    • now integrated into CSF (two-day queue)
    • and Redqueen (rabbit nodes)
Redqueen (CSF prototype?)
  • Refreshed 2013 summer (using RGF — any old clusters in MACE?)
    • RI Test and Devel Cluster. . .
  • Atmos. Physics — daily weather forecasts: www.manunicast.com.




 

The Whitepaper and The CSF

The Whitepaper
  • 2010 Manchester Informatics
  • 90k   :-)
Danzek — The CSF — a success!
  • Approx 1.5 million direct academic investment since 2010 Dec.
  • Approx 5000 cores.
    • MACE are our biggest customer!

Now moving beyond only batch compute. . .




 

The UoM CIR Ecosystem

The UoM CIR Ecosystem

. . .to interactive compute, VMs, Web services. . .

. . .storage, network. . .

[ Posters Available ]




 

Computational Resources

Computational Resources

Batch
  • Redqueen, Danzek (the CSF) and Condor
  • Code Saturne, Code Aster, StarCCM+, Fluent, Abaqus. . .
Interactive — The iCSF (aka INCLINE)
  • Abaqus CAE, mesh generation, post-processing. . .
More. . .
  • Emerging technology
  • Big data and Web services




 

Danzek: The Computational Shared Facility

The UoM flagship "HPC" cluster

What?
  • Batch (non-interactive) computation.
  • Contribution-based:
    • 90k from UoM — 1.5m from academic contribs.
Specs — 5000 cores and growing:
  • Intel: 12 cores, 48 GB RAM; 16 cores 64 GB RAM; small pool of Infiniband.
  • AMD: 2GB per core RAM; all Infiniband.
  • Four nodes at 256 GB RAM; one at 512 GB RAM.




 

Condor

All those spare CPU cycles on campus. . .

Condor — what is it?
  • Grabs spare CPU cycles on unused PCs
    • on hundreds of desktop PCs around UoM (esp. public clusters)
Provides
  • Provides a high-throughput computing (HTC) environment.
    • ideal for lots of small/short jobs
FATPOU
  • Freely available to all researchers at the University
  • Run by EPS IS.




 

INCLINE

The iCSF — aka The INteractive Computational LINux Environment

Sometimes you need to do interactive computation:

  • short development runs;
  • GUI-based work.
  • INCLINE is a compute engine without queues. . .
    • . . .simply SSH-in and use. . .
    • No queues: strength and weakness.
  • Most nodes 16-core, 64 GB RAM;
    • two at 256 GB RAM.




 

Emerging Tech: Zrek

A home for all the weird stuff. . .

  • The very latest GPUs (e.g., Nvidia K40)
  • Xeon Phi cards, FPGAs. . .




 

Big Data: Hydra

Got some really big data to crunch?

All nodes:

  • 512 GB RAM
  • Infiniband for IO bandwidth to scratch




 

How do I...

I want to use this stuff, how do I. . .

Accounts
  • Email: its-ri-team@manchester.ac.uk
SSH and X-Windows
  • Linux — trivial
  • Apple OS-X: XQuartz
  • MS Windows: MobaXTerm
Documentation




 

Isilon

Storage

IT Services Research Data Storage (RDS) Service




 

What is RDS/Isilon?

What?
  • Network (attached) storage — one storage cluster in each data centre at UoM.
  • Resilient (replicated) and
  • snapshots (cf. backups).
  • UoM committed to procuring 500 TB more per year (100 TB for EPS).
Why?
  • Facilitates RDM (mandated by RCUK).
    • . . .contrast all those USB drives. . .




 

How much does Isilon cost?

  • EPS have 100 TB per year to allocate FATPOU;
  • more can be purchased. . .



RDS charges (over and above FATPOU):

Cost ModelReplicatedUnreplicated
Uni Research Use300150
Commercial Use (FEC)24001200

Charges for 1 TB per annum (all snapped)




 

Accessing the Storage

Either. . .
  1. MS Windows-style share (SMB/CIFS)
  2. Similar to P-drive
  3. Any desktop/laptop on campus; VPN off campus
. . .or. . .
  1. Unix-style share (NFS)
  2. Fast access on CSF, iCSF, Redqueen, etc.
  3. Desktop/laptop access via SFTP, SSHFS, etc (on and off campus).




 

Snapshots

Recover your own files from snapshots

MS Windows
Right click. . .
Linux and OS-X
  • Top of share/export: .snapshot
  • Hidden — not even ls -a (enter path manually)




 

Snapshots Example

Example:

simonh@doolittle:/mnt/pdrive$ ls -a
.         Desktop         file3.yes          My Videos       
..        desktop.ini     latex_cookbook.ps  public_html     
.emacs.d  untitled-1.aux  untitled-1.log     untitled-1.tex

simonh@doolittle:/mnt/pdrive$ cd .snapshot
simonh@doolittle:/mnt/pdrive/.snapshot$ ls
.    .
.    .
kilburn_24hr_2014-01-20_13-30-00  kilburn_24hr_2014-01-20_06-30-00   
kilburn_24hr_2014-01-20_14-30-00  kilburn_24hr_2014-01-20_07-30-00   
.    .
.    .




 

How do I get an Isilon Share?

I want to use this storage, how do I. . .

Either email
  • EPS-ITResearch@manchester.ac.uk
or visit




 

RVMS

RVM Service

  • Research Virtual Machine Service
  • Resilient, supported infrastructure for researchers
    • esp. Web servers
  • Free at the point of use
  • In pilot
  • Eliminate "under-desk servers"?
  • http://ri.itservices.manchester.ac.uk/rvms




 

RVM Service Use Cases

What is the RVM Service suitable for?

  • Public facing Web sites with moderate traffic;
  • shared development environments;
  • Web front ends to DBs, HPC. . .
. . .NOT. . .?
  • Compute-intensive work;
  • memory or IO-intensive work.




 

RVMS: How do I...

I want a RM, how do I. . .

Email
its-ri-team@manchester.ac.uk.
N.B. Pilot
  • Service is in pilot phase. . .




 

The Ecosystem: A Complete Solution?

The Goal

  • Reduce desktop requirements to thin clients (plus email, office, surfing. . .).
  • All CIR requirements addressed and integrated:
    • all infrastructure run by IT staff (not postgrads/docs);
    • hardware shared — fewer wasted CPU cycles, access to "bigger" hardware than otherwise. . . (cf. CSF model).
  • Data lives within Ecosystem — available everywhere.



  • What's missing? (Viz.?)




 

Workflow

Example Ecosystem Workflow



 

ManUniCast

An Existing Ecosystem Use Case

  • Weather and pollution forecasts at the University of Manchester!
  • www.manunicast.com
  • Daily:
    • download input datasets from NOAA (USA);
    • computational runs on Redqueen;
    • post-processing on Redqueen, results on RVMS Web server.
  • Poster available.




 

Gateways to the Ecosystem

How to I get access from off-campus?

  • VPN
  • SSH gateway
  • Research Virtual Desktop ****




 

More

More. . .

These slides:
RI Team Web site:
Ecosystem poster




 

Finally

Questions?
its-ri-team@manchester.ac.uk