MACE Workshop 2010



MACE Research Computing Workshop 2010

Simon Hood

Research Computing Services



  • RCS?
  • HPC clusters?
RCS HPC Cluster News
  • Horace, Man2e (NWGrid), Vidar (NGS). . .
Developments on Mace01 and RedQueen
  • More filespace
  • More compute nodes
MACE Common Filesystem
  • and SSHFS
Virtual Desktops
  • Virtual desktops (VNC) on login nodes for editing, etc
  • Use with interactive queues for running e.g. Fluent interactive jobs


Who are RCS?

RCS? Who? What?

And why should I care?

  • Research Computing Services
    • Specialist part of IT Services.

The short version:

  • Look after Horace, Man2e, Redqueen, Mace01. . .
    • plus application support.



More about RCS

What is Research Computing?
  • Computing to support research! Examples:
    • physics-based computational models, data-mining, Monte-Carlo methods. . .
    • use of HPC/HTC (Linux) clusters
Contact Details


HPC Clusters

HPC Clusters

  • The nature of an HPC cluster
  • Why you should be using them


Why use an HPC cluster?


The Nature of an HPC Cluster


UoM HPC Cluster News

UoM HPC Cluster News

RCS-owned (i.e., not MACE-owned and excluding RedQueen) clusters:

  • Man2(e)
  • Horace
  • Vidar



  • Hardware maintenance of head/compute nodes renewed for last time
    • not disk array
  • There will be no further investment in this system.
  • If you need more than a year's compute resource, find an alternative cluster!



  • Leased; service ends this July (2010)
  • No centrally-funded replacement
  • If you use Horace, find an alternative compute resource now!



  • NGS cluster (256 cores)
  • Now 75% owned by UoM:
    • provide stop-gap service after Horace
    • already 2.5 years old. . .


Developments on Mace01 and RedQueen

Developments on Mace01 and RedQueen


Developments on Mace01

New File Servers
  • Two new fileservers (thanks to Alistair Revell, Peter Stansby)
  • New login node coming. . .
Filesystems and Backups
  • home-dirs occasionally snapshotted to RedQueen (weekly?)
    • backed up properly one day. . .
  • plenty of scratch space
Mace01a and VNC
  • Virtual desktop service
    • more + demo later. . .
Revolving Green Fund
  • Replace old kit with new, more efficient kit, using RGF loan?
  • Decision currently with IT Services SMT. . .


Developments on RedQueen

RSN: New Hardware
  • New disk arrays, total 24TB
  • 36 new compute nodes (now 36 + 11 nodes, total 520 cores)
  • Fast interconnect — Infiniband — for the 36
Filesystems and Backups
  • /home1 and /home2 are backed up (total 270GB)
  • /snap1, /snap2, ...
  • TBs of scratch space coming.
  • 24-core nodes (AMD Magny-Cours) — who has some money?


MACE Common Filesystem

MACE Common Filesystem

Wouldn't it be nice if my home-dir was the same on RedQueen, Mace01, Man2e. . .


MACE Common Filesystem Details

  • /mace-common-1 is a filesystem common to RedQueen, Mace01 and Man2e
    • only login nodes currently
  • Distributed, replicated/mirrored (not otherwise backed up — yet)



Getting a MACE Common Dir

  • Ask
  • Need UID alignment across clusters



Mount my home-dir and mace-common-1 on my local desktop?

  • Mounts any filesystem accessible via SSH
  • User-space filesystem based on FUSE
    • unpriviliged users can mount
  • Read/write access same as after SSH login
Debian (and Ubuntu?):
  apt-get install sshfs
Fedora 12:
  yum install fuse-sshfs



  desktop> mkdir ~/redqueen ~/mace-common-1
  desktop> sshfs ~/redqueen
  desktop> sshfs ~/mace-common-1
Access Files
  desktop> ls  ~/redqueen

  benchmarking    test_openmpi_gcc44_gfortran44
  bin     test_openmpi_gcc_gfortran
  CLUSTER    test_openmpi_gcc_gfortran_mx
  .               .            .
  .               .            .
  fusermount -u ~/redqueen
  fusermount -u ~/mace-common-1


Virtual Desktops

Virtual Desktops

  • What?
  • A virtual office desktop
  • Using with interactive jobs



  • mace01



VNC on Mace01a


Using VNC with SGE Interactive Queues