Click [slideshow] to begin presentation.

 

Slide Navigation






[[ Slide navigation:

Forwards:right arrow, space-bar or enter key
Reverse:left arrow

]]    




 

Introductions

The UoM CIR Ecosystem


Simon Hood

its-ri-team@manchester.ac.uk

Research Infrastructure Coordinator, IT Services

CIR:

  • Computationally-Intensive Research

RI Team:

  • Pen Richardson, George Leaver




 

Agenda

The UoM CIR Ecosystem

  • Compute resources,
    • batch,
    • interactive and
    • emerging technology;
  • research data storage — aka Isilon — (TMR4BH);
  • virtual machines for researchers.

The Goal:

  • A complete environment for CIR.




 

Ecosystem Diagram

Breaking news: Hydra!



 

Computational Resources

Computational Resources

Batch
  • Condor
  • CSF
  • Kadmon
  • Hydra
Interactive
  • iCSF




 

Isilon

Isilon Storage

IT Services Research Data Storage (RDS) Service




 

What is Isilon?

What?
  • Network (attached) storage — cf. MS Windows shared areas/drives
  • Resilient (replicated) and snapshots (cf. backups).
  • Both:
    • MS Windows-style shares (CIFS);
    • Unix/Linux-style exports (NFS).
  • UoM committed to procuring 500 TB more per year (100 TB for FLS).
How/where is it accessible?
  • Desktops, laptops, laboratories. . .
  • Computational clusters — common folders/dirs: CSF, Kadmon, Hydra. . .




 

How much does Isilon cost?

  • FLS have 100 TB per year to allocate FATPOU;
  • more can be purchased. . .




RDS charges (over and above FATPOU):

Cost ModelReplicatedUnreplicated
Uni Research Use300150
Commercial Use (FEC)24001200

Charges for 1 TB per annum (all snapped)




 

How do I get an Isilon Share?

Either email

  • FLS-ITResearch@manchester.ac.uk
or visit




 

Mounting CIFS shares on MS Windows

  1. Locate the Map Network Drive option:
    • XP: Right-click on My Computer and select Map network drive.
    • Win 7: Start --> Computer --> Click "Map network drive"
  2. In the Drive list select the required drive letter (e.g., R:).
  3. In the Folder box enter the path according to your faculty and mount point. For example
        \\nasr.man.ac.uk\epsrss$\snapped\replicated\myshare
        
  4. Username: one of
        ds\mxyzpqr2
        mxyzpqr2@ds.man.ac.uk
        
  5. Password: central/ITServices.




 

Mounting CIFS shares on Linux

If it is a one-off:
  mount -t cifs -o \
      user,noauto,username=mxyzpqr7,domain=ds.man.ac.uk \
      //nasr.man.ac.uk/epsrss$/snapped/replicated/myshare \
      /mnt/myshare/
If it is a many-off, use /etc/fstab:
  //nasr.man.ac.uk/epsrss$/snapped/replicated/myshare \
      /mnt/myshare/ \
      cifs \
      user,noauto,username=mxyzpqr7,domain=ds.man.ac.uk \
      0 0
then simply
  mount /mnt/myshare




 

Mounting CIFS shares on OS-X

Maverick (10.9), also 10.6
  • Use finder:
    smb://nasr.man.ac.uk/flsrss$/snapped/replicated/myshare
10.7 and 10.8 (Lion and Mountain Lion)




 

Snapshots

Recover your own files from snapshots

  • Top of share/export: .snapshot
  • Hidden — not even ls -a (enter path manually)

Example:

simonh@doolittle:/mnt/pdrive$ ls -a
.         Desktop         file3.yes          My Videos       
..        desktop.ini     latex_cookbook.ps  public_html     
.emacs.d  untitled-1.aux  untitled-1.log     untitled-1.tex

simonh@doolittle:/mnt/pdrive$ cd .snapshot
simonh@doolittle:/mnt/pdrive/.snapshot$ ls
.    .
.    .
kilburn_24hr_2014-01-20_13-30-00  kilburn_24hr_2014-01-20_06-30-00   
kilburn_24hr_2014-01-20_14-30-00  kilburn_24hr_2014-01-20_07-30-00   
.    .
.    .




 

Desktop Backups

This is very simple — no sophisticated script required

  1. Get an Isilon CIFS share;
  2. configure your desktop to ensure it is always mapped/mounted;
  3. set up a cron job to perform an rsync each day;
  4. snapshots automatically provide file retrieval for 28 days.


But better to simply use your Isilon share for your files???




 

INCLINE

INCLINE

The INteractive Computational LINux Environment

Sometimes you need to do interactive computation:

  • short development runs;
  • GUI-based work.
  • INCLINE is a compute engine without queues. . .
    • . . .simply SSH-in and use. . .




 

RVMS

RVM Service

  • Research Virtual Machine Service
  • Resilient, supported infrastructure for researchers
    • esp. Web servers
  • Free at the point of use
  • In pilot
  • Eliminate "under-desk servers"?
  • http://ri.itservices.manchester.ac.uk/rvms




 

RVM Service Use Cases

What is the RVM Service suitable for?

  • Public facing Web sites with moderate traffic;
  • shared development environments;
  • Web front ends to DBs, HPC. . .
. . .NOT. . .?
  • Compute-intensive work;
  • memory or IO-intensive work.




 

Getting a VM

How do I get an RVMS VM?

  • Ask the FLS CIR people (Nick, Simon, Ziaul), or
  • email
            its-ri-team@manchester.ac.uk

N.B.

  • Service is in pilot phase. . .




 

The Ecosystem: A Complete Solution?

The Goal

  • Reduce desktop requirements to thin clients (plus email, office, surfing. . .).
  • All CIR requirements addressed and integrated:
    • all infrastructure run by IT staff (not postgrads/docs);
    • hardware shared — fewer wasted CPU cycles, access to "bigger" hardware than otherwise. . . (cf. CSF model).




 

Workflow




 

ManUniCast

An Existing Ecosystem Usecase

  • Weather and pollution forecasts at the University of Manchester!
  • www.manunicast.com
  • Daily:
    • download input datasets from NOAA (USA);
    • computational runs on Redqueen;
    • post-processing on Redqueen, results on RVMS Web server.
  • Poster available.




 

More

More. . .

These slides:
RI Team Web site:
Ecosystem poster




 

Finally

Questions?
its-ri-team@manchester.ac.uk