Slide Navigation

[[ Slide navigation:

]]    

Introductions

The UoM CIR Ecosystem

CIR:

• Computationally-Intensive Research

RI Team:

• Pen Richardson, George Leaver

Agenda

The UoM CIR Ecosystem

• Compute resources,
• batch,
• interactive and
• emerging technology;
• research data storage — aka Isilon — (TMR4BH);
• virtual machines for researchers.

The Goal:

• A complete environment for CIR.

Ecosystem Diagram

Computational Resources

Computational Resources

Batch

• Condor
• CSF
• Kadmon
• Hydra

Interactive

• iCSF

Isilon

Isilon Storage

IT Services Research Data Storage (RDS) Service

• FLS has 100 TB per year to allocate, free-at-the-point-of-use. . .
• http://ri.itservices.manchester.ac.uk/rds

What is Isilon?

What?

• Network (attached) storage — cf. MS Windows shared areas/drives
• Resilient (replicated) and snapshots (cf. backups).
• Both:
  • MS Windows-style shares (CIFS);
  • Unix/Linux-style exports (NFS).
• UoM committed to procuring 500 TB more per year (100 TB for FLS).

How/where is it accessible?

• Desktops, laptops, laboratories. . .
• Computational clusters — common folders/dirs: CSF, Kadmon, Hydra. . .

How much does Isilon cost?

• FLS have 100 TB per year to allocate FATPOU;
• more can be purchased. . .

How do I get an Isilon Share?

Either email

• FLS-ITResearch@manchester.ac.uk

• http://ri.itservices.manchester.ac.uk/rds
• and follow link to request form.

Mounting CIFS shares on MS Windows

1. Locate the Map Network Drive option:
   • XP: Right-click on My Computer and select Map network drive.
   • Win 7: Start --> Computer --> Click "Map network drive"
2. In the Drive list select the required drive letter (e.g., R:).
3. In the Folder box enter the path according to your faculty and mount point. For example

       \\nasr.man.ac.uk\epsrss$\snapped\replicated\myshare
       

4. Username: one of

       ds\mxyzpqr2
       mxyzpqr2@ds.man.ac.uk
       

5. Password: central/ITServices.

Mounting CIFS shares on Linux

If it is a one-off:

  mount -t cifs -o \
      user,noauto,username=mxyzpqr7,domain=ds.man.ac.uk \
      //nasr.man.ac.uk/epsrss$/snapped/replicated/myshare \
      /mnt/myshare/

If it is a many-off, use /etc/fstab:

  //nasr.man.ac.uk/epsrss$/snapped/replicated/myshare \
      /mnt/myshare/ \
      cifs \
      user,noauto,username=mxyzpqr7,domain=ds.man.ac.uk \
      0 0

then simply

  mount /mnt/myshare

Mounting CIFS shares on OS-X

Maverick (10.9), also 10.6

• Use finder:
  smb://nasr.man.ac.uk/flsrss$/snapped/replicated/myshare

10.7 and 10.8 (Lion and Mountain Lion)

• Finder does not work — use supplied script.

Snapshots

Recover your own files from snapshots

• Top of share/export: .snapshot
• Hidden — not even ls -a (enter path manually)

Example:

simonh@doolittle:/mnt/pdrive$ ls -a
.         Desktop         file3.yes          My Videos       
..        desktop.ini     latex_cookbook.ps  public_html     
.emacs.d  untitled-1.aux  untitled-1.log     untitled-1.tex

simonh@doolittle:/mnt/pdrive$ cd .snapshot
simonh@doolittle:/mnt/pdrive/.snapshot$ ls
.    .
.    .
kilburn_24hr_2014-01-20_13-30-00  kilburn_24hr_2014-01-20_06-30-00   
kilburn_24hr_2014-01-20_14-30-00  kilburn_24hr_2014-01-20_07-30-00   
.    .
.    .

Desktop Backups

This is very simple — no sophisticated script required

1. Get an Isilon CIFS share;
2. configure your desktop to ensure it is always mapped/mounted;
3. set up a cron job to perform an rsync each day;
4. snapshots automatically provide file retrieval for 28 days.

But better to simply use your Isilon share for your files???

INCLINE

INCLINE

The INteractive Computational LINux Environment

Sometimes you need to do interactive computation:

• short development runs;
• GUI-based work.
• INCLINE is a compute engine without queues. . .
• . . .simply SSH-in and use. . .

RVMS

RVM Service

• Research Virtual Machine Service
• Resilient, supported infrastructure for researchers
• esp. Web servers
• Free at the point of use
• In pilot
• Eliminate "under-desk servers"?
• http://ri.itservices.manchester.ac.uk/rvms

RVM Service Use Cases

What is the RVM Service suitable for?

• Public facing Web sites with moderate traffic;
• shared development environments;
• Web front ends to DBs, HPC. . .

. . .NOT. . .?

• Compute-intensive work;
• memory or IO-intensive work.

Getting a VM

How do I get an RVMS VM?

• Ask the FLS CIR people (Nick, Simon, Ziaul), or
• email

          its-ri-team@manchester.ac.uk

N.B.

• Service is in pilot phase. . .

The Ecosystem: A Complete Solution?

The Goal

• Reduce desktop requirements to thin clients (plus email, office, surfing. . .).
• All CIR requirements addressed and integrated:
• all infrastructure run by IT staff (not postgrads/docs);
• hardware shared — fewer wasted CPU cycles, access to "bigger" hardware than otherwise. . . (cf. CSF model).

Workflow

ManUniCast

An Existing Ecosystem Usecase

• Weather and pollution forecasts at the University of Manchester!
• www.manunicast.com
• Daily:
• download input datasets from NOAA (USA);
• computational runs on Redqueen;
• post-processing on Redqueen, results on RVMS Web server.
• Poster available.

More

More. . .

These slides:

• http://ri.itservices.manchester.ac.uk/~simonh

RI Team Web site:

• http://ri.itservices.manchester.ac.uk

Ecosystem poster

• User View

Finally