Dos and don’ts

Running CPU and/or memory intensive work on the CSF

All users are reminded that they MUST NOT run applications or test anything resource intensive on the login node. All jobs should be submitted to the batch system. For short test jobs and development work we provide the short environment in batch and the qrsh command for interactive work. For advice on this please contact us via: its-ri-team@manchester.ac.uk

Computational work found running on the login nodes will be killed without warning. Anyone who causes disruption to the service may have restrictions applied to their account.

Need to run many similar jobs?

Please use SGE Job Arrays as these cause the least strain to the batch system.
Questions to: its-ri-team@manchester.ac.uk.

Home disk space is limited

• All users are reminded that the home filesystems are quite small and must be shared by many people. Please regularly review your usage and tidy up.
• Do not run jobs from your home directory, the output often consumes too much space and batch jobs can also put considerable strain on the filesystem. Please use the very large high performance scratch area. Jobs with a lot of IO will also run much better in scratch. Copy important results / files back to home (which is backed-up).
• Jobs run from home are at risk of failure if the filesystem fills up. As advised above run in scratch.
• When home filesystems fill up it has a negative impact on the ability of everyone in your group to use the CSF, as it hampers doing even basic tasks on the system.
• Files older than 3-months in scratch will be deleted automatically. Run jobs in scratch then copy important results / files back to home (which is backed-up).
• Small files and empty files (e.g. some output files produced by SGE but not written to) take up a disproportionate amount of space. Please tarball these together. This advice applies to all filesystems.
• Large files, on all file systems, should be compressed using gzip or bzip2 when they are not in use.
• Resarch Group PIs may request additional storage, mountable on the CSF, from the Research Data Storage Service.
• A quota may be applied to anyone deemed to be using more than their fair share of home space.

Questions to: its-ri-team@manchester.ac.uk.

Moving large amounts of data to and from the CSF

Please do not use the login nodes for this as it can put a lot of strain on them and each usually has around 100 users logged in at the same time. We have introduced a new service, RDS-SSH for doing large file transfers. Please contact its-ri-team@manchester.ac.uk for access.

Scratch policy

Please remember that your scratch space is not suitable for long-term storage of files. The agreed policy is that any files of three months or older may be deleted. If you need to keep large amounts of files (which are too big for your home directory) for more than three months, please email its-ri-team@manchester.ac.uk