Storage quotas and usage
To ensure fair-usage of storage, the home directory storage and any additional Research Data Storage (RDS) areas have quotas applied to them. The following sections show how to check the quota of the areas of storage you have access to and how to measure your usage of the storage.
We generally ask users to keep their home storage to 50GB or less. RDS storage areas have their own (larger) quota, arranged with the owner of the storage. There are no quotas on scratch but an automatic scratch cleanup policy is enforced to delete old scratch files.
Home Directories
Home directories are located on the Isilon storage platform and therefore have quotas applied to them.
Group Storage Quota and Usage
Your home directory (folder) is itself in a parent folder associated with your research group. The group area has a storage quota applied to it – the overall usage is restricted, usually to a total that depends on the size of the group. We request that users try not to use more than 50GB each.
You can find out the overall quota and usage for your group by running the following command:
df -h $HOME
Filesystem Size Used Avail Use% Mounted on
nas.isilonr.manchester.ac.uk:/ifs/nas/path... 4.0T 2.5T 1.6T 63% /mnt/iusers01
# # #
# # #
# # The overall size: 4TB #
# You can ignore the Filesystem name # Storage used: 2.5TB # You can ignore the
# Still available: 1.6TB # Mounted on name
# Storage is 63% full
Remember, this is the overall usage and quota for your group’s storage area used for home directories. If you use too much storage you will fill your group’s area, preventing you and your colleagues from working!
Your home total usage
To measure your own home directory usage please submit the following batch job. We use a batch job because running the command on the login node can slow down the login node for all other users. Measuring your usage can also take some time to complete so running it in batch will allow you to carry on working (or you can log off and collect the result later).
# Batch command run on the login node to measure your home directory usage: sbatch -p serial -t 15 -J home-dir-usage -o %x.o%j --wrap="du -schx ~"
The total storage usage will be reported in a plain text file (in your home directory) named home-dir-usage.o12345 where 12345 will be a job-id number unique to your job.
Scratch
The scratch area ~/scratch is a different filesystem (different physical storage) to your home directory. There is no quota applied to this directory but you are expected to delete unwanted / temporary files after jobs have completed to ensure you don’t use excessive amounts of storage.
The system can also delete files older than 3-months to free up space! (please see the scratch cleanup policy for more information)
Your scratch total usage
To see how much scratch space you are using and how many files you have stored there, run the following command on the login node:
scrusage
# Example output
Your scratch usage: 450.7GB, 84,053 files
#
# This user is using 450 GB of scratch space (via 84,053 files).
# This is a lot. They should delete some files!
Additional RDS areas
If your group has access to additional Research Data Storage, usually accessible via a shortcut in your home directory named ~/data (or ~/rds) then you can check the group’s overall quota and your usage using similar commands to those used for the home directory:
# Overall quota and usage stats for additional RDS area
df -h ~/rds/
#
# Use the name of your shortcut to your RDS area
# Your total usage in the additional RDS area
sbatch -p serial -t 15 -J rds-dir-usage -o %x.o%j --wrap="du -schx ~/rds/"
Faster and detailed interactive usage scanning
Running the scan
You can use the ncdu utility to interactively scan any directory you have Read access to, including your home directory, scratch directory and any other place like an RDS share. This method is quicker than using the du command and you can look through the sub-directories and find what is consuming most of your space.
# First start an interactive job/session srun -p interactive -n 8 -t 0-1 --pty bash # Load the ncdu module which will do the scanning module purge module load apps/binapps/ncdu/2.9.1 # cd to the directory which you want to scan cd #or cd ~/scratch/ # Start scanning ncdu -t $SLURM_NTASKS -r
Scanning
While scanning is still in progress, you will see this view:
Let the scanning complete.
Scan result
When the scan is completed, you will be presented with the scan results.
At the top line starting with --- the path of the scanned directory is shown.
Then the sub-directories are listed in descending order, based on their size.
At the bottom left you will see the Total disk usage of the directory. It is 1.1TB in the above example.
At the same line, towards the right end Items shows the total number of files and folders of the directory (3,693,319 in the example).
Navigating
You can navigate into sub-directories by using the arrow keys and then hitting Enter. In this example using the down arrow key, we have scrolled down and selected/highlighted the .conda sub-directory:
With the .conda sub-directory selected/highlighted, when you hit Enter, you will be presented with the contents and detailed usage of the ..conda sub-directory:
To return to the previous/upper-level directory, navigate towards the top to highlight the item /.. and hit Enter.
You can use i to toggle on and off more information about a file/directory – e.g. it will tell you how many files in a particular folder.
Quitting
Press q to quit ncdu.
Type exit and hit enter to exit the interactive job/session.
TIP: If you have lots of files and folders in the directory you are scanning, request a larger number of cores for your interactive job (e.g. -n 16). More number of cores means faster scanning. 8-Core is good for few hundred GB.