{"id":2238,"date":"2019-02-08T16:11:05","date_gmt":"2019-02-08T16:11:05","guid":{"rendered":"http:\/\/ri.itservices.manchester.ac.uk\/csf3\/?page_id=2238"},"modified":"2026-08-12T15:18:44","modified_gmt":"2026-08-12T14:18:44","slug":"quota","status":"publish","type":"page","link":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/filesystems\/quota\/","title":{"rendered":"Storage quotas and usage"},"content":{"rendered":"

To ensure fair-usage of storage, the home<\/em> directory storage and any additional Research Data Storage<\/em> (RDS) areas have quotas applied to them. The following sections show how to check the quota of the areas of storage you have access to and how to measure your usage of the storage.<\/p>\n

We generally ask users to keep their home<\/em> storage to 50GB or less<\/strong>. RDS storage areas have their own (larger) quota, arranged with the owner of the storage. There are no quotas on scratch<\/em> but an automatic scratch cleanup policy<\/a> is enforced to delete old scratch files.<\/p>\n

Home Directories<\/h2>\n

Home directories are located on the Isilon<\/em> storage platform and therefore have quotas applied to them.<\/p>\n

Group Storage Quota and Usage<\/h3>\n

Your home directory<\/em> (folder) is itself in a parent folder associated with your research group. The group area has a storage quota<\/em> applied to it – the overall usage is restricted, usually to a total that depends on the size of the group. We request that users try not to use more than 50GB each<\/strong>.<\/p>\n

You can find out the overall quota<\/em> and usage<\/em> for your group by running the following command:<\/p>\n

df -h $HOME\r\n\r\nFilesystem                                     Size  Used Avail Use%<\/strong> Mounted on\r\nnas.isilonr.manchester.ac.uk:\/ifs\/nas\/path...  4.0T  2.5T  1.6T  63%<\/strong> \/mnt\/iusers01<\/em>\r\n     #                                           #                          #\r\n     #                                           #                          #\r\n     #                                           # The overall size:  4TB   #\r\n     # You can ignore the Filesystem<\/em> name        # Storage used:    2.5TB   # You can ignore the\r\n                                                 # Still available: 1.6TB   # Mounted on<\/em> name\r\n                                                 # Storage is 63% full\r\n<\/pre>\n
Remember, this is the overall usage and quota for your group’s storage area used for home directories. If you use too much storage you will fill your group’s area, preventing you and your colleagues from working!<\/strong><\/p>\n
Your home total usage<\/h3>\n
To measure your own home directory usage please submit the following batch job. We use a batch job because running the command on the login node can slow down the login node for all other users. Measuring your usage can also take some time to complete so running it in batch will allow you to carry on working (or you can log off and collect the result later).<\/p>\n
# Batch command run on the login node to measure your home directory usage:\r\nsbatch -p serial -t 15 -J home-dir-usage -o %x.o%j --wrap=\"du -schx ~\"\r\n<\/pre>\n
The total storage usage will be reported in a plain text file (in your home directory) named home-dir-usage.o12345<\/em><\/code> where 12345<\/em><\/code> will be a job-id number unique to your job.<\/p>\n
Scratch<\/h2>\n
The scratch<\/em> area ~\/scratch<\/code> is a different filesystem (different physical storage) to your home directory. There is no quota applied to this directory but you are expected to delete unwanted \/ temporary files after jobs have completed to ensure you don’t use excessive amounts of storage.<\/p>\n
The system can also delete files older than 3-months to free up space!<\/strong> (please see the scratch cleanup policy<\/a> for more information)<\/p>\n
Your scratch total usage<\/h3>\n
To see how much scratch space you are using and how many files you have stored there, run the following command on the login node:<\/p>\n
scrusage\r\n\r\n# Example output\r\nYour scratch usage: 450.7GB, 84,053 files\r\n                      #\r\n                      # This user is using 450 GB of scratch space (via 84,053 files).\r\n                      # This is a lot. They should delete some files!\r\n<\/pre>\n
Additional RDS areas<\/h2>\n
If your group has access to additional Research Data Storage<\/em>, usually accessible via a shortcut in your home directory named ~\/data<\/code> (or ~\/rds<\/code>) then you can check the group’s overall quota and your usage using similar commands to those used for the home<\/em> directory:<\/p>\n
# Overall quota and usage stats for additional RDS area\r\ndf -h ~\/rds<\/em>\/\r\n        #\r\n        # Use the name of your shortcut to your RDS area\r\n\r\n# Your total usage in the additional RDS area\r\nsbatch -p serial -t 15 -J rds-dir-usage -o %x.o%j --wrap=\"du -schx ~\/rds<\/em>\/\"\r\n<\/pre>\n
Faster and detailed interactive usage scanning<\/h2>\n
Running the scan<\/h3>\n
You can use the ncdu<\/code> utility to interactively scan any<\/em> directory you have Read<\/em> access to, including your home directory, scratch directory and any other place like an RDS share. This method is quicker than using the du<\/code> command and you can look through the sub-directories and find what is consuming most of your space.<\/p>\n
# First start an interactive job\/session<\/a>\r\nsrun -p interactive -n 8 -t 0-1 --pty bash\r\n\r\n# Load the ncdu module which will do the scanning\r\nmodule purge\r\nmodule load apps\/binapps\/ncdu\/2.9.1\r\n\r\n# cd to the directory which you want to scan\r\ncd\r\n#or\r\ncd ~\/scratch\/\r\n\r\n# Start scanning\r\nncdu -t $SLURM_NTASKS -r\r\n\r\n<\/pre>\n
Viewing and navigating the results<\/h3>\n
Scanning<\/h4>\n
While scanning is still in progress, you will see this view:
\n\"\"
\nLet the scanning complete.<\/p>\n
Scan result<\/h4>\n
When the scan is completed, you will be presented with the scan results.
\nAt the top line starting with ---<\/code> the path of the scanned directory is shown.
\nThen the sub-directories are listed in descending order, based on their size.
\n\"\"<\/p>\n
At the bottom left you will see the Total disk usage<\/strong> of the directory. It is 1.1TB<\/strong> in the above example.
\nAt the same line, towards the right end Items<\/strong> shows the total number of files and folders of the directory (3,693,319 in the example).<\/p>\n
Navigating<\/h4>\n
You can navigate into sub-directories by using the arrow keys<\/strong> and then hitting Enter<\/strong>. In this example using the down arrow key<\/strong>, we have scrolled down and selected\/highlighted the .conda<\/code> sub-directory:
\n\"\"<\/p>\n
With the .conda<\/code> sub-directory selected\/highlighted, when you hit Enter<\/strong>, you will be presented with the contents and detailed usage of the ..conda<\/code> sub-directory:
\n\"\"
\nTo return to the previous\/upper-level directory, navigate towards the top to highlight the item \/..<\/code><\/strong> and hit Enter<\/strong>.<\/p>\n
You can use i<\/code> to toggle on and off more information about a file\/directory – e.g. it will tell you how many files in a particular folder.<\/p>\n
Quitting<\/h4>\n
Press q<\/code><\/strong> to quit ncdu<\/strong>.
\nType exit<\/code><\/strong> and hit enter<\/strong> to exit the interactive job\/session.<\/p>\n
TIP<\/strong>: If you have lots of files and folders in the directory you are scanning, request a larger number of cores for your interactive job (e.g. -n 16<\/strong>). More number of cores means faster scanning. 8-Core is good for few hundred GB.<\/p>\n","protected":false},"excerpt":{"rendered":"
To ensure fair-usage of storage, the home directory storage and any additional Research Data Storage (RDS) areas have quotas applied to them. The following sections show how to check the quota of the areas of storage you have access to and how to measure your usage of the storage. We generally ask users to keep their home storage to 50GB or less. RDS storage areas have their own (larger) quota, arranged with the owner of.. Read more »<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"parent":94,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-2238","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/2238","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/comments?post=2238"}],"version-history":[{"count":21,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/2238\/revisions"}],"predecessor-version":[{"id":12607,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/2238\/revisions\/12607"}],"up":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/94"}],"wp:attachment":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/media?parent=2238"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}