Two Types of Storage<\/h3>\n
The Research Data Storage (RDS) Service offers two different types<\/a> of storage shares<\/em> to researchers:<\/p>\n All users of the CSF \/ iCSF will have a home directory<\/em> on those systems. This is also “NFS” storage.<\/li>\n<\/ul>\n Some users require both types of storage and so it would be convenient to be able to see both types of storage in the same place<\/em>. For example, to copy files between the desktop-accessible storage and the CSF\/iCSF-accessible storage. The RDS-SSH service makes this possible!<\/p>\n When requesting a new storage allocation, please indicate which type of storage you need – SMB (for desktops\/laptops) or NFS (for use on the CSF\/iCSF). We will need this information from you before we can allocate storage. How to decide?<\/p>\n We can discuss further with you which type of storage best suits your requirements. But indicating your preference in your storage request will speed up the allocation of your storage.<\/p>\n You can log in to the rds-ssh<\/em> server and access your “CIFS” (desktop) research data storage and your “NFS” (CSF\/iCSF) research data storage at a Linux command-line. Copying files between the two types of storage on the rds-ssh server is fast.<\/p>\n As noted above, the “NFS” (CSF\/iCSF) storage is usually only visible on the CSF\/iCSF, and it isn’t possible to mount<\/em> that storage directly<\/em> on your desktop PC \/ laptop. But there are 3rd-party applications that can be installed on Windows, Mac and Linux desktop\/laptops to mount this storage via<\/em> the rds-ssh server.<\/p>\n The rds-ssh server can also be used to upload \/ download large files to your “NFS” (CSF\/iCSF) storage. This keeps large file-transfers away from the CSF login node, which is used by many users for other tasks.<\/p>\n Please note<\/strong>: For off-campus access to the rds-ssh<\/em> server please read our dedicated guide<\/a> (login required.)<\/p>\n Please read on for how to obtain an account on the rds-ssh server<\/em>, how to log in, and how to access your various storage shares<\/em>.<\/p>\n To obtain an account on RDS-SSH, please contact us via our Connect Portal Form<\/a> (select “Provide Access to Existing Storage” from the drop-down menu in the form and indicate in the text that you require an RDS-SSH account.<\/p>\n Please tell us if you need a particular Research Data Storage share<\/em> (i.e., storage area) made available on the rds-ssh server. Note that all CSF \/ iCSF home<\/em> storage areas are already visible on the rds-ssh server (and when you log in you’ll be in<\/em> you CSF home directory, if you have a CSF account).<\/p>\n Various methods of accessing the rds-ssh server are described below. In all cases you should use the following settings:<\/p>\n When accessing on-campus resources such as the RDS-SSH service from off-campus you must be signed in to the University VPN, which can be accessed using the Global Protect software.<\/p>\n By connecting to the RDS-SSH service, you can access your RDS shares using various tools and commands, such as SCP, SFTP, SSHFS and\/or rsync-over-SSH and directly at the Linux command-line on the rds-ssh server\/. For details of each, see below.<\/p>\n You can log in to the rds-ssh<\/em> server using an ssh<\/em> app to perform file management commands directly on the server. For example, this allows you to copy files between different shares<\/em> directly on the server using the fast data-centre network.<\/p>\n You can transfer files to and from the rds-ssh<\/em> server using a scp<\/em> app (e.g., WinSCP on Windows or the Follow the instructions below on how to use these methods of accessing your files.<\/p>\n On Windows you can install MobaXTerm and\/or WinSCP to transfer files to\/from the rds-ssh server. Please follow the CSF MobaXTerm instructions<\/a> and simply change any references to:<\/p>\n to<\/p>\n If logging in from off-campus you will first need to be connected to the University VPN (GlobalProtect)<\/a> – You may already connect to the University VPN in order to access your email.<\/p>\n If on-campus and not connected to the University VPN you will need to authenticate using your 2FA device.<\/p>\n If you use the Duo mobile app as your 2FA device, Enter ‘1’ at the prompt and press Enter. Once the push notification has been received via the Duo mobile app ‘Approve’ the request in order to log on.<\/p>\n OR<\/strong><\/p>\n If you use a Duo fob as your 2FA device, generate a passcode with the fob, type the passcode at the prompt and press Enter<\/p>\n <\/p>\n WinSCP is a graphical file transfer program which can be used to drag-n-drop files between your local desktop and the rds-ssh server. Please follow the CSF WinSCP instructions<\/a> and simply change any references to:<\/p>\n to<\/p>\n If logging in from off-campus you will first need to be connected to the University VPN (GlobalProtect)<\/a> – You may already connect to the University VPN in order to access your email.<\/p>\n If on-campus and not connected to the University VPN you will need to authenticate using your 2FA device.<\/p>\n WinSCP will present a dialogue box similar (see image below)<\/p>\n If you use a Duo fob as your 2FA device, generate a passcode with the fob, type the passcode at the prompt and press Enter<\/p>\n From a linux or MacOS desktop\/laptop SSH, SCP & RSYNC can be done at the command-line. For example, to SSH in to the rds-ssh server use:<\/p>\n Replace If logging in from off-campus you will first need to be connected to the University VPN (GlobalProtect)<\/a> – You may already connect to the University VPN in order to access your email.<\/p>\n If on-campus and not connected to the University VPN you will need to authenticate using your 2FA device.<\/p>\n If you use the Duo mobile app as your 2FA device, Enter ‘1’ at the prompt and press Enter. Once the push notification has been received via the Duo mobile app ‘Approve’ the request in order to log on.<\/p>\n OR<\/strong><\/p>\n If you use a Duo fob as your 2FA device, generate a passcode with the fob, type the passcode at the prompt and press Enter<\/p>\n You can then use linux command-line file management tools (such as We have also provided the graphical file manager, XFE,<\/a> on the rds-ssh server which will allow you to browse your files and to copy files between directories available on RDS-SSH. You cannot transfer files to your desktop with XFE.<\/p>\n SCP (or RSYNC) can be used from the linux or MacOS command-line (and MobaXterm command line) to transfer files between your local desktop\/laptop and the rds-ssh server. For example:<\/p>\n SFTP (secure file transfer protocol) is another method of transferring files to and from the rds-ssh<\/em> server. It seems to be less frequently used on systems nowadays as apps such as WinSCP, MobaXTerm and the and supply your central IT Services username and password when prompted.<\/p>\n <\/p>\n Use this method if you want an RDS share (e.g., your CSF\/iCSF\/Redqueen\/Hydra home directory) to appear as a network drive on your desktop\/laptop (in Windows or Linux\/MacOS).<\/p>\n Note: Don’t be tempted to mount two different RDS shares (e.g., as drives E: and F:) and copy (or move) files from one to the other. This is slow because the files will be transfered via<\/em> your desktop\/laptop which could involve either the slower campus network, wi-fi (usually quite slow) or home broadband (uploading can be very<\/em> slow). If you want to copy\/move files between different RDS shares please see this FAQ entry<\/a> for how to do it using only the fast network in the data-centre. This will make your RDS share appear as a network drive in Windows, even if your RDS share is an NFS-share (normally only CIFS-shares can be mapped as network drives in Windows).<\/p>\n Please see our SSHFS On Windows<\/a> documentation for full details.<\/p>\n The user should log out of the system and back in for this to take effect.<\/li>\n<\/ol>\n Once SSHFS is installed and configured, as above, make a directory to use as a mount point:<\/p>\n Next mount your share<\/p>\n On campus<\/strong><\/p>\n OR<\/strong> Once you no longer need your share mounted, unmount it like this:<\/p>\n These are set within which means that other users on your Linux host can access may access the filesystem.<\/dd>\n<\/dl>\n You will need to download and install macFUSE then SSHFS from macfuse.io<\/a>. There are further installation notes for macFUSE<\/a> and installation notes for SSHFS<\/a> on that site’s Wiki.<\/p>\n Once the software is installed, you can then use SSHFS from a BASH command-line, exactly as above<\/a>, for Linux, to mount a RDS share on your Mac \u2014 once mounted the volume will appear as an icon on your desktop.<\/p>\n To unmount the volume, simply right-click on the icon and select Eject<\/em>.<\/p>\n <\/a><\/p>\n A number of download tools from the UK BioBank and European Genome Archive (EGA) are available on the RDS-SSH servers. These are automatically in your The following UKBioBank download tools are installed – please do not download your own version, it may not work. You can simply run these commands at the prompt once you’ve logged in to For more information on how to run these commands, please see http:\/\/biobank.ndph.ox.ac.uk\/showcase\/download.cgi<\/a>. Follow the links for each tool – this will take you through to a description of the tool where you can also access the user-guide.<\/p>\n Other download tools are also installed. The links below will take you to the instructions for each tool, provided by the authors of the software.<\/p>\n Overview The RDS-SSH server allows you to see both types of storage offered by the Research Data Storage (RDS) Service, in one place. Two Types of Storage The Research Data Storage (RDS) Service offers two different types of storage shares to researchers: Most commonly, RDS shares are just like an IT Services P-drive, i.e., a MS Windows-style “SMB” share which is only accessible on a desktop PC or laptop. Linux-style “NFS” shares are also available.. Read more »<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-361","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/pages\/361","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/comments?post=361"}],"version-history":[{"count":25,"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/pages\/361\/revisions"}],"predecessor-version":[{"id":1174,"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/pages\/361\/revisions\/1174"}],"wp:attachment":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/media?parent=361"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
\n
What do I use the rds-ssh system for?<\/h3>\n
Getting an Account and Accessing the Service<\/h2>\n
Login Details<\/h3>\n
\n
rds-ssh.itservices.manchester.ac.uk<\/code><\/li>\nThe University VPN and GlobalProtect<\/h3>\n
\n
How can you access your files? SSH, SCP, SFTP, SSHFS and rsync-over-SSH<\/h2>\n
What can you use RDS-SSH for?<\/h2>\n
\n
Logging in – Simple SSH or SCP Access<\/h2>\n
scp<\/code> command on linux). This is simple way of uploading files to your shares (e.g., to upload data ready for use on the CSF) or to transfer results back from your shares.<\/p>\nWindows (Mobaxterm)<\/h3>\n
csf.itservices.manchester.ac.uk<\/pre>\n
rds-ssh.itservices.manchester.ac.uk<\/pre>\n
Off-campus<\/b><\/h4>\n
\n
On-campus<\/b><\/h4>\n
$ ssh username<\/em>@rds-ssh.itservices.manchester.ac.uk\r\nPassword:\r\nEnter a passcode or select one of the following options: \r\n\r\n1. Duo Push to +XX XXXX XXX555\r\n\r\nPasscode or option (1-1):1\r\n<\/pre>\n
$ ssh username<\/em>@rds-ssh.itservices.manchester.ac.uk\r\nPassword:\r\nDuo two-factor login for username<\/em>\r\n\r\nPasscode: 123456\r\n<\/pre>\n
\n
\n![\"Do](\"http:\/\/ri.itservices.manchester.ac.uk\/wp-content\/uploads\/mobaxterm-save-pw-red.png\")
<\/a>
\nWe strongly<\/em> recommend you select No<\/strong> because saving passwords inside applications is a security risk. You tell MobaXterm to never save your passwords by ticking the box labelled ‘Do not show this message again’.<\/li>\n<\/ul>\n\n
\n
\n![\"Mobaxterm](\"http:\/\/ri.itservices.manchester.ac.uk\/wp-content\/uploads\/Screenshot-from-2020-07-31-09-53-46.png\")
<\/a><\/p>\n\n
\n
\n
Windows (WinSCP)<\/h3>\n
csf.itservices.manchester.ac.uk<\/pre>\n
rds-ssh.itservices.manchester.ac.uk<\/pre>\n
Off-campus<\/b><\/h4>\n
\n
On-campus<\/b><\/h4>\n
![\"WinSCP](\"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-content\/uploads\/winscp-2fa-300x235.png\")
\nIf you use the Duo mobile app as your 2FA device, Enter ‘1’ at the prompt and press Enter. Once the push notification has been received via the Duo mobile app ‘Approve’ the request in order to log on.<\/p>\nLinux \/ MacOS<\/h3>\n
ssh username<\/em>@rds-ssh.itservices.manchester.ac.uk\r\n<\/pre>\n
username<\/em><\/code> with your central University username. Your password is your central IT Services password. You may then be presented with a 2FA stage.<\/p>\nOff-campus<\/b><\/h4>\n
\n
On-campus<\/b><\/h4>\n
$ ssh username<\/em>@rds-ssh.itservices.manchester.ac.uk\r\nPassword:\r\nEnter a passcode or select one of the following options: \r\n\r\n1. Duo Push to +XX XXXX XXX555\r\n\r\nPasscode or option (1-1):1\r\n<\/pre>\n
$ ssh username<\/em>@rds-ssh.itservices.manchester.ac.uk\r\nPassword:\r\nDuo two-factor login for username<\/em>\r\n\r\nPasscode: 123456\r\n<\/pre>\n
mv<\/code>, cp<\/code>, rm<\/code>, mkdir<\/code> rmdir<\/code>).<\/p>\nSCP (or RSYNC)<\/h2>\n
# Example: transfer a file to the Isilon storage directory ~\/data\/experiment1 \r\n# (~ is shorthand for your home directory).\r\nscp my-local-file.txt username<\/em>@rds-ssh.itservices.manchester.ac.uk:~\/data\/experiment1\/\r\n\r\n# scp example transfer a file from your Isilon home directory to your local computer\r\n# (notice the . at the end - shorthand for 'current local directory')\r\nscp username<\/em>@rds-ssh.itservices.manchester.ac.uk:~\/run.log .\r\n\r\n# Example: transfer a file to the Isilon storage directory ~\/data\/experiment1 \r\n# (~ is shorthand for your home directory).\r\nrsync -avz my-local-file.txt username<\/em>@rds-ssh.itservices.manchester.ac.uk:~\/data\/experiment1\/\r\n\r\n# rsync example of transferring a file from your Isilon home directory\r\n# (notice the . at the end - shorthand for 'current local directory')\r\nrsync -avz username<\/em>@rds-ssh.itservices.manchester.ac.uk:~\/run.log .\r\n\r\n<\/pre>\n
SFTP<\/h2>\n
scp<\/code> command have become more popular. However, if you have an SFTP program (e.g., FileZilla or FireFTP) then rds-ssh<\/em> will accept SFTP connections. Note that non-secure (i.e., ordinary FTP) connections are not possible. You application will ask for the host to connect to – simply use:<\/p>\nrds-ssh.itservices.manchester.ac.uk\r\n<\/pre>\n
Mapping your RDS (CIFS or) NFS Share on your Desktop\/Laptop via SSHFS<\/h2>\n
\n<\/a><\/p>\nUsing SSHFS from MS Windows<\/h3>\n
Using SSHFS on Linux<\/h3>\n
\n
\n
apt-get install sshfs\r\n<\/pre>\n<\/li>\n
fuse-sshfs<\/code> RPM from the EPEL repository:\n# First add epel to your repos then do:\r\nyum install fuse-sshfs\r\n<\/pre>\n<\/li>\n<\/ul>\n<\/li>\n
fuse<\/code> group<\/strong>. For example, to add a user to the fuse group (you must have root privileges):\nusermod -a -G fuse username<\/em>\r\n<\/pre>\n
mkdir my-rds-share\r\n<\/pre>\n
sshfs mpqrxyz2@rds-ssh.itservices.manchester.ac.uk:\/mnt\/iusers01\/support\/mpqrxyz2 my-rds-share\r\n # ...enter your IT Services password when prompted...\r\n # ...if on-campus you will need to authenticate via duo ...\r\n<\/pre>\n
Enter a passcode or select one of the following options: \r\n\r\n1. Duo Push to +XX XXXX XXX555\r\n\r\nPasscode or option (1-1):1\r\n<\/pre>\n
\nIf you use a Duo fob as your 2FA device, generate a passcode with the fob, type the passcode at the prompt and press Enter<\/p>\nDuo two-factor login for username<\/em>\r\nPasscode: 123456\r\n<\/pre>\n
fusermount -u my-rds-share\r\n<\/pre>\n
SSHFS Options on Linux<\/h4>\n
\/etc\/fuse.conf<\/code>:<\/p>\n\n
allow_other<\/code> option when mounting a filesystem via<\/em> SSHFS, e.g.,<\/p>\nsshfs -o allow_other mpqrxyz2@rds-ssh.itservices.manchester.ac.uk my-rds-share\r\n<\/pre>\n
Using SSHFS from MacOS-X<\/h3>\n
UKBioBank \/ EGA Download Tools<\/h2>\n
PATH<\/code> upon login – simply login and run the commands you would normally run. If the command requires a keyfile, ensure you have copied the key file to your home directory. For example:<\/p>\nukbfetch -e123456 -d6789_1_0 -ak12345.key\r\n<\/pre>\n
rds-ssh.itservices.manchester.ac.uk<\/code>:<\/p>\nukbmd5 # Calculate size and MD5 of a file\r\nukbconv # Convert unpacked UKB data to other formats\r\nukbunpack # Unpack (decrypt and decompress) UKB data\r\nukbfetch # The bulk data download tool\r\nukblink # Download Returned-datasets and link between Applications\r\nukbgene # Download approved genetic data. This tool supersedes a tool named gfetch.\r\n<\/pre>\n
\n
egaclient<\/code> (this will automatically load the EgaDemoClient.jar<\/code> file) and egacryptor<\/code> – EgaDemoClient download client<\/a> (java downloader and file decrypter)<\/li>\nascp<\/code> (uses the default aspera keyfile) and ascp_noid<\/code> (you should add the -i PRIVATE-KEY-FILE<\/em><\/code> flags) – Aspera download client<\/a> (Linux command-line downloader)<\/li>\nbasemount<\/code> Illumina BaseSpace command-line tools – Illumina Basemount<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"