File Transfer
Estimated Time to Complete: 20 minutes
Overview
Working on the various compute platforms such as the iCSF and CSF you will often find yourself having to transfer files between your local computer and the platform you are working on. There are various methods of transferring files to and from the various compute platforms Service. This Tutorial will show how to transfer files using the recommended method between your local computer and any of the compute platforms. All methods take advantage of the Secure copy Protocol (scp). The scp command allows you to copy files over a ssh connection. This is useful if you want to transport files between a local desktop and remote host e.g. CSF/iCSF. The scp command uses ssh therefore any files copied in this way are done so in a secure manner via. The scp command can be used in three ways:
- Copy from a (remote) server to your computer
- Copy from your computer to a (remote) server
- Copy from a (remote) server to another (remote) server.
Requirements
- Windows – WinSCP installed
- Linux/MacOS – Some basic command line knowledge
- CSF/iCSF Account
- Sample Data – SampleData.zip
- If Off-Campus you will need to be connected to the UoM VPN (Global Protect) with 2FA
Further information and instructions on other methods can be found by following this link.
WinSCP
Video Demonstration
How To
-
- If you haven’t already download and extract the SampleData.zip dataset to your local desktop.
- Open WinSCP and from the Login window create a new session.
Please note: If the Login window does not appear you can create a new session by doing the following:
Session > New Session
-
- Under the Login Window enter the following in
- File Protocol: Select SCP
- Host name: Enter one of the following hosts depending on your destination.
- Under the Login Window enter the following in
System | Hostname used for Access |
---|---|
iCSF/incline | incline256.itservices.manchester.ac.uk |
CSF3 | csf3.itservices.manchester.ac.uk |
Zrek | zrek.itservices.manchester.ac.uk |
RDS-SSH | rds-ssh.itservices.manchester.ac.uk |
Condor | submitter.itservices.manchester.ac.uk |
Please note:For the purposes of this tutorial we will connect to
incline256.itservices.manchester.ac.uk
-
-
- Port: Enter
22
- User Name: Enter your
central username
- Password: Leave blank
- Select Advanced Under the Connection option enter the following
-
- Server Response timeout: Enter
60
- Select OK
- Server Response timeout: Enter
</ul
-
- Select Save
- Select Login
- Port: Enter
-
Please note: The first time you connect using WinSCP you will likely receive the following message ‘Continue connecting to an unknown server and add its host key to cache…‘ This is a fundamental feature of SSH, helping to prevent your data being intercepted by a third party by providing a known “key” Select
Yes
-
- When prompted for Password: Enter your
central password
- When prompted for Password: Enter your
- The main WinSCP commander window should open this is similar to a Windows explorer window. You should notice the window is split into a left pane and right pane.
- Left Pane = Local Computer
- Right Pane = Remote Directory e.g. incline.
- Now lets transfer a file. If you haven’t already download and extract the SampleData.zip dataset to your local desktop.
-
- Now in left pane of the WinSCP commander window navigate to where you extracted the SampleData.zip.
Please note: It the file is not appearing it maybe necessary to click the refresh button
- Now in the right pane lets create a new directory for the file. Select
>Directory>
and give it a name e.g. uploads - Navigate into to the newly created directory
- In order to transfer the SampleData directory simple drag’n’drop from one pane to the other
-
- Now lets connect to incline and confirm the transfer was successful
$ ssh -X incline256.itservices.manchester.ac.uk
- In you incline home directory list contents by using the
ls
command – You should notice the uploads directory has been created. - Lets list the contents of the uploads directory using the following command
ls uploads
– You should see theSampleData
has successfully been transferred.
- Imagine you performed some analysis on the data and produced a plot. You should notice a
Rplot.pdf
file in the SampleData/Outputs folder lets copy that from incline back to our Downloads folder simply by drag’n’droping the file from the left hand pane to the right hand pane using WinSCP
SCP via Terminal
The SCP utility is available via the Terminal appication in both the Linux and MacOS operarting systems.
In the below example we use incline256.itservices.manchester.ac.uk
as our remote destination/host. This can be changed depending on the system in use see below table a for list of the available various.
System | Hostname used for Access |
---|---|
iCSF/incline | incline256.itservices.manchester.ac.uk |
CSF3 | csf3.itservices.manchester.ac.uk |
Zrek | zrek.itservices.manchester.ac.uk |
RDS-SSH | rds-ssh.itservices.manchester.ac.uk |
Condor | submitter.itservices.manchester.ac.uk |
Video Demonstration
How to
- If you haven’t already download SampleData.zip
- Open the Terminal Application
- Change directory to the location downloaded the SampleData
cd ~/Downloads
unzip SampleData.zip
ls
- Now lets copy the SampleData folder over to Incline (or other compute platform)
scp -r SampleData mzysscg3@incline256.itservices.manchester.ac.uk:~/uploads/
PLEASE NOTE: The syntax of the scp command is as follows
scp [options][source][Destination]
. The destination directory also needs to exist prior to copying any data. In the above example we included the-r
option so it will copy the directory recursively i.e. everything within the directory. The source file is theSampleData
folder and the Destination ismzysscg3@incline256.itservices.manchester.ac.uk:~/uploads
anything after:
denotes where the file/will be copied to. In this particular example we are coping to~/uploads
the~
denotes our home directory . If you wanted you could specify a different directory e.g.:/some/other/directory/
- Now lets hop onto incline256 to check the folder has been copied successfully over
ssh -X username@incline256.itservices.manchester.ac.uk
cd ~
Moves us into our home directoryls
You should see the SampleData directory listed- Let check the contents of the SampleData directory
ls SampleData
you should see everything was successfully copied over.
- Now imagine you performed some analysis on the data and produced a plot. You should notice a .pdf file in the SampleData/Outputs folder lets copy that from incline to our desktop.
- On our desktop/laptop lets first create a directory for the file to be copied into e.g.
mkdir -p ~/Downloads/fromIncline
the-p
creates any parent directoriesscp -r mzysscg3@incline256.itservices.manchester.ac.uk:~/SampleData/Outputs/Rplots.pdf ~/Downloads/fromIncline
Next Steps
You should now be able to transfer files between your local computer and various computational platforms. Have a go a transferring some some more files and directories.
If you experienced any problems or require further support then please email its-ri-team@manchester.ac.uk
Further Reading
Virtual Desktop Service
WinSCP
Research IT Documentation on File Transfers.