File Transfers to and from Condor

Introduction

Files can be transferred between your PC/laptop and Condor in a number of ways. GUI programs (such as WinSCP on Windows) or command-line programs (such as rsync on Mac/Linux) can be used. The sections in the menu on the left show our recommended / preferred method for each platform.

Filesystems

It is possible to transfer files to and from your home file store. You home area is your private folder on the submitter node. It is also the same as the home folder on the CSF and iCSF (if you happen to have access to those systems.) All transfers will go via the “submitter” login node.

Note that the home storage area on “submitter” (shared between CSF / iCSF) is NOT visible on the Condor compute nodes. Similarly, your scratch directory on “submitter” (~/scratch/, which is a shortcut to /scratch/username) is NOT visible on the Condor compute nodes. You can submit Condor jobs from either of these storage areas while on the “submitter” node. Condor will then copy files over the network to its compute nodes, according to the directives you place in your Condor submit script.

This differs from the campus HPC systems (CSF, iCSF) which have the home and scratch filesystems visible on all compute nodes. But that is not how Condor works.

We recommend that you submit your Condor jobs from the scratch filesystem (~/scratch) to avoid the possibility of an incorrect submit script causing a job to fill up the shared isilon home area. But you must then copy any results files you wish to keep from scratch to the home area, which is backed up (scratch is NOT backed up.)

On Campus

You must be on-campus when transferring to or from Condor. This means using a wired office / cluster PC or being on the Eduroam / UoM WiFI network. If you are signed in to the GlobalProtect VPN (e.g., to read your email) then this will also work.

Off Campus

When off-campus you should run the GlobalProtect VPN software on your PC/laptop to appear on-campus. You will then be able to log in to Condor.

If you are unable to use GlobalProtect then please see our RDS-SSH service for an alternative.

Please note: Research IT cannot provide support for the GlobalProtect VPN. Please contact the main IT Services Helpdesk for assistance.

File Transfer Apps

If you have another file transfer program you like to use that we have not documented (see the sections in the menu on the left), you should still be able to use it using the following details:

• It must support secure file transfer using ssh. FileZilla, for example, can do this.
• You should connect to submitter.itservices.manchester.ac.uk on port 22
• Use your central IT username and password when asked to log in to Condor

We cannot provide support for all file-transfer programs but please see the examples in this section for more information.