{"id":350,"date":"2014-06-02T07:29:36","date_gmt":"2014-06-02T07:29:36","guid":{"rendered":"http:\/\/www.rds.itservices.manchester.ac.uk\/?page_id=350"},"modified":"2025-11-05T10:34:12","modified_gmt":"2025-11-05T10:34:12","slug":"copying-files-between-cifs-and-nfs-shares","status":"publish","type":"page","link":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/user-faq\/copying-files-between-cifs-and-nfs-shares\/","title":{"rendered":"Copying Files between (CIFS and NFS) Shares"},"content":{"rendered":"<h2>Introduction<\/h2>\n<p>Reminder: In the following notes, the term &#8220;share&#8221; means a &#8220;storage area&#8221; &#8211; i.e., Research Data Storage that has been allocated to you.<\/p>\n<p>Research Data Storage is provided in two &#8220;flavours&#8221;: NFS <em>shares<\/em> are available <em>only<\/em> on Research IT run Linux systems, such as the compute platforms &#8211; the CSF and iCSF. CIFS(SMB) <em>shares<\/em> are not available on these systems but can be <em>mapped as network drives<\/em> on your desktop\/laptop.<\/p>\n<p>Some people have both types of storage. This page details how to transfer files between a CIFS(SMB) share and an NFS share.<\/p>\n<p>The information here also applies if you have additional Research Data Storage NFS shares (e.g., extra storage on the HPC platforms) and wish to move a large number of files efficiently between that and your CSF\/iCSF home directory. By using the &#8220;rds-ssh&#8221; server, you reduce the load on the compute system login nodes.<\/p>\n<h2>rds-ssh Overview<\/h2>\n<p>Both flavours of storage &#8211; CIFS(SMB) shares and Linux NFS shares &#8211; are available on a server named <em>rds-ssh<\/em>. This makes it easy to copy files from one flavour of storage to the other simply by logging in to the <em>rds-ssh<\/em> server and doing the file copy there.<\/p>\n<p>It is <em>not<\/em> necessary to map either share onto your desktop\/laptop to copy files from one share to another. In fact we discourage doing this. By doing the copy on the rds-ssh server (details below) you ensure the file transfer takes place entirely within the data-centre and is usually much faster than transfers over the campus network (or your home broadband) via your desktop\/laptop.<\/p>\n<h2>How To Copy Files Efficiently between CIFS(SMB) and NFS Shares<\/h2>\n<p>This is a summary of the procedure to efficiently copy files from one storage share to another &#8211; by using the rds-ssh server:<\/p>\n<ol class=\"gaplist\">\n<li>Obtain an account on the <a href=\"\/rds\/the-rds-ssh-service\/\">RDS-SSH service<\/a>, please contact us via our <a href=\"https:\/\/manchester.service-now.com\/esc?id=sc_cat_item&#038;sys_id=ab56f2c01b36861002ea404cd34bcb4e\">Connect Portal Form<\/a>, select \u201cProvide Access to Existing Storage\u201d from the drop-down menu in the form and indicate in the text that you require an RDS-SSH account.<\/li>\n<li>Log-in to the RDS-SSH service by using an SSH client (on MS Windows this could be Mobaxterm or PuTTY; on Linux\/MacOS use the command-line <code>ssh<\/code>)<\/li>\n<li>On the rds-ssh server, run the following command (to ensure all of your required CIFS(SMB) shares are available):\n<pre>mount-my-cifs<\/pre>\n<p>If anything is missing please contact us via our <a href=\"https:\/\/manchester.service-now.com\/esc?id=sc_cat_item&#038;sys_id=ab56f2c01b36861002ea404cd34bcb4e\">Connect Portal Form<\/a>. Note that your NFS storage will already be available, it is only the CIFS(SMB) storage that you have to <em>mount<\/em> because it requires <em>your<\/em> password to mount it &#8211; the sysadmins cannot bypass this security.<\/li>\n<li>On the rds-ssh server, copy files from one share to another \u2014 see the examples below.<\/li>\n<\/ol>\n<p>See below for more details on how to copy files between your storage areas once you&#8217;ve done the above steps. You can do the copy at the <a href=\"#cmdline\">Linux command-line<\/a> or using a <a href=\"#gui\">graphical file browser<\/a>.<\/p>\n<h2>How NOT To Do it!<\/h2>\n<p>When copying files from one RDS share to another, it is important to ensure the copy is taking place <em>on the RDS-SSH server<\/em> and <em>not via your local desktop\/laptop<\/em>.<\/p>\n<p>At <strong>no<\/strong> point should you download the files to your desktop\/laptop from one share then upload it to another! This can be terribly slow because you usually have only a slow network connection to your desktop\/laptop (and you actually transfer the files twice over that slow network &#8211; download then upload).<\/p>\n<p>If you have <em>mapped-as-a-drive<\/em> one or both shares to your desktop, it is still quicker to log in to the rds-ssh server and do the copy there. Doing the copy on your desktop\/laptop means the files go over your local network (e.g., the campus network or your home broadband.) This could be slow. By logging in to the rds-ssh server, the files only ever go over the fast data-centre network.<\/p>\n<p><a name=\"cmdline\"><\/a><\/p>\n<h2>How TO do it method 1 \u2014 Using the rds-ssh server&#8217;s command-line<\/h2>\n<p>Here we log in to the rds-ssh server (using any ssh tool) and do the file copying on the rds-ssh server&#8217;s command-line:<\/p>\n<pre>ssh <em>username<\/em>@rds-ssh.itservices.manchester.ac.uk\r\n         #\r\n         # Use your own central username (and password, then DUO, when prompted)\r\n<\/pre>\n<p>We are now at the rds-ssh home directory. If you have a CSF\/iCSF account this will be the same home directory used on those machines.<\/p>\n<p>Your other RDS shares will be available in the <code>\/mnt\/<\/code> directory. The RI Team will have given you the exact path names.<\/p>\n<p>Once you&#8217;ve logged in to rds-ssh, if your have <strong>CIFS(SMB)<\/strong> storage shares, run:<\/p>\n<pre>[<em>username<\/em>@gorg ~]$  mount-my-cifs\r\n                         #\r\n                         # If asked, enter <em>your<\/em> UoM password to make the CIFS(SMB) storage available.\r\n                         # The Research IT sysadmins cannot do this for you.\r\n<\/pre>\n<ul>\n<li>Note that the <code>rds-ssh<\/code> server is actually made up of two servers named <code>gorg<\/code> and <code>zola<\/code>. So you may see <code>zola<\/code> mentioned in the prompt instead:\n<pre>[<em>username<\/em>@<strong>zola<\/strong> ~]$  mount-my-cifs\r\n<\/pre>\n<p>If you log in to <code>rds-ssh.itservices.manchester.ac.uk<\/code> you will land on one of the above servers. Running <code>mount-my-cifs<\/code> after logging in will ensure your storage has been mounted on the particular server you have landed on.<\/li>\n<\/ul>\n<h3>Copying files using the cp command<\/h3>\n<p>Here we show an example of copying from one of our CIFS shares to our CSF home directory:<\/p>\n<pre># Ensure we are at the rds-ssh home directory (same as our CSF\/iCSF home dir)\r\ncd ~\r\n\r\n# Copy all .dat files from our mounted CIFS share to a CSF directory (sampledata)\r\n# <strong>Research IT will tell you the correct <em>path<\/em> for your storage share<\/strong>\r\ncp \/mnt\/cifs-<em>fac<\/em>01\/mxyzabc1-ProjectName\/*.dat  ~\/sampledata\/\r\n\r\n# Copy an entire sub-directory from our CIFS share to <em>another<\/em> of our NFS shares\r\n# <strong>Research IT will tell you the correct <em>path<\/em> for your storage shares<\/strong>\r\ncp -r \/mnt\/cifs-<em>fac<\/em>01\/mxyzabc1-ProjectName\/results\/  \/mnt\/fac01-data01\/mxyzabc1\/\r\n<\/pre>\n<p>The main thing to notice here is that your CIFS(SMB) shares will be available via a directory name similar to <code>\/mnt\/cifs-<em>fac<\/em>01\/<em>username<\/em>-<em>ProjectName<\/em><\/code> (or something similar), your other NFS shares will be in the <code>\/mnt\/<\/code> directory and your CSF\/iCSF home directory (if you have an account on the central compute platforms) is also your <em>home directory<\/em> on rds-ssh.<\/p>\n<p>The usual Linux command-line file management commands, e.g <code>rsync<\/code>, can all be used.<\/p>\n<h3>Copying files using the rsync command<\/h3>\n<p>Here we show an example of using <code>rsync<\/code> from one of our CIFS(SMB) shares to our CSF home directory:<\/p>\n<pre># Ensure we are at the rds-ssh home directory (same as our CSF\/iCSF home dir)\r\ncd ~\r\n\r\n# Copy a directory from our mounted CIFS share to a CSF directory (sampledata)\r\n# <strong>Research IT will tell you the correct <em>path<\/em> for your storage share<\/strong>\r\nrsync -av \/mnt\/cifs-<em>fac<\/em>01\/mxyzabc1-ProjectName\/mydatafiles  ~\/sampledata\r\n                                                          #\r\n                                                          # <strong>NO<\/strong> \/ here. This is important\r\n                                                          # Using a \/ here changes the\r\n                                                          # behaviour of rsync!\r\n<\/pre>\n<p>Note that the <code>rsync<\/code> command is fussy about <code>\/<\/code> characters at the end of directory names. If you put a <code>\/<\/code> at the end of the <em>source<\/em> directory name, <code>rsync<\/code> will copy the files from inside the directory but won\u2019t create an actual directory in your CSF home.<\/p>\n<pre>rsync -av dirA ~\/targetDir\r\n              #\r\n              # <strong>No<\/strong> \/ here.\r\n              # The result of this command is\r\n              # ~\/targetDir\/<strong>dirA<\/strong>\/<em>all files from dirA<\/em>\r\n\r\nrsync -av dirA\/ ~\/targetDir\r\n              #\r\n              # \/ here.\r\n              # The result of this command is\r\n              # ~\/targetDir\/<em>all files from dirA<\/em>\r\n<\/pre>\n<p><a name=\"gui\"><\/a><\/p>\n<h2 id=\"xfe\">How TO do it method 2 \u2014 Using the rds-ssh server&#8217;s GUI app<\/h2>\n<p>A graphical file manager named <code>xfe<\/code> is installed on the rds-ssh server if you prefer not to use command-line tools. You must have an X11 server running on your desktop machine to use this (on Windows we recommend using MobaXTerm to log in to the rds-ssh server because it provides X11 for you without any extra setup).<\/p>\n<p>Log in to rds-ssh as follows (from MobaXTerm or linux\/MacOS command-line):<\/p>\n<pre>ssh -X <em>username<\/em>@rds-ssh.itservices.manchester.ac.uk\r\n     #   #\r\n     #   # Use your own central username (and password when prompted)\r\n     #\r\n     # Mac users: Please try -Y instead if you have problems with the GUI.\r\n<\/pre>\n<p>Once you&#8217;ve logged in to rds-ssh, if your have <strong>CIFS(SMB)<\/strong> storage shares, run:<\/p>\n<pre>[<em>username<\/em>@gorg ~]$  mount-my-cifs\r\n                         #\r\n                         # If asked, enter <em>your<\/em> UoM password to make the CIFS(SMB) storage available.\r\n                         # The Research IT sysadmins cannot do this for you.\r\n<\/pre>\n<ul>\n<li>Note that the <code>rds-ssh<\/code> server is actually made up of two servers named <code>gorg<\/code> and <code>zola<\/code>. So you may see <code>zola<\/code> mentioned in the prompt instead:\n<pre>[<em>username<\/em>@<strong>zola<\/strong> ~]$  mount-my-cifs\r\n<\/pre>\n<p>If you log in to <code>rds-ssh.itservices.manchester.ac.uk<\/code> you will land on one of the above servers. Running <code>mount-my-cifs<\/code> after logging in will ensure your storage has been mounted on the particular server you have landed on.<\/li>\n<\/ul>\n<p>Now start the file manager:<\/p>\n<pre>xfe\r\n<\/pre>\n<p>When the GUI pops up, press <code>CTRL<\/code>+<code>F3<\/code> to open two panels (both initially showing the same directory) to make dragging and dropping files between directories easier. You can also open two panels from the menus via <em>View -&gt; Two Panels<\/em>.<\/p>\n<p>Initially we are at the rds-ssh home directory. If you have a CSF\/iCSF account this will be the same home directory used on those machines.<\/p>\n<p>Your other RDS shares will be available in the <code>\/mnt\/<\/code> directory. The RI Team will have given you the exact path names. Navigate to that directory in the GUI and find your shares &#8211; CIFS shares will usually be named similar to <code>\/mnt\/cifs-<em>fac<\/em>01\/<em>username<\/em>-<em>ProjectName<\/em><\/code> (or something similar), and your CSF\/iCSF home directory is also your home directory on rds-ssh.<\/p>\n<p>The image shows the xfe browser (opened on rds-ssh from a Windows desktop). Click on the image for a large version.<br \/>\n<a href=\"http:\/\/www.rds.itservices.manchester.ac.uk\/wp-content\/uploads\/xfe-in-windows-red.png\"><img decoding=\"async\" class=\"alignnone size-full wp-image-459\" src=\"http:\/\/www.rds.itservices.manchester.ac.uk\/wp-content\/uploads\/xfe-in-windows-red.png\" alt=\"XFE file browser\" width=\"700\" srcset=\"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-content\/uploads\/xfe-in-windows-red.png 1002w, https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-content\/uploads\/xfe-in-windows-red-300x160.png 300w\" sizes=\"(max-width: 1002px) 100vw, 1002px\" \/><\/a><br \/>\nYou can now drag and drop files between the two storage areas.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction Reminder: In the following notes, the term &#8220;share&#8221; means a &#8220;storage area&#8221; &#8211; i.e., Research Data Storage that has been allocated to you. Research Data Storage is provided in two &#8220;flavours&#8221;: NFS shares are available only on Research IT run Linux systems, such as the compute platforms &#8211; the CSF and iCSF. CIFS(SMB) shares are not available on these systems but can be mapped as network drives on your desktop\/laptop. Some people have both.. <a href=\"https:\/\/ri.itservices.manchester.ac.uk\/rds\/user-faq\/copying-files-between-cifs-and-nfs-shares\/\">Read more &raquo;<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":8,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-350","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/pages\/350","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/comments?post=350"}],"version-history":[{"count":22,"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/pages\/350\/revisions"}],"predecessor-version":[{"id":1144,"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/pages\/350\/revisions\/1144"}],"up":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/pages\/8"}],"wp:attachment":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/rds\/wp-json\/wp\/v2\/media?parent=350"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}