{"id":22,"date":"2018-08-28T16:34:08","date_gmt":"2018-08-28T15:34:08","guid":{"rendered":"http:\/\/ri.itservices.manchester.ac.uk\/csf3\/?page_id=22"},"modified":"2025-05-22T15:54:40","modified_gmt":"2025-05-22T14:54:40","slug":"batch","status":"publish","type":"page","link":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/batch\/","title":{"rendered":"Running Jobs \u2013 The Batch System (SGE)"},"content":{"rendered":"<p><script type=\"text\/javascript\">\n    function toggle() {\n        var x = document.getElementById(\"hidetext\");\n        if (x.style.display === \"none\") {x.style.display = \"block\";}\n        else {x.style.display = \"none\";}\n    }\n<\/script><\/p>\n<div class=\"warning\">The SGE batch system has been shutdown and the CSF upgraded to use the Slurm batch system. Please read the <a href=\"\/csf3\/batch-slurm\">CSF3 Slurm documentation<\/a> instead.<\/p>\n<p>To display this old SGE page, <a href=\"javascript:toggle()\">click here<\/a>\n<\/div>\n<div id=\"hidetext\" style=\"display: none\">\n<h2>Why use a batch system?<\/h2>\n<p>All jobs must be run in the batch system (<acronym title=\"Son of Grid Engine\">SGE<\/acronym>). This allows you to specify the resources (cores, memory, GPUs) you need for your jobs and ensures the jobs only run when those resources become available.<\/p>\n<p>It also ensures fair usage of the system &#8211; there are many jobs making different demands of the system and many users submitting jobs. The batch system will schedule your jobs according to resources requested and size of your group&#8217;s contribution to the system.<\/p>\n<h3>Be kind to the login nodes and other users<\/h3>\n<p>Applications should <em>not<\/em> be run directly on the login nodes. These are relatively small, light-weight nodes (not many cores, small memory) used to access the system, edit files, submit jobs. Many users will be connected to the login nodes. If you run an application there, you may prevent all of those users from doing their work.<\/p>\n<div class=\"warning\">\nPlease do NOT run your application\/programs\/codes on the login nodes, or directly on compute nodes. You must submit your work to the batch system.<\/p>\n<p>The sysadmins will kill, without notice, any applications running on the login nodes, or jobs submitted to batch system which are oversubscribing resources by not setting parallel options properly.\n<\/p><\/div>\n<h3>Check the documentation of your application<\/h3>\n<p>Every parallel program\/code\/library has different methods for controlling the number of cores\/threads used by it. Please check the documentation so that you know how to run your app correctly. If using an app that is already installed on the CSF, see our <a href=\"\/csf3\/software\/a-z\/\">Application Documentation<\/a> &#8211; we provide example jobscripts.<\/p>\n<h3>Do not log in to compute nodes<\/h3>\n<p>If you want to diagnose \/ debug a problem with your application (e.g., quick test runs, trying different parameters, possibly modifying and recompiling code) without using a batch script, please use an <em>interactive<\/em> job (see <a href=\"\/csf3\/batch\/qrsh\/\">qrsh<\/a>). Using <code>ssh<\/code> to access a compute node is not permitted.<\/p>\n<p>Please take the time to learn how to submit jobs to the batch system.<\/p>\n<h2>Batch Tutorial<\/h2>\n<p>If you are unfamiliar with running jobs in a batch system please see our <a href=\"\/csf3\/getting-started\/tutorial\/\">10 minute tutorial<\/a> on running jobs on the CSF. <\/p>\n<h2>Submitting Jobs and Requesting Resources<\/h2>\n<p>You will need to write a small <em>jobscript<\/em>, <\/p>\n<pre>\r\ngedit myjobscript\r\n<\/pre>\n<p>which is a simple text file that specifies<\/p>\n<ol class=\"gaplist\">\n<li>Any <em>additional or specific<\/em> resources your job needs (number of CPU cores, the architecture\/type of CPU, memory, GPUs).<br \/>[The default is 1 CPU-core, any Intel CPU type, 4GB RAM, no GPU]<\/li>\n<li>The actual commands \/ application your job should execute.<\/li>\n<\/ol>\n<p>Further details on how to write jobscripts, and some example job scripts, are in the sections on <a href=\"\/csf3\/batch\/serial-jobs\">serial jobs<\/a> and <a href=\"\/csf3\/batch\/parallel-jobs\">parallel jobs<\/a>. The menu on the left also has pages for more advanced job options. Our <a href=\"\/csf3\/software\">software pages<\/a> also have example jobscripts for each application we have installed.<\/p>\n<p>Then submit the jobscript to the batch system using<\/p>\n<pre>\r\nqsub <em>myjobscript<\/em>\r\n<\/pre>\n<p>You may also wish to check on your job (is it still running?) using<\/p>\n<pre>\r\nqstat\r\n<\/pre>\n<p>See the <a href=\"\/csf3\/batch\/q-commands\/\">batch commands<\/a> for more information.\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>The SGE batch system has been shutdown and the CSF upgraded to use the Slurm batch system. Please read the CSF3 Slurm documentation instead. To display this old SGE page, click here Why use a batch system? All jobs must be run in the batch system (SGE). This allows you to specify the resources (cores, memory, GPUs) you need for your jobs and ensures the jobs only run when those resources become available. It also.. <a href=\"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/batch\/\">Read more &raquo;<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-22","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/22","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/comments?post=22"}],"version-history":[{"count":20,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/22\/revisions"}],"predecessor-version":[{"id":10075,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/22\/revisions\/10075"}],"wp:attachment":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/media?parent=22"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}