{"id":9199,"date":"2025-04-01T16:33:19","date_gmt":"2025-04-01T15:33:19","guid":{"rendered":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/?page_id=9199"},"modified":"2026-04-16T11:18:32","modified_gmt":"2026-04-16T10:18:32","slug":"jupyter-notebook-slurm","status":"publish","type":"page","link":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/software\/applications\/jupyter-notebook-slurm\/","title":{"rendered":"Jupyter Notebook (slurm)"},"content":{"rendered":"

Overview<\/h2>\n

Jupyter Notebook<\/a> is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text.<\/p>\n

Please read carefully the instructions below when connecting a web-browser to your jupyter notebook on the CSF (slurm) – there are some extra steps needed that you would not normally do if you were running a jupyter notebook on your local desktop\/laptop (or the x2go virtual desktop<\/a> if using that).<\/p>\n

It is possible to run jupyter notebook on the Nvidia GPU nodes. However, you need to request being added to the relevant group to access GPUs<\/a> before you can run Jupyter Notebook on them.<\/strong><\/p>\n

Restrictions on use<\/h2>\n

There are no restrictions on accessing this software.<\/p>\n

Set up procedure<\/h2>\n

To access the software you must first load the modulefile:<\/p>\n

\r\nmodule load apps\/binapps\/jupyter-notebook\/any    # loads Anaconda3 2024.10 by default \r\n<\/pre>\n
If you’ve not already loaded a specific anaconda python modulefile, the above “any” modulefile will automatically load the latest Anaconda3 Python modulefile for you (jupyter is provided by Anaconda Python). If you wish to use a specific version of Anaconda python, you<\/em> should load its modulefile first.<\/p>\n
Initial Setup<\/h2>\n
No initial setup is needed to run jupyter notebooks.<\/p>\n
It is possible, if you wish, to password protect your notebooks, which can be done using the commands below. But this is optional. By default a jupyter notebook will present you with a login page when you fist visit the notebook with your web-browser. You can setup a password via that<\/em> webpage. You can also use the token<\/em>-based security which the notebook uses if you don’t set up a password. This requires you to enter a random string of letters and numbers, which are reported in the job output .out<\/code> file for the job running the notebook.<\/p>\n
Hence only you<\/em> can access your notebooks when they are running on the CSF, even if you don’t set up a notebook password.<\/p>\n
Our recommendation is to use the default token-based authentication and not bother setting up a password for the notebooks.<\/strong><\/p>\n
Optional Setup<\/h3>\n
After loading the Jupyter Notebook modulefile, run the following on the login node:<\/p>\n
jupyter-notebook --generate-config\r\n\r\n# It will output:\r\nWriting default config to: \/mnt\/iusers01\/xy01<\/em>\/mabcxyz1<\/em>\/.jupyter\/jupyter_notebook_config.py\r\n   #\r\n   # The xy01<\/em> group and mabcxyz1<\/em> username will differ\r\n<\/pre>\n
The command has reported where it has generated the configuration file (within your home area). The file is at (where ~<\/code> means your home<\/em> directory):<\/p>\n
~\/.jupyter\/jupyter_notebook_config.py\r\n<\/pre>\n
\nOptional Password Protection (skip this) [click to expand]<\/summary>\n
Password protecting the access to your running notebooks is optional. If you do not<\/em> set a password then the notebooks are still only accessible by you – a notebook’s webpage will ask for a token<\/em> (a random set of letters and numbers) to be entered, which only you can get from the job output slurm-NNNNN<\/em>.out<\/code> file (see below). It is very unlikely another user would be able to guess this token!<\/p>\n
In fact we recommend that you skip password protection<\/strong> and use the token method instead – i.e., you don’t need to do anything here and can jump straight to running the notebook<\/a>.<\/p>\n
If you do wish to create a password to protect access to your notebooks when the server is running, do NOT use your central IT password<\/strong>. This is not a terribly secure method of protecting your notebooks but will offer some protection. Please do NOT use a password that is used on other systems or is valuable to you.<\/p>\n
Run the following on the login node:<\/p>\n
\r\npython -c \"from jupyter_server.auth import passwd; print(passwd())\"\r\n\r\n# It will display:\r\nEnter password: Enter your new notebook password and press [return]<\/strong>\r\nVerify password: Enter the same password again and press [return]<\/strong>\r\nargon2:$argon2id$v=19$m=10240,t=10,p=8$X4CyNDaKJw2rZhCQatsjbw$ZDUmJx5hfcUNsmGOBxNn3EfYa1SL3hzvt+2ZdU9rQDQ<\/em>\r\n  #\r\n  # This line is the result (yours will be different).\r\n  # We need to copy this in to the config file generated earlier.\r\n<\/pre>\n
Take a copy of the above line. Now edit the config file generated earlier:<\/p>\n
xnedit ~\/.jupyter\/jupyter_notebook_config.py\r\n  #\r\n  # Could also use nano or vi or emacs editors\r\n<\/pre>\n
Search the file for a line that looks like:<\/p>\n
# c.ServerApp.password = ''\r\n<\/pre>\n
and change it to be:<\/p>\n
c.ServerApp.password = 'argon2:$argon2id$v=19$m=10240,t=10,p=8$X4CyNDaKJw2rZhCQatsjbw$ZDUmJx5hfcUNsmGOBxNn3EfYa1SL3hzvt+2ZdU9rQDQ<\/em>'\r\n#\r\n# Notice we have remove the '#' at the start\r\n<\/pre>\n
where the argon2:....<\/code> is copied from the output generated earlier.<\/p>\n
Now search the config file for a line that looks like:<\/p>\n
# c.ServerApp.ip = 'localhost'\r\n<\/pre>\n
Change this to be:<\/p>\n
c.ServerApp.ip = '*'\r\n#\r\n# Notice we have remove the '#' at the start\r\n<\/pre>\n
When you connect to a jupyter notebook server with a web-browser you will be asked for the password you specified earlier. Note that this does NOT use https<\/code> (at the moment). So you should NOT use a password that is of value to you.<\/p>\n
That is all of the setup we need to do. You can now start a jupyter-notebook job as described below.
\n<\/details>\n
We recommend that you skip password protection<\/strong> and use the token method instead – i.e., you don’t need to do anything here and can jump straight to running the notebook<\/a>.<\/p>\n
<\/a><\/p>\n
Running the application<\/h2>\n
Your jupyter notebook will run in the batch system on a compute node. The basic procedure is as follows:<\/p>\n
    \n
  1. Submit a jupyter notebook job on the CSF and wait for it to run. We do this using a helper script named jupyter-notebook-csf<\/code> run on the login node – no need to write a jobscript yourself! You can also use the command: jupyter-notebook-csf-slurm<\/code> (it is the same script as jupyter-notebook-csf<\/code>)<\/li>\n
  2. On your local desktop\/laptop (or the x2go virtual desktop<\/a> if using that) create an SSH tunnel to the CSF compute node running your jupyter notebook.<\/li>\n
  3. On your local desktop\/laptop (or the x2go virtual desktop<\/a> if using that) connect a web-browser to the SSH tunnel – it will then connect to the CSF job.<\/li>\n<\/ol>\n
    Using an SSH tunnel may seem like a complicated method but it is the only way to reach the CSF compute node where your jupyter job is running. It also improves security – any traffic between your web-browser and the jupyter server will be encrypted.<\/p>\n
    The above procedure is now shown in detail below.<\/p>\n
    jupyter-notebook-csf<\/code> helper script<\/h3>\n
    Note that you can also use the command: jupyter-notebook-csf-slurm<\/code> (it is the same script as jupyter-notebook-csf<\/code>).<\/p>\n
    The jupyter-notebook-csf<\/code> helper script mentioned above (used to submit a batch job from the login node) accepts the following flags:<\/p>\n
    \r\njupyter-notebook-csf [-p PARTITION] [-n NUMCORES] [-t TIMELIMIT] [-m 32|46|62|125] [-g NUMGPUS]   [-j] [-k] [-P PORTNUM] [-c JOBID] [-T JOBID]\r\n\r\nStarts a Jupyter Notebook server on a compute node.\r\n\r\n-p PARTITION -- Run in PARTITION. Default is serial partition.\r\n                -----------------------------------------------------------------------------------------\r\n                PARTITION NAME  - DESCRIPTION\r\n                -----------------------------------------------------------------------------------------\r\n                multicore       - Run on the AMD nodes, 2-168 cores\r\n                multicore_small - Run on the Intel nodes, 2-32 cores\r\n                serial          - Intel nodes only, will run using 1 core\r\n                himem           - Run on high memory nodes! 1-32 cores, -m MUST BE SPECIFIED (See below)\r\n                vhimem          - RESTRICTED ACCESS Run on very high memory node (126GB\/core) 1-32cores, -m must be specified\r\n                gpuV            - !!! NO LONGER AVAILABLE !!!\r\n                gpuA            - Run on the Nvidia A100 80GB GPU nodes IF YOU HAVE ACCESS TO SUCH NODES! 1-48 cores\r\n                gpuA40GB        - Run on the Nvidia A100 40GB GPU nodes IF YOU HAVE ACCESS TO SUCH NODES! 1-48 cores\r\n                gpuL            - Run on the Nvidia L40s GPU nodes IF YOU HAVE ACCESS TO SUCH NODES! 1-48 cores cores\r\n                -----------------------------------------------------------------------------------------------------\r\n-n NUMCORES  -- Request NUMCORES cores. Default is 1, max 168.\r\n-t TIMELIMIT -- Time MUST be specified (default 1 day, max 7 days).\r\n                Acceptable time formats include 'minutes', 'minutes:seconds', 'hours:minutes:seconds', 'days-hours', 'days-hours:minutes'\r\n-m 32        -- Run on a 32GB\/core (512GB) node (max 16 cores).\r\n-m 46        -- Run on a 46GB\/core (1500GB or 1.5TB) node (max 32 cores).\r\n-m 62        -- Run on a 62GB\/core (2000GB or 2TB) node (max 32 cores).\r\n-m 125       -- run on a 125GB\/core (4000GB or 4TB) node (max 32 cores). [REQUEST ACCESS BEFORE USING]\r\n                If no -m flag given, the system will choose an ordinary compute node for you.\r\n                If no '-p PARTITION' flag given, will default to the 'himem' or 'vhimem' partition.\r\n                NOTE: you cannot use the -m flag if running on GPU nodes (-g flag).\r\n-j           -- Generate the jobscript but DO NOT submit it. Debugging only.\r\n-k           -- Keep the jobscript file after it has been submitted. Deleted by default.\r\n                The autogenerated jobscript is not very useful to you.\r\n-P PORTNUM   -- Port number from which to start searching for a free port for the\r\n                the notebook server to listen on. Default starting port is 8888.\r\n-g NUMGPUS   -- Run on the Nvidia GPU nodes IF YOU HAVE ACCESS TO SUCH NODES!\r\n                Depending on your level of access you can request 1 -- 4 GPUs\r\n                The '-p PARTITION' flags are required to specify the type of GPU (see above.)\r\n                NOTE: you cannot use a GPU flag if running on himem nodes (-m flag).\r\n\r\nTo check whether a jupyter notebook job is running:\r\n\r\n-c JOBID     -- report squeue status of job\r\n\r\nOnce a job is running, get info about accessing the notebook via a web-browser:\r\n\r\n-T JOBID     -- read the job slurm.out file and report the token used for authentication.\r\n                You can only do this AFTER the jupyter notebook batch job has started.\r\n                This is to help with logging in to the notebook in a web-browser.\r\n\r\n-h           -- Display this help.\r\n\r\nCheck the slurm.out file for instructions on how to connect\r\na web-browser to the server once the job runs.\r\n<\/pre>\n
    We now go through the complete steps to using a Jupyter Notebook.<\/p>\n
    Complete Steps to Running a Jupyter Notebook<\/h3>\n
      \n
    1. Submit a jupyter notebook job using the helper script run on the CSF login node:\n
      module load apps\/binapps\/jupyter-notebook\/any\r\n\r\n# Run one<\/em> of the following examples on the login node:\r\n\r\njupyter-notebook-csf                               # Submit 1-core serial job w\/ 1 hour timelimt \r\njupyter-notebook-csf -p multicore -n 4 -t 0-6      # Submit a parallel 4-core job using the AMD nodes w\/ 6 hour timelimit \r\njupyter-notebook-csf -p highmem -n 4 -m 32 -t 0-6  # As above but requests 32GB\/core RAM, runs on Intel himem node\r\njupyter-notebook-csf -p gpuA -n 12 -g 1 -t 0-6     # 12 core parallel job using 1xA100 GPU\r\n<\/pre>\n
      The output from the helper script can be one of two things:<\/p>\n
        \n
      1. If the job runs straight away in the batch system you’ll see:\n
        \r\nValid time format: 6:00:00\r\n4 cores selected\r\nSubmitted batch job 1518\r\nChecking if job has already started...\r\n..\r\nPLEASE READ THE FOLLOWING:\r\nStarting jupyter-notebook on node1260 port 8888\r\n\r\nYou must now ssh in to the CSF3 from your local machine and tunnel\r\nto the backend compute node. Use the following command on *your*\r\ncomputer (not on the CSF3):\r\n\r\n  ssh -L 8888:node1260<\/em>:8888 username<\/em>@csf3.itservices.manchester.ac.uk\r\n\r\nYou should then start a web-browser on *your* computer and browse to:\r\n\r\n  http:\/\/localhost:8888\r\n\r\nIf you are asked for a token to login to the notebook, have a look in\r\nthis job's .out file for the token by running the following command:\r\n\r\n  jupyter-notebook-csf -T 1518\r\n\r\n(you should only run the above command *after* the batch job has started)\r\n\r\nIf you press the 'Quit' button in your notebook's web-page it will terminate the batch job.\r\nIf you press the 'Logout' button in your notebook's web-page or simply close the web-browser\r\nthen you *must* run the following command on the CSF3 to terminate the Jupyter batch job:\r\n\r\n  scancel 1518\r\n\r\nDon't forget to log out of your tunnelled ssh session.\r\n<\/pre>\n
        This means that the jupyter-notebook job started immediately on a compute node so you are able to proceed with setting up the SSH tunnel from your local computer (or the x2go virtual desktop<\/a> if using that) to the CSF and then connecting your web-browser to it by following the instructions in the above message (see below for an example).<\/li>\n
      2. If the job doesn’t run immediately but waits in the batch queue:\n
        \r\nValid time format: 6:00:00\r\nSubmitted batch job 1528\r\nChecking if job has already started...\r\n....................\r\n\r\nThe job is probably still waiting to run. Here is your squeue output....\r\n\r\n          JOBID PRIORITY  PARTITION NAME            USER     ST SUBMIT_TIME    START_TIME     TIME        NODES  CPUS NODELIST(REASON)\r\n           1528 0.0000002 gpuA      jnotebook.25040 username PD 02\/04\/25 17:11 02\/04\/25 17:11             1      12 (Priority)\r\n\r\nYou *cannot* connect a web-browser to the CSF3 until the job runs.\r\nPlease be patient.\r\n\r\nTo check your job's status, run:\r\n\r\n   jupyter-notebook-csf -c 1528\r\n\r\n   (or simply run 'squeue')\r\n\r\nWhen the job eventually runs you should then run the following command\r\nto get further instructions on how to connect your web-browser to the\r\nJupyter notebook:\r\n\r\n   cat slurm-1528.out\r\n\r\nFollow the instructions given in that file.\r\n\r\n<\/pre>\n
        Note that the job-ID 1528<\/code><\/strong> will be different for your job. The above message means that you must check on your job using the squeue<\/code> command to see when it is running. When it is running you can proceed with the instructions given in the file: slurm-NNNNN.outNNNNN<\/em><\/code> where NNNNN<\/em> is the job-ID of your own job.<\/li>\n<\/ol>\n<\/li>\n
      3. Once the notebook job is running you can now set up the SSH tunnel on your<\/em> computer (not the CSF). You’ll need to open another terminal window<\/strong> so that you have one that is NOT logged in to the CSF<\/strong>.For example, if you are on Windows use MobaXterm and press the new tab icon to open a Local Terminal (see image below), if on Mac use the Terminal<\/em> application and if on linux use an Xterm or GNOME Terminal.
        \n        \"MobaXterm<\/a>Then run:<\/p>\n
        # Run this on your<\/em> computer (PC\/laptop), NOT in a terminal logged in to the CSF<\/strong>\r\n# The terminal window you run this in (e.g., MobaXterm) must NOT be logged in to the CSF.\r\nssh -L 8888<\/strong>:node780<\/strong>:8888<\/strong> username<\/em>@csf3.itservices.manchester.ac.uk\r\n         ^     ^      ^      ^\r\n         |     |      |      |\r\n         |     |      |      +----- use your own central IT username here\r\n         +-----+------+\r\n               |\r\n               |\r\n               +---------- These values (port numbers and compute node name) are reported in\r\n                           the above output when you submitted the notebook job. You should\r\n                           use the values appropriate to your job (the port will likely be\r\n                           8888 but could be different. The node name will very likely be\r\n                           a different node number).\r\n                           Note also, some high-memory nodes as named h<\/strong>nodeNNN<\/em>.\r\n\r\n<\/pre>\n<\/li>\n
      4. Now start a web-browser on your PC<\/em> and browse to:\n
        http:\/\/localhost:8888<\/strong>\r\n                   #\r\n                   # Use the same port number as used above\r\n<\/pre>\n<\/li>\n
      5. You should be taken to the Jupyter Notebook login page. If you created a password earlier (most people don’t) then enter it in the webpage to access the jupyter notebook webpage. If you did not<\/em> set up a password, the notebook’s webpage will indicate that token authentication is in use. You will need to get the notebook’s token<\/em> (see below).<\/li>\n<\/ol>\n
        You can now use the Jupyter Notebook through your own web-browser. Any code you enter is executed on the CSF compute node on which the Jupyter Notebook server is running. Hence you should ensure your code uses the correct number of cores you reserved when you submitted the notebook job.<\/p>\n
        Obtaining the Notebook Token<\/h3>\n
        When you visit your notebook in a web-browser you may be asked for a token<\/em> if you didn’t set up a password. The token is similar to a random auto-generated password.<\/p>\n
        To find out what the token is for your notebook run the following command on the login node when your notebook is running and after<\/strong> you have pointed a web-browser at the notebook page (you’ll see the login page):<\/p>\n
        # Run this on the CSF login node, in the folder where you submitted the jupyter job from\r\njupyter-notebook-csf -T JOBID<\/em>\r\n<\/pre>\n
        where JOBID<\/em> is the job id of your notebook job (run squeue<\/code> if you are unsure). You should see lines similar to the following:<\/p>\n
        \r\nAuthentication token for your notebook is 3827b48cc078578c5b50232f7835e6cb809c128a25a74294<\/strong>\r\nType it in to the webpage at: http:\/\/localhost:8888\/  (if asked for a token)\r\nOR browse to http:\/\/localhost:8888\/?token=3827b48cc078578c5b50232f7835e6cb809c128a25a74294<\/strong>\r\n<\/pre>\n
        Option 1:<\/strong> You can either browse directly to the notebook by appending the ?token=...<\/code> text to the localhost<\/em> URL we used above:<\/p>\n
        \r\n# Go directly to your notebook\r\nhttp:\/\/localhost:8888\/?token=3827b48cc078578c5b50232f7835e6cb809c128a25a74294<\/em><\/strong>\r\n                   #                    #\r\n                   #                    # Use the token shown in your<\/em> output above, not this example!\r\n                   #                                   \r\n                   # Use the same port number as used above\r\n<\/pre>\n
        Option 2:<\/strong> or copy the token part of the address:<\/p>\n
        \r\n# Type this into to the box at the top of the page at http:\/\/localhost:8888\r\n3827b48cc078578c5b50232f7835e6cb809c128a25a74294<\/strong>\r\n             #\r\n             # Use the token shown in your<\/em> output above, not this example!\r\n<\/pre>\n
        in to the type-in box in the notebook’s login web-page – use the box at the top of the screen, as show below:<\/p>\n
        \"Jupyter<\/a><\/p>\n
        Example Notebook<\/h3>\n
        Once you have connected to your notebook, try the following in your web-browser:<\/p>\n
          \n
        1. Select New -- Python3<\/code> to start a new python notebook.<\/li>\n
        2. Enter the following code in the first cell<\/em> (see image below):\n
          import os\r\nfile=os.getenv(\"JUPYTER_HOME\")+\"\/example_job\/mandelbrot.py\"\r\n%load $file\r\n<\/pre>\n
          \"Load<\/a><\/li>\n
        3. Press the Run<\/code> button twice<\/em> – firstly to load the mandelbrot.py<\/code> in to the notebook. Then secondly to execute the notebook code.<\/li>\n
        4. You should see a Mandelbrot fractal image in your web-browser. The code was executed in the batch job running on the compute node!
          \n          \"Mandelbrot<\/a><\/li>\n<\/ol>\n
          Stopping a Notebook<\/h3>\n
          When you have finished with the jupyter notebook you can either:<\/p>\n