{"id":11203,"date":"2025-10-20T12:15:58","date_gmt":"2025-10-20T11:15:58","guid":{"rendered":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/?page_id=11203"},"modified":"2026-04-02T16:43:08","modified_gmt":"2026-04-02T15:43:08","slug":"ollama","status":"publish","type":"page","link":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/software\/applications\/ollama\/","title":{"rendered":"Ollama"},"content":{"rendered":"

Overview<\/h2>\n

OLLAMA is the easiest way to get up and running with large language models such as gpt-oss, Gemma 3, DeepSeek-R1, Qwen3 and more.
\nPlease note that it is for running LLM’s only, not for training them. You can however create customised models from existing models and even run those customised models.<\/p>\n

Restrictions on use<\/h2>\n

OLLAMA is open source and freely distributed under MIT License<\/a><\/strong>.<\/p>\n

Please note that the LLM’s license is different from that of ollama. Please check their respective license terms before using them.<\/p>\n

Set up procedure<\/h2>\n

To access the software you must first load the appropriate modulefile.<\/p>\n

\r\napps\/binapps\/ollama\/0.12.3\r\napps\/binapps\/ollama\/0.16.2\r\napps\/binapps\/ollama\/0.19.0\r\n<\/pre>\n
Location of downloaded LLM’s<\/h2>\n
When you try to run any model in Ollama, it will first download the model to the directory ~\/.ollama<\/code> in your home directory and then run it from there. This is the default location. Over time this directory can grow large as you run different LLM’s. It is therefore advised to changed the default location where Ollama stores the LLM’s to somewhere within your ~\/scratch<\/code> directory.<\/p>\n
The default location which Ollama will use for storing LLM’s can be controlled by the environment variable OLLAMA_MODELS<\/code><\/strong>.<\/p>\n
To change the default storage location, first create a directory inside your ~\/scratch<\/code> directory:<\/p>\n
mkdir ~\/scratch\/ollama_models<\/pre>\n
Next, set that path as OLLAMA_MODELS by adding the following line to your ~\/.bashrc<\/code> file:<\/p>\n
export OLLAMA_MODELS=\"~\/scratch\/ollama_models\"<\/pre>\n
Finally, re-login to CSF3 or source your edited ~\/.bashrc<\/code> file:<\/p>\n
source ~\/.bashrc<\/pre>\n
Interactive mode testing<\/h2>\n
Please do NOT<\/strong> run OLLAMA on the login node. Jobs should be submitted to the compute nodes via batch.
\nYou can test run OLLAMA in an interactive job\/session<\/a>. However, we strongly<\/strong> advise that you use batch jobs rather than interactive jobs. Use this for short duration testing only.<\/p>\n
\r\n# Start an interactive job.\r\n# Using long flag names\r\nsrun --partition=gpuL --gpus=1 --ntasks=1 --time=1-0 --pty bash\r\n\r\n# Or using short flag names\r\nsrun -p gpuL -G 1 -n 1 -t 1-0 --pty bash\r\n\r\n# Once resources are assigned for interactive job and you are logged in to an interactive node, run the following:\r\nmodule purge\r\nmodule load apps\/binapps\/ollama\/0.12.3\r\n\r\nunset ROCR_VISIBLE_DEVICES\r\n\r\nollama serve &                       # This will start the ollama server\r\n                                     # Uses port 11434 by default\r\nollama -v                            # Verify that ollama server is running\r\nollama run llama3.2                  # This  will download Llama 3.2 LLM and run it\r\nollama ps                            # This will list the running LLM\r\n\r\n# You will be able to interact with the running LLM at this stage.\r\nHello                                # Interact with LLM\r\n\/bye                                 # Exit interaction when done testing\r\nollama stop llama3.2                 # Stop the running LLM\r\nollama ps                            # Verify that the LLM has been stopped\r\nexit                                 # This will end the interactive job\/session and\r\n                                     # ollama server will be stopped.\r\n<\/pre>\n
GPU batch job submission<\/h2>\n
Single GPU job<\/h3>\n
Write a job submission script, for example:<\/p>\n
\r\n#!\/bin\/bash --login\r\n#SBATCH -p gpuL   # GPU partition. Available options for all: gpuL(L40s-48GB), gpuA(A100-80GB)\r\n                  # GPU partitions with restricted access: gpuA40GB\r\n#SBATCH -G 1      # (or --gpus=N) Number of GPUs \r\n#SBATCH -t 1-0    # Wallclock timelimit (1-0 is one day, 4-0 is max permitted)\r\n### Optional flags\r\n#SBATCH -n 1      # (or --ntasks=) Number of CPU (host) cores (default is 1)\r\n                  # up to 12 CPU cores per GPU is permitted for gpuL, gpuA and gpuA40GB\r\n                  # Also affects host RAM allocated to job unless --mem=num used.\r\n\r\n\r\n# Load the module\r\nmodule purge\r\nmodule load cuda\/12.6.2\r\nmodule load apps\/binapps\/ollama\/0.12.3\r\n\r\n# Without the following line Ollama will run in CPU instead of GPU\r\nunset ROCR_VISIBLE_DEVICES\r\n\r\n# Start ollama server process and run the desired LLM\r\nexport OLLAMA_HOST=0.0.0.0:11434        # This enables you to interact with the API remotely\r\nollama serve &\r\nollama run llama3.2\r\n\r\n# below lines are for graceful stopping of the LLM\r\n# adjust as per wallclock time you have set for the job\r\nsleep 23h                 # To delay the execution of the command in the next line\r\nollama stop llama3.2      # To stop Llama 3.2 model. However, this does not stops \r\n                          # the ollama server, which is stopped when the job ends\r\n<\/pre>\n
Submit the jobscript using: sbatch scriptname<\/code><\/p>\n
Interacting with the LLM’s running in CSF3 compute node<\/h2>\n
Once your desired LLM is running in a GPU compute node in CSF3, you will want to interact with it.
\nYou can use the different API’s provided by Ollama to interact with the LLM’s.
\nYou can either:<\/p>\n
    \n
  1. Interact from CSF3 login node itself<\/strong><\/li>\n
  2. Interact directly from your laptop<\/strong><\/li>\n<\/ol>\n
    Following are some examples to do that:<\/p>\n
    1. Interacting from CSF3 login node<\/h3>\n
    Once your job is running, check which node the job is running in using squeue<\/code> command.
    \nThen you can interact from the CSF3 login node by running:<\/p>\n
    \r\ncurl http:\/\/nodeNNN:11434\/api\/generate -d '{\r\n  \"model\": \"llama3.2\",\r\n  \"prompt\": \"Tell me a joke\"\r\n}'\r\n<\/pre>\n
    You will get responses in chunks instead of a single blob since the default behaviour is to respond in JSON chunks.
    \nIf you prefer answer in a single blob, which is easier to read run the following instead, which prevents streaming in chunks:<\/p>\n
    \r\ncurl http:\/\/nodeNNN:11434\/api\/generate -d '{\r\n  \"model\": \"llama3.2\",\r\n  \"prompt\": \"Tell me a joke\",\r\n  \"stream\": false\r\n}'\r\n<\/pre>\n
    2. Interacting from your laptop<\/h3>\n
    To interact with the LLM running in a compute node of CSF3, you will first need to set up a SSH Tunnel to the compute node of CSF3 from your laptop.
    \nYou can setup the SSH tunnel by running the following command from your laptop (terminal or windows command prompt):<\/p>\n
    \r\nssh -L 11434:nodeNNN:11434 @csf3.itservices.manchester.ac.uk\r\n<\/pre>\n
    After completing the authentication, you will be logged in to CSF3. Now it is important that you leave this terminal\/window aside.<\/p>\n
    Linux\/MAC Laptop example<\/strong>
    \nNext, open a new terminal in your Linux\/MAC laptop and then run the same commands as above, just change nodeNNN<\/code> to localhost<\/code> this time.<\/p>\n
    \r\ncurl http:\/\/localhost:11434\/api\/generate -d '{\r\n  \"model\": \"llama3.2\",\r\n  \"prompt\": \"Tell me a joke\",\r\n  \"stream\": false\r\n}'\r\n<\/pre>\n
    Windows Laptop example<\/strong>
    \nThe command syntax for curl<\/code> is different from that of Linux\/MAC. Open a new command prompt window in your Windows laptop and then run the following command.<\/p>\n
    \r\ncurl -X POST http:\/\/localhost:11434\/api\/generate -H \"Content-Type: application\/json\" ^\r\n     -d \"{\\\"model\\\":\\\"llama3.2\\\",\\\"prompt\\\":\\\"Tell me a joke\\\",\\\"stream\\\":false}\"\r\n<\/pre>\n
    Other API’s<\/H3>
    \nThe above example uses the GENERATE API which is fine for individual single-turn interactions.
    \nFor longer multi-turn conversations, which needs remembering the previous messages or the context, you can use the CHAT API.<\/p>\n
    Here’s and example:<\/p>\n
    \r\n#Linux\/MAC Example<\/strong>\r\ncurl http:\/\/localhost:11434\/api\/chat -d '{\r\n  \"model\": \"llama3.2\",\r\n  \"messages\": [\r\n    { \"role\": \"user\", \"Content\": \"Tell me a nerd joke\" }\r\n  ],\r\n  \"stream\": false\r\n}'\r\n\r\n\r\n#Windows Example<\/strong>\r\ncurl -X POST http:\/\/localhost:11434\/api\/chat -H \"Content-Type: application\/json\" ^\r\n     -d \"{\\\"model\\\":\\\"llama3.2\\\",\\\"messages\\\":[{\\\"role\\\":\\\"user\\\",\\\"content\\\":\\\"Tell me a nerd joke\\\"}],\\\"stream\\\":false}\"\r\n<\/pre>\n
    Further info<\/h2>\n