{"id":1302,"date":"2014-02-07T14:18:28","date_gmt":"2014-02-07T14:18:28","guid":{"rendered":"http:\/\/ri.itservices.manchester.ac.uk\/csf-apps\/?page_id=1302"},"modified":"2019-01-28T15:52:08","modified_gmt":"2019-01-28T15:52:08","slug":"anaconda-python","status":"publish","type":"page","link":"https:\/\/ri.itservices.manchester.ac.uk\/csf-apps\/software\/applications\/anaconda-python\/","title":{"rendered":"Anaconda Scientific Python Distribtuion"},"content":{"rendered":"

Overview<\/h2>\n

Anaconda is a completely free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing general-purpose statistical software package. It has 100+ of the most popular Python packages for science, math, engineering, data analysis.<\/p>\n

Versions 2.3.0, 2.1.0, 1.9.1 and 1.8.0 of Anaconda are installed on the CSF. These provide python 2.7.x.<\/p>\n

Version 5.1.0, 4.2.0, 4.1.1, and 2.3.0 of Anaconda3 is also installed. They provide Python 3.6.x, 3.5.x, and 3.4.x.<\/p>\n

To see what is available in these versions load the appropriate modulefile (see below) and then run the command:<\/p>\n

\r\nconda list\r\n<\/pre>\n
Note: none of the python 2.6 or python 3.3 packages have been installed.<\/p>\n
Restrictions on Use<\/h2>\n
None, but all users should read the End User License Agreement<\/a> before using the software.<\/p>\n
Set up procedure<\/h2>\n
Load one<\/em> of the following modulefiles for the version you wish to use:<\/p>\n
For Python 2.7.x<\/h3>\n
\r\nmodule load apps\/binapps\/anaconda\/2.5.0\r\nmodule load apps\/binapps\/anaconda\/2.3.0\r\nmodule load apps\/binapps\/anaconda\/2.1.0\r\nmodule load apps\/binapps\/anaconda\/1.9.1\r\nmodule load apps\/binapps\/anaconda\/1.8.0\r\n<\/pre>\n
For Python 3.4<\/h3>\n
\r\n# Notice the extra \/3\/ in the modulefile name\r\n\r\nmodule load apps\/binapps\/anaconda\/3\/2.3.0\r\n<\/pre>\n
For Python 3.5<\/h3>\n
\r\n# Notice the extra \/3\/ in the modulefile name\r\n\r\nmodule load apps\/binapps\/anaconda\/3\/4.2.0\r\nmodule load apps\/binapps\/anaconda\/3\/4.1.1\r\n<\/pre>\n
For Python 3.6<\/h3>\n
\r\n# Notice the extra \/3\/ in the modulefile name\r\n\r\nmodule load apps\/binapps\/anaconda\/3\/5.1.0\r\n<\/pre>\n
Running the application<\/h2>\n
After loading the modulefile you need to have a batch job submission script. Below are some examples.<\/p>\n
Simple, Python-only job<\/strong><\/p>\n
Here is a simple Python script (download fib.py<\/a>)<\/p>\n
\r\nparents, babies = (1, 1)\r\nwhile babies < 100:\r\n    print 'This generation has %d babies' % babies\r\n    parents, babies = (babies, parents + babies)\r\n<\/pre>\n
This is an example SGE submission script (download fib.sge<\/a>)<\/p>\n
\r\n#!\/bin\/bash\r\n#$ -S \/bin\/bash\r\n#$ -N Python_fib\r\n#$ -cwd\r\n#$ -o outputfile.log\r\n#$ -j y\r\n#$ -V\r\npython fib.py\r\n<\/pre>\n
Submit with the command<\/p>\n
\r\nqsub fib.sge\r\n<\/pre>\n
A numpy job<\/strong><\/p>\n
No extra SGE configuration is necessary in order to use modules such as numpy or scipy.  For example, to run the following script<\/p>\n
\r\nimport numpy\r\n\r\ndef test_eigenvalue():\r\n  i=500\r\n  data=numpy.random.rand(i,i)\r\n  result=numpy.linalg.eig(data)\r\n  return result\r\n\r\nprint(test_eigenvalue())\r\n<\/pre>\n
you can use the following submission script, assuming that you've called the above script eig.py<\/p>\n
\r\n#!\/bin\/bash\r\n#$ -S \/bin\/bash\r\n#$ -N Python_fib\r\n#$ -cwd\r\n#$ -o outputfile.log\r\n#$ -j y\r\n#$ -V\r\nexport OMP_NUM_THREADS=$NSLOTS        ### For serial jobs auto sets to 1 core. DO NOT set a number here.\r\n\r\npython eig.py\r\n<\/pre>\n
Parallel Jobs - using more than one core<\/h2>\n
If you have some parallelism in your job (i.e. it has the ability to use more than 1 core) or you have used a package\/module which can use multiple cores you must request a number of cores from the smp.pe<\/code> batch parallel environment and include the OMP_NUM_THREADS<\/code> variable in your job to ensure that it uses the allocated resources correctly. Example:<\/p>\n
\r\n#!\/bin\/bash\r\n#$ -S \/bin\/bash\r\n#$ -N Python_fib\r\n#$ -cwd\r\n#$ -o outputfile.log\r\n#$ -j y\r\n#$ -V\r\n#$ -pe smp.pe 4 ### Request 4 cores from the batch system. Min 2, max 24.\r\n\r\nexport OMP_NUM_THREADS=$NSLOTS   ## For parallel jobs auto sets to the no. on the pe line. DO NOT set a number here.\r\n\r\npython eig.py\r\n<\/pre>\n
PyLab<\/h2>\n
If you wish to import NumPy, SciPy, Matplotlib and all in one easy step:<\/p>\n
\r\nimport pylab\r\n<\/pre>\n
Adding packages<\/h2>\n
You will need to set the following modulefile loading on the command line before proceeding:<\/p>\n
\r\nmodule load tools\/env\/proxy\r\n<\/pre>\n
OR alternativey, set the following manually:<\/p>\n
\r\nexport http_proxy=http:\/\/proxy.man.ac.uk:3128\r\nexport https_proxy=http:\/\/proxy.man.ac.uk:3128\r\n<\/pre>\n
Anaconda packages<\/h3>\n
To see what is already installed:<\/p>\n
\r\nconda list\r\n<\/pre>\n
To see if a package is available<\/p>\n
\r\nconda list package\r\n<\/pre>\n
where package<\/code><\/em> is replaced with the name of the package you want. If the package is listed as available for install please contact: its-ri-team@manchester.ac.uk<\/a> and we will try to add it to the central installation.<\/p>\n
pip\/pypi installation<\/h3>\n
To search for packages available to the pip<\/code> installer:<\/p>\n
\r\npip search keyword<\/em>\r\n<\/pre>\n
To install a package within your home folder storage:<\/p>\n
\r\npip install --user package<\/em>\r\n<\/pre>\n
where package<\/code> is replaced with the name of the package you want. This will install the package to a hidden directory called .local<\/code> in your home directory. It should be automatically picked up by python, you can test thus:<\/p>\n
\r\npython\r\nimport package<\/em>\r\nhelp (package<\/em>)\r\n<\/pre>\n
Further information<\/h2>\n