Anaconda Python

Overview

Anaconda is a completely free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing general-purpose statistical software package. It has 100+ of the most popular Python packages for science, math, engineering, data analysis.

Several versions are installed on zrek, please see ‘Set up Procedure’ below for details.

To see what is available in these versions load the appropriate modulefile (see below) and then run the command:

conda list

Note: none of the python 2.6 or python 3.4 or 3.5 packages have been installed.

Restrictions on Use

None, but all users should read the End User License Agreement before using the software.

Set up procedure

Load one of the following modulefiles for the version you wish to use:

For Python 2.7.x

module load apps/binapps/anaconda/2.5.0
module load apps/binapps/anaconda/2.4.1
module load apps/binapps/anaconda/2.3.0

For Python 3.4

# Notice the extra /3/ in the modulefile name

module load apps/binapps/anaconda/3/2.5.0
module load apps/binapps/anaconda/3/2.4.1
module load apps/binapps/anaconda/3/2.3.0

For Python 3.5

# Notice the extra /3/ in the modulefile name

module load apps/binapps/anaconda/3/4.2.0

Running the application

After loading the modulefile you need to have a batch job submission script. Below are some examples.

Simple, Python-only job

Here is a simple Python script (download fib.py)

parents, babies = (1, 1)
while babies < 100:
    print 'This generation has %d babies' % babies
    parents, babies = (babies, parents + babies)

This is an example SGE submission script (download fib.sge)

#!/bin/bash
#$ -S /bin/bash
#$ -N Python_fib
#$ -cwd
#$ -o outputfile.log
#$ -j y
#$ -V
python fib.py

Submit with the command

qsub fib.sge

A numpy job

No extra SGE configuration is necessary in order to use modules such as numpy or scipy. For example, to run the following script

import numpy

def test_eigenvalue():
  i=500
  data=numpy.random.rand(i,i)
  result=numpy.linalg.eig(data)
  return result

print(test_eigenvalue())

you can use the following submission script, assuming that you've called the above script eig.py

#!/bin/bash
#$ -S /bin/bash
#$ -N Python_fib
#$ -cwd
#$ -o outputfile.log
#$ -j y
#$ -V
python eig.py

PyLab

If you wish to import NumPy, SciPy, Matplotlib and all in one easy step:

import pylab

Adding packages

You will need to set the following variables on the command line before proceeding:

export http_proxy=http://webproxy.its.manchester.ac.uk:3128
export https_proxy=http://webproxy.its.manchester.ac.uk:3128

Anaconda packages

To see what is already installed:

conda list

To see if a package is available

conda list package

where package is replaced with the name of the package you want. If the package is listed as available for install please contact: its-ri-team@manchester.ac.uk and we will try to add it to the central installation.

pip/pypi installation

pip install --user package

where package is replaced with the name of the package you want. This will install the package to a hidden directory called .local in your home directory. It should be automatically picked up by python, you can test thus:

python
import package
help (package)

Further information

• Anaconda Python documentation, FAQ, mailing list etc
• IT Services runs an Introduction to numerical computing with Python workshop approximately three times a year.

Hints and Tips

Got a handy tip? Please send it in to its-ri-team@manchester.ac.uk ...

Plotting graphs with Pyplot

If you want to plot a graph to a PNG file, say, in batch, try the following (see this stackoverflow question and answer):

# Create a file named graph.py:

import matplotlib as mpl
# Agg backend wil render without X server on a compute node in batch
mpl.use('Agg')
import matplotlib.pyplot as plt

fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(range(10))
fig.savefig('temp.png')
# End of python file.

# Load python and submit:
module load apps/binapps/anaconda/3/2.3.0
qsub -b y -cwd -V -l short python ./graph.py

# When job completes, view the graph
eog temp.png