Pypar has finally moved to Sourceforge

You can now browse the Subversion repository for pypar: here

You can also download the last (ANU) version of pypar here: pypar_1.9.3.tgz

Back to Ole Nielsen's Home Page

Pypar (last update: 24 April 2007)

Parallel Programming in the spirit of Python!

Pypar is an efficient but easy-to-use module that allows programs/scripts written in the Python programming language to run in parallel on multiple processors and communicate using message passing. Pypar provides bindings to an important subset of the message passing interface standard MPI. Other Python MPI bindings available from other developers include: PyMPI, Scientific Python and pythonMPI.

Pypar is characterised by the following features

• Pypar is flexible:
  General Python objects of any type can be communicated - not just arrays. Optional functionality is available through keyword arguments.

• Pypar is easy to use:
  The user need only specify what to send and to which processor. Pypar takes care of details about data types, size, dimension and MPI specifics such as tags, communicators and buffers. A typical send and receive pair looks like
  

  	pypar.send(x, destination=1)	# Send object x to processor 1
  	y=pypar.receive(source=1)	# Receive object y from processor 1

• Pypar is efficient:
  Full bandwidth of C-MPI programs is achieved for consecutive Numerical arrays. Latency is less than twice that of pure C-MPI programs. Test programs to verify this are included (pytiming, ctiming.c).

• Pypar is easy to install:
  Installs using standard distutils.

• Pypar is light weight:
  Pypar consists of only two files: pypar.py and the c-extension mpiext.c. It depends only Python, an MPI library (C), Numeric Python and a C compiler.

• Pypar does not require the Python interpreter to be modified or recompiled:
  Parallel python programs use the standard Python and need merely import the pypar module. This means for example that you can upgrade Python independently of your parallel codes.

• Pypar is easy to extend:
  With the framework in place it is not hard to include additional MPI functionality should the need arise. In fact, many contributors have done exactly that in the past (see README for a list of contributors) Alternatively, drop me a note if there is some functionality you would like to see included and I'll add it to my 'todo' list.

Open Source - Of Course

• pypar.py: The Python interface
• mpiext.c: The C-extension

Documentation

• README
• DOC

You can also try out the

• PyPar Tutorial (pdf)

• Python Crash Course (pdf)
• Maple Tutorial (pdf)

Links to people who have contributed and projects that use pypar

Download

You can download the source, tests documentation and examples for Pypar at http://datamining.anu.edu.au/software/pypar/pypar_1.9.3.tgz

Example

Here is an example of a simple program using Pypar ring_example.py. It basically communicates some text in a ring structure. To run it on four MPI processes, say, simply issue the command:

mpirun -np 4 python ring_example.py

and thanks to Constantinos Makassikis for helping me render this code nicely in html with code2blog. Thanks heaps!

import pypar # The Python-MPI interface numproc = pypar.size() # Number of processes as specified by mpirun myid = pypar.rank() # Id of of this process (myid in [0, numproc-1]) node = pypar.get_processor_name() # Host name on which current process is running print 'I am proc %d of %d on node %s' %(myid, numproc, node) if myid == 0: # Actions for process 0 msg = 'P0' pypar.send(msg, destination=1) # Send message to proces 1 (right hand neighbour) msg = pypar.receive(source=numproc-1) # Receive message from last process print 'Processor 0 received message "%s" from processor %d' %(msg, numproc-1) else: # Actions for all other processes source = myid-1 destination = (myid+1)%numproc msg = pypar.receive(source) msg = msg + 'P' + str(myid) # Update message pypar.send(msg, destination) pypar.finalize()