intermediate python - colorado software summit 2009 · mitchell smith — intermediate python page...

Colorado Software Summit: October 23 – 28, 2005 © Copyright 2005, Array BioPharma Inc.

Mitchell Smith — Intermediate Python Page

Intermediate Python

Mitchell W. Smith

Array BioPharma [email protected]



Agenda

Quick intro to Python

XML and SOAP support in Python

wxWindows

Twisted Framework

Web services using Twisted

SciPy and Numeric



Python Language

Open Source (OSI Certified)

copyrighted but use not restricted

owned by independent non-profit, PSF

Mature (15+ years old)

Extremely portable

Linux, Windows, Mac, PalmOS, WindowsCE, Symbian,etc.

Compiles to interpreted byte code

Also .NET and Java byte code implementations

Automatic memory management



Python High-Level Properties

Everything is an object

Packages, modules, classes, functions

Exception handling

Dynamic typing, polymorphism

Static scoping

Operator overloading

Indentation for block structure



Language Data Types

Numbers: int, long, float, complex

Boolean

Strings, Unicode: immutable

Lists, sets, and dictionaries: containers

Other types for e.g., binary data, callable types,call stack, etc.

Extension modules can define new“built-in” data types (such as multi-dimensionalarrays)



Language Elements

if, for, while

Full list operations and slices

List comprehensions

Iterators and generators

apply, eval, and exec

Functions, classes

Error handling and exceptions

assert



Python Language Examples

vec1 = [2, 4, 6, 7]

vec2 = [4, 3, -9, 1]

[x*y for x in vec1 for y in vec2 if x%2 == 0]

class example:

x = 1

def printX(self):

print self.x

str = 'The rain in Spain falls mainly on the plane.'

str[12:16]

import math

eval('(math.e**(2*math.pi*complex(0,-1))).real')

try:

fp = open('autotab.txt')

for line in fp:

print line

fp.close()

except:

print 'No file found.'

def foo():

print 'Hello world'

foo

foo()

from Numeric import *

arr1=array(([1,2,3],[4,5,6]))

arr2=array((2,4,6))

arr1*arr2

phoneExt = {'King Author' : 1234, 'Lancelot' : 3421}

for x in phoneExt:

print x

questions = ['What is your name? ',

'What is your quest? ',

'What is your favorite colour? ']

for question in questions:

raw_input(question)

colour = raw_input('What is your favorite colour? ')

if colour == 'blue':

print 'You may pass'

else:

print 'Arrr'



Large Standard Library

POSIX API

Network protocols, web protocols, email

Regular Expressions

Math, random, checksum

Encryption

Threads, processes, signals

Compression, archiving

XML



Python XML

Part of standard distribution

Latest implementation can be found at:

• http://pyxml.sourceforge.net/

Parser, DOM, SAX, XPath, XSLT, and SOAPsupport.

Currently, no XML Schema support.

Python bindings for a number of otherimplementations.



Python XML – Parsers

expat

PyXML – fast, non-validating parser

libxml2

Gnome – validating XML parser

xmlproc

PyXML – validating XML parser



Python XML – DOM

minidom

PyXML – lightweight DOM implementation

domlette

4Suite – lightweight XPath-oriented DOM

pulldom

PyXML – Lazy node evaluation DOM implementation

pxdom

DOXdesk – DOM level 3 Core and LS Recommendationscompliant



Python XML – SAX

PySAX

PyXML – SAX1 and SAX2 libraries



Python XML – XPath

libxml2

GNOME – XPath 1.0 compliant

4XPath

PyXML – XPath 1.0 compliant. Developed by 4Suite.

xpath

4Suite – An XPath 1.0 implementation for Domlettedocuments.



Python XML – XSLT

libxslt

Gnome – XSLT 1.0 compliant

Pyana

Pyana – Python wrappers to Xalan

4XLST

4Suite – XSLT 1.0 compliant



Two main projects:

SOAPpy

• SOAP 1.0 compliant

• WSDL ?? compliant

• Built on PyXML

• Globus support

Zolera SOAP Infrastructure (ZSI)

• SOAP 1.1 compliant

• WSDL 1.1 compliant

• Built on PyXML

Main web page: http://pywebsvcs.sourceforge.net/

Python Web Services



SOAPpy – A Simple Server

#Server code

import SOAPpy

def add(x, y):

return x + y

server = SOAPpy.SOAPServer(("localhost", 8080))

server.registerFunction(add)

server.serve_forever()

#Client code

import SOAPpy

server = SOAPpy.SOAPProxy("http://localhost:8080/")

print server.add(4, 5)



SOAPpy – Getting Startedfrom SOAPpy import SOAPProxy

url = "http://services.xmethods.com/soap"

namespace = "urn:xmethods-delayed-quotes"

server = SOAPProxy(url, namespace)

server.config.dumpSOAPOut = 1

server.config.dumpSOAPIn = 1

print server.getQuote(symbol = "IBM")



SOAPpy – WSDL Examplefrom SOAPpy import WSDL

wsdlFile = 'C:\\Python24\\Lib\\site-packages\\wsdl\\ndfdXML.wsdl'

server = WSDL.Proxy(wsdlFile)

#let’s see the methods

server.methods.keys()

#get the input parameters

[[x.name, x.type] for x in server.methods['NDFDgen'].inparams]

#get the output parameters

[[x.name, x.type] for x in server.methods['NDFDgen'].outparams]

#make the request

WSDL.Config.dumpSOAPIn = 1

WSDL.Config.dumpSOAPOut = 1

server.NDFDgen( 39.0000, -77.0000, 'time-series', '2005-10-23T17:00', '2005-10-28T12:00', [])



ZSI – A Simple Server

#Server code

from ZSI import dispatch

def add(x, y):

return x + y

dispatch.AsServer()

#Client code

from ZSI.client import Binding

b=Binding(url='http://localhost')

print b.add(4, 5)



ZSI Python Class Generationfrom WSDL

from ZSI.wstools import WSDLTools

from ZSI import wsdl2python

reader = WSDLTools.WSDLReader()

wsdl = reader.loadFromFile(

'C:\\Python24\\Lib\\site-packages\\wsdl\\ndfdXML.wsdl')

wsm = wsdl2python.WriteServiceModule(wsdl)

wsm.write(output_dir='C:\\Python24\\Lib\\site-packages\\wsdl\\')



ZSI WSDL Examplefrom ZSI import ServiceProxy

import ndfdXML_services

import sys

#get locator and port type. Trace to stdout for debugging.

loc = ndfdXML_services.ndfdXMLLocator()

kw={'tracefile':sys.stdout}

portType = loc.getndfdXMLPortType(**kw)

#generate request and parameters

request = ndfdXML_services.NDFDgenByDayRequestWrapper()

request._latitude = 40

request._longitude = -105.2775

request._startDate = [2005, 10, 25, 0, 0, 0]

request._numDays=10

request._format = '24 hourly'

#run web service

response = portType.NDFDgenByDay(request)

print response



ZSI – Apache mod_python… HelloAndSum module

def helloWorld():

return "Hello, world"

def sum(*args):

return reduce(lambda x,y: x + y, args)

…mod_python

from ZSI import dispatch

from mod_python import apache

import HelloAndSum

mod = __import__(’encodings.utf_8’, globals(), locals(), ’*’)

mod = __import__(’encodings.utf_16_be’, globals(), locals(), ’*’)

def handler(req):

dispatch.AsHandler(modules=(HelloAndSum,), request=req)

return apache.OK

…Client side

from ZSI.client import Binding

fp = open(’debug.out’, ’a’)

b = Binding(url=’/cgi-bin/simple-test’, tracefile=fp)

fp.close()

print b.sum(*range(1,100))

print b.helloWorld()



wxPython

wxPython is a GUI toolkit for Python.

wxPython is an extension module (native code)that wraps the popular wxWidgets cross-platformGUI library, which is written in C++.

wxWidgets use the native widgets/controls and platformindependent widgets as well.

wxWidgets runs on Linux, Windows, OS X, Unix, etc.• Work is under way to port it to embedded devices.

wxPython comes with a resource editor to design aGUI graphically and, at run-time, load in an XMLresource file.



wxPython Demo



wxPython Widgets

wx.Button

wx.BitmapButton

wx.RadioButton

wx.SpinButton

wx.Choice

wx.RadioBox

…



Date, Time, and Mask Controls



Some Common Dialogs



wx.Notebook with wx.Listbox

A notebook is a series of tabs each with a window.

A list control displays in list view, report view, icon

view and small icon view.

A list view items

can be virtual.



wx.TreeCtrl andwx.gizmos.TreeListCtrl

Supports editing, multiple selection,images.

Virtual children.



wx.Grid

Lots of functionality

Highly customizable

Renderers

Editors

Drag and drop

Rows and columns

can be dragged

Spreadsheet-like

Virtual Rows



Sizers

Closely related to layout in Java (LayoutManagers)

and other GUI toolkits.

Layout determined by the sizer’s class algorithm.

Sizers can be nested.

gridbaglayout now supported.

def makeSimpleBox(win):

box = wx.BoxSizer(wx.HORIZONTAL)

box.Add(wx.Button(win, 1010, "one"), 0, wx.EXPAND)

box.Add(wx.Button(win, 1011, "two"), 0, wx.EXPAND)

box.Add(wx.Button(win, 1012, "three"), 0, wx.EXPAND)

box.Add((60, 20), 0, wx.EXPAND)

box.Add(wx.Button(win, 1013, "five"), 1, wx.EXPAND)

return box

…



XRCed – Widget Example



XRCed – Dialog Example



XRCed – Dialog Resource XML <object class="wxDialog" name="DIALOG_EDIT_LIBRARY">

<title>Edit Library</title>

<style>wxDEFAULT_DIALOG_STYLE|wxRESIZE_BORDER</style>

<object class="wxBoxSizer">

<orient>wxVERTICAL</orient>

<object class="sizeritem">

<object class="wxBoxSizer">

<orient>wxHORIZONTAL</orient>


<object class="wxFlexGridSizer">

<cols>2</cols>


<object class="wxStaticText">

<label>Name:</label>

</object>

<flag>wxALIGN_CENTRE_VERTICAL</flag>

</object>


<object class="wxTextCtrl" name="TXT_LIBRARYNAME"/>

<flag>wxGROW</flag>

</object>


<object class="wxStaticText">

<label>Comments:</label>

</object>

<flag>wxALIGN_CENTRE_VERTICAL</flag>

</object>

…



Dialog Code Sample#LOAD APPLICATION

class App(wx.App):

def OnInit(self):

resourcefile = join(dirname(argv[0]), 'resources.xrc')

self.resources = wx.xrc.XmlResource(resourcefile)

…

#CLASS FOR LIBRARY DIALOG

class EditLibraryDialog(XrcDialog):

def __init__(self, parent, resources, id=None, name='', comments='',

isCustom=True, forSale=False, defaultPath=None):

XrcDialog.__init__(self, parent, resources,

'DIALOG_EDIT_LIBRARY')

self.id = id

self.txtLibraryName = wx.xrc.XRCCTRL(self, 'TXT_LIBRARYNAME')

self.txtComments = wx.xrc.XRCCTRL(self, 'TXT_COMMENTS')

self.chkCustomLibrary = wx.xrc.XRCCTRL(self, 'CHK_CUSTOMLIBRARY')

self.chkForSale = wx.xrc.XRCCTRL(self, 'CHK_FORSALE')

self.dirDefaultPath = wx.xrc.XRCCTRL(self, 'DIR_DEFAULTPATH')

self.txtLibraryName.SetValue(name)

self.txtComments.SetValue(comments)

self.chkCustomLibrary.SetValue(isCustom)

self.chkForSale.SetValue(forSale)

if defaultPath:

self.dirDefaultPath.SetPath(defaultPath)

…

#INVOCATION

dlg = EditLibraryDialog(self.frame, self.resources,

id, name, comments, isCustom, forSale,

defaultPath)

if wx.ID_OK == dlg.ShowModal():

…



Twisted Framework

Written in Python.

Supports numerous internet and other protocols.

Application server/framework/library.

Meant for programmers – no configuration system.

MIT open source license



Twisted High-Level Overview



Twisted Protocols

Twisted support numerous protocols.

Client-side calls to the server are written withoutknowledge of the underlying protocol.

Twisted is highly extensible and readily supportsthe addition of new protocols.

Twisted Internet also covers the interfaces for thevarious transports which allows network code to bewritten without regard to the underlyingimplementation.



Twisted Deferreds

Used for the programming logic after a call to theTwisted Server.

Provide for post-actions to a server call such asinvoking a dialog and error-handling.

Allow for deferred actions.

Client actions continue while back-end processing takesplace.

Subsequent client/server actions are deferred until theback-end processing completes.

Used for cooperative multitasking and good serverperformance.



Twisted Deferred Example def EditCurrentLibrary(self, evt=None, newLibrary=False):

def _onGetLibraryData(data):

...

dlg = widgets.EditLibraryDialog(self.frame,

self.resources,

id, name, comments,

isCustom, forSale,

defaultPath)

if wx.ID_OK == dlg.ShowModal():

return (dlg.GetId(), dlg.GetName(), dlg.GetComments(),

dlg.GetIsCustom(), dlg.GetForSale(),

dlg.GetDefaultPath())

def _onLibraryChanged(data, self):

if data:

id, name, comments, isCustom, forSale, defaultPath = data

deferred = self.rref.callRemote('createOrUpdateLibrary',

id, name, comments, isCustom,

forSale, defaultPath)

return deferred

if newLibrary:

library = None

else:

library = self.library

deferred = self.rref.callRemote('getLibraryDetails', library)

deferred.addCallbacks(_onGetLibraryData, self.onError)

deferred.addCallbacks(_onLibraryChanged, self.onError, (self,))

return deferred



Twisted SOAP Server

from twisted.web import soap, server

from twisted.internet import reactor, defer

class Example(soap.SOAPPublisher):

def soap_processRequest(self, id, name, comments,

isCustom, forSale, defaultPath):

print repr([id, name, comments, isCustom, forSale, defaultPath])

return defer.succeed('OK') #Trivial deferred – Real example go to Database.

reactor.listenTCP(8080, server.Site(Example()))

reactor.run()



Twisted/SOAP/wxPythonClient Dialog

import wx, wx.grid, wx.xrc

import SOAPpy

from wxPython.wx import NULL

from twisted.internet import wxreactor

wxreactor.install()

from twisted.internet import reactor, defer

import twisted.web.soap

from os.path import dirname, join

from sys import argv, platform

ID_EXIT = 101

ID_DIALOG = 102

class XrcDialog(wx.Dialog):

def __init__(self, parent, resources, xrcName):

pre = wx.PreDialog()

resources.LoadOnDialog(pre, parent, xrcName)

self.this = pre.this

self._setOORInfo(self)

class EditLibraryDialog(XrcDialog):

def __init__(self, parent, resources, id=None, name='',

comments='',isCustom=True, forSale=False,

defaultPath=None):

XrcDialog.__init__(self, parent, resources,

'DIALOG_EDIT_LIBRARY')

self.id = id

self.txtLibraryName = wx.xrc.XRCCTRL(self,

'TXT_LIBRARYNAME')

self.txtComments = wx.xrc.XRCCTRL(self, 'TXT_COMMENTS')

self.chkCustomLibrary = wx.xrc.XRCCTRL(self,

'CHK_CUSTOMLIBRARY')

self.chkForSale = wx.xrc.XRCCTRL(self, 'CHK_FORSALE')

self.dirDefaultPath = wx.xrc.XRCCTRL(self,

'DIR_DEFAULTPATH')

self.txtLibraryName.SetValue(name)

self.txtComments.SetValue(comments)

self.chkCustomLibrary.SetValue(isCustom)

self.chkForSale.SetValue(forSale)

if defaultPath:

self.dirDefaultPath.SetPath(defaultPath)

def GetId(self): return self.id

def GetName(self): return self.txtLibraryName.GetValue()

def GetComments(self): return self.txtComments.GetValue()

def GetIsCustom(self): return self.chkCustomLibrary.IsChecked()

def GetForSale(self): return self.chkForSale.IsChecked()

def GetDefaultPath(self): return self.dirDefaultPath.GetPath()



Twisted/SOAP/wxPythonClient Main

class MyApp(wx.App):

def OnInit(self):

self.frame = MyFrame(NULL, -1, "Main Application")

self.frame.Show(True)

self.SetTopWindow(self.frame)

wx.EVT_CLOSE(self.frame, self.Exit)

return True

def Exit(self, evt=None):

if self.frame:

self.frame.Destroy()

reactor.stop()

def demo():

app = MyApp(0)

reactor.registerWxApp(app)

reactor.run(0)

if __name__ == '__main__':

demo()



Twisted/SOAP/wxPythonClient Frame

class MyFrame(wx.Frame):

def __init__(self, parent, ID, title):

wx.Frame.__init__(self, parent, ID, title, wx.DefaultPosition, wx.Size(300, 200))

self.frame = self

menu = wx.Menu()

menu.Append(ID_DIALOG, "Edit Library...", "Edit library")

menu.Append(ID_EXIT, "E&xit", "Terminate the program")

menuBar = wx.MenuBar()

menuBar.Append(menu, "&File")

self.SetMenuBar(menuBar)

wx.EVT_MENU(self, ID_EXIT, self.DoExit)

wx.EVT_MENU(self, ID_DIALOG, self.DoDialog)

resourcefile = join(r'C:\Documents and Settings\msmith\Desktop\CSS Python', 'resources.xrc')

self.resources = wx.xrc.XmlResource(resourcefile)

self.connection = SOAPpy.SOAPProxy('http://localhost:8080/')

def DoExit(self, event):

self.frame.Destroy()

reactor.stop()

def DoDialog(self, event):

self.dlg = EditLibraryDialog(self.frame, self.resources)

wx.EVT_BUTTON(self.dlg, wx.xrc.XRCID('wxID_OK'), self.FinishDialog)

self.dlg.ShowModal()

def FinishDialog(self, event=None):

self.connection.processRequest(self.dlg.GetId(), self.dlg.GetName(), self.dlg.GetComments(),

self.dlg.GetIsCustom(), self.dlg.GetForSale(),

self.dlg.GetDefaultPath())

self.dlg.EndModal(0)



SciPy

SciPy is a collection of scientific and numericaltools

Built on top of Numeric and subsumes its functionality

Core code C++ with Python bindings

BSD License



SciPy Major Packages

Integration

Optimization

Interpolation

Signal Processing

Genetic Algorithms

FFT

Linear Algebra

Integrates with Python Imaging Library



Numeric Functions- array - NumPy Array construction

- zeros - Return an array of all zeros

- empty - Return an unitialized array (200x faster than zeros)

- shape - Return shape of sequence or array

- rank - Return number of dimensions

- size - Return number of elements in entire array or a

certain dimension

- fromstring - Construct array from (byte) string

- take - Select sub-arrays using sequence of indices

- put - Set sub-arrays using sequence of 1-D indices

- putmask - Set portion of arrays using a mask

- reshape - Return array with new shape

- repeat - Repeat elements of array

- choose - Construct new array from indexed array tuple

- cross_correlate - Correlate two 1-d arrays

- searchsorted - Search for element in 1-d array

- sum - Total sum over a specified dimension

- average - Average, possibly weighted, over axis or array.

- cumsum - Cumulative sum over a specified dimension

- product - Total product over a specified dimension

- cumproduct - Cumulative product over a specified dimension

- alltrue - Logical and over an entire axis

- sometrue - Logical or over an entire axis

- allclose - Tests if sequences are essentially equal



More Numeric Functions- arrayrange (arange) - Return regularly spaced array

- asarray - Guarantee NumPy array

- sarray - Guarantee a NumPy array that keeps precision

- convolve - Convolve two 1-d arrays

- swapaxes - Exchange axes

- concatenate - Join arrays together

- transpose - Permute axes

- sort - Sort elements of array

- argsort - Indices of sorted array

- argmax - Index of largest value

- argmin - Index of smallest value

- innerproduct - Innerproduct of two arrays

- dot - Dot product (matrix multiplication)

- outerproduct - Outerproduct of two arrays

- resize - Return array with arbitrary new shape

- indices - Tuple of indices

- fromfunction - Construct array from universal function

- diagonal - Return diagonal array

- trace - Trace of array

- dump - Dump array to file object (pickle)

- dumps - Return pickled string representing data

- load - Return array stored in file object

- loads - Return array from pickled string

- ravel - Return array as 1-D

- nonzero - Indices of nonzero elements for 1-D array

- shape - Shape of array

- where - Construct array from binary result

- compress - Elements of array where condition is true

- clip - Clip array between two values

- ones - Array of all ones

- identity - 2-D identity array (matrix)



Numeric Math Functions add logical_or exp

subtract logical_xor log

multiply logical_not log10

divide maximum sin

divide_safe minimum sinh

conjugate bitwise_and sqrt

power bitwise_or tan

absolute bitwise_xor tanh

negative invert ceil

greater left_shift fabs

greater_equal right_shift floor

less arccos arctan2

less_equal arcsin fmod

equal arctan hypot

not_equal cos around

logical_and cosh sign

arccosh arcsinh arctanh



Numeric Module

Mlab

Support basic functions of Matlab

RandomArray

Generates arrays with random values

• Supports binomial, chi square, poisson, etc. distributions

ArrayPrinter

Converts arrays to pretty-printed strings



Resourceswww.python.org

Python home site – downloads, community, documentation,conferences, 3rd Party add-ins, etc.

www.pythonology.org

Success stories

http://pywebsvcs.sourceforge.net/

Python web services home page

http://twistedmatrix.com/

Twisted homepage.

http://www.scipy.org/

SciPy and Numeric home pages



Resources

http://www.wxpython.org/wxPython Home Page

http://pyxml.sourceforge.net/XML Python Home Page

http://www.doxdesk.com/software/py/pxdom.htmlpxdom module Home Page

http://www.ailab.si/orangeOrange – a great Data Mining package written in Python

Python and XMLChristopher A. Jones & Fred L. Drake, Jr.

• O’Reilly Press, Dec. 2001.



Questions?