Extreme Testing, Integration andSandwiches with Python

Andy RobinsonCEO / Chief Architect, ReportLab Inc.

BCS OOPS3 July 2002

Part 1

Introduction

What is Python?

• High level, object oriented scripting language

• Open Source

• Cross-platform - based on Ansi C or Java

• Cross-platform GUI and DB libraries

• very easy to read

• very dynamic

What is used for?

• Friendly alternative to Perl

• Quick and dirty scripting

• CGI scripts

• Large scale application development

• Integrating other systems

Python as an Integration Tool

• Good with files

• Dynamic or static linking to C-based languages

• Great COM support

• CORBA etc.

• Great networking libraries

• Great Java integration

"Low-threat" uses in the corporate world...or how to get the boss to say "Yes"

• Adding a macro language to applications

• Rapid Prototyping of object models and algorithms

• Building test harnesses for other systems

• Data Cleaning and Transformation

• Python as Glue

Getting Popular....• 500,000+ users

• 15+ books

• Used at NASA, Yahoo, Red Hat, Infoseek, Industrial Light andMagic, IBM, Fidelity Investments and many more

Part 2 - The Python Environment

Where to get it - http://www.python.org/

What you getWindows distribution as an example (CD available today)

• Python 2.0 - command line program

• Full documentation

• Windows extensions - GUI and COM support

Many competing IDEs now available.

Demo time...

Python core on Windows• python20.dll - 652kb, the language, exports almost everything

• python.exe - 5kb console mode program

• pythonw.exe - 21kb non-console mode program - avoids uglyblack DOS boxes when you don't want standard input/output

File Types• .py - python source (.java)

• .pyc - byte-compiled, rebuilt on demand (.class)

• .pyd - DLL with a Python-specific entry point

Part 3

A Crash Course in Python

Hello World

PythonWin 2.1.3 (#35, Apr 8 2002, 17:47:50) [MSC 32 bit (Intel)] onwin32.Portions Copyright 1994-2001 Mark Hammond (MarkH@ActiveState.com) -see 'Help/About PythonWin' for further copyright information.

>>> print 'hello world'hello world

Contrast that to C++ or Java

Numbers, Strings and Variables

>>> 2+24

You can assign to variables with '=' :>>> x=3>>> x3>>> y=x*2>>> y6>>>

Python supports all standard arithmetical operators :>>> 22/73>>> 22.0/7.03.1428571428571428>>>

Complex numbers are supported using the letter j :>>> (3 + 1j) * (3 -1j)(10+0j)>>>

Strings can be wrapped in double or single quotes:>>> greeting = 'hello'>>> epithet = "stranger"

They can be concatenated (using '+') and repeated (using '*'):>>> greeting + ", " + epithet'hello, stranger'

>>> "spam" * 10'spamspamspamspamspamspamspamspamspamspam'>>>

Lists

Lists are wrapped in square brackets.They can contain any Python objects.

>>> mylunch = ['spam', 'eggs', 'guiness', 'raspberries', 'wafer-thin mint']>>> mylunch[0]'spam'>>> mylunch[1:3], mylunch[-1](['eggs', 'guiness'], 'wafer-thin mint')>>>

Tuples

Parentheses indicate a tuple.Tuples are similar to lists but can't be modified once created.

>>> mylunch[2] = 'tea'>>> mylunch['spam', 'eggs', 'tea', 'raspberries', 'wafer-thin mint']>>> meal_deal = ('burger', 'fries', 'coke') # a tuple>>> meal_deal[1] = 'onion rings'Traceback (most recent call last): File "<interactive input>", line 1, in ?TypeError: object doesn't support item assignment>>>

Control Structures

The For Loop

The for loop operates over lists not numbers:

>>> for item in mylunch:... print item... spameggstearaspberrieswafer-thin mint>>>

The While Loop>>> x = 2>>> while x < 50:... x = x * 2... print x... 48163264>>>

If and Else

The else clause is optional:

>>> if 'chocolate' in mylunch:... print "that's not allowed"... else:... print "enjoy your meal"... enjoy your meal>>>

Elif

You can also have a number of intermediate elif (else-if) clauses.Something like switch or case in other languages.

>>> salary = 20000>>> if salary < 4000:... tax_rate = 0... elif salary < 29000:... tax_rate = 0.25... elif salary < 100000:... tax_rate = 0.4... else:... emigrate() # that's a function call... >>>

Functions>>> def double(x):... return x * 2... >>> double(2)4>>> def first_and_last(aList):>>> def first_and_last(aList):... return(aList[0], aList[-1])... >>> first_and_last(range(5))(0, 4)>>> def sayHello():... print 'hello'... >>> sayHello()hello>>>

Functions with default arguments>>> def makeCoffee(size, milk=None, sugar=None):... order = 'one ' + size + ' coffee'... if milk and sugar:... order = order + ' with milk and sugar'... elif milk:... order = order + ' with milk'... elif sugar:... order = order + ' with sugar'... else:... pass # pass means 'do nothing'... return order...

>>> makeCoffee('large')'one large coffee'>>> makeCoffee('large', 1)'one large coffee with milk'>>> makeCoffee('large', milk=0, sugar=1)'one large coffee with sugar'>>>

Dictionaries

Dictionaries are based on a hash table.Lookup time is almost constant.Enclosed by braces.Keys and values separated by a colon.

>>> fur_colours = {}>>> fur_colours['Tinky-Winky'] = 'purple'>>> fur_colours['Dipsy'] = 'green'>>> fur_colours['LaLa'] = 'yellow'>>> fur_colours{'Tinky-Winky': 'purple', 'Dipsy': 'green', 'LaLa': 'yellow'}>>> fur_colours['LaLa']'yellow'

Dictionary Methods

Dictionaries support some useful methods of searching:by keys, values and whether or not a certain key is present.

>>> fur_colours.keys()['Tinky-Winky', 'Dipsy', 'LaLa']>>> fur_colours.values()['purple', 'green', 'yellow']>>> fur_colours.items() # converts to a list of tuples[('Tinky-Winky', 'purple'), ('Dipsy', 'green'), ('LaLa', 'yellow')]>>> len(fur_colours)3>>> fur_colours.has_key('Po')0>>>

Modules

Python code is organised into modules.Some are built into Python, others are stored in external files.

>>> import math>>> math.sin(math.pi/2)1.0>>>

>>> from string import split, join>>> split('We are the knights who say Ni')['We', 'are', 'the', 'knights', 'who', 'say', 'Ni']>>>

Classes

>>> class Car:... def __init__(self):... self.milespergallon = 25.0... self.travelled = 0... self.color = 'blue'... self.gas = 20... def drive(self, miles):... self.travelled = self.travelled + miles... self.gas = self.gas - (miles / self.milespergallon)... >>> c = Car()>>> c.drive(100)>>> c.travelled100>>> c.gas16.0>>>

Exception Handling

>>> def func1(arg):... func2(arg)... >>> def func2(arg):... func3(arg)... >>> def func3(arg):... # this should cause an error... return arg / 0... >>> func1(17)Traceback (most recent call last): File "<interactive input>", line 1, in ? File "<interactive input>", line 2, in func1 File "<interactive input>", line 2, in func2 File "<interactive input>", line 3, in func3ZeroDivisionError: integer division or modulo by zero>>>

The Except Clause

The except clause can optionally specify particularerror types to look for:

>>> try:... y_scale_factor = plotheight / (dataMax - dataMin)... except ZeroDivisionError:... y_scale_factor = 0... >>>

The Finally Clause

The try...finally clause always executes the finally sectionwhether an error occurs or not:

>>> f = open('somefile.dat', 'w')>>> try:... # fetch some data... # store it in the file... finally:... f.close() # make sure file is closed... # even if errors occurred... >>>

Have you noticed...?

...the lack of block delimiters?

'statement grouping is done by indentation instead of begin/endbrackets'

Part 4

Integration with Python

Integration Tools

• DB API

• XML

• Numerous web protocols inc. SOAP and XML-RPC

• Pyro and others

• COM

• CORBA

• Java - Jython, JPE

• C/C++ - Python C API, SWIG, CXX, Pyrex...

Python Database APIs>>> import mx.ODBC.Windows

>>> conn = mx.ODBC.Windows.connect(DATASOURCE, USER, PASSWORD)>>> cur = conn.cursor()>>> cur.execute("""SELECT JobDescriptionId, DocGroupId, Description... FROM RPAJobDescriptions... WHERE UserId = 'alice' ORDER BY DocGroupId""")>>> data = cur.fetchall()

>>> pprint.pprint(data)[(6, 'multipagesamples', 'simple multipage doc #1'), (7, 'multipagesamples', 'simple multipage doc #2'), (8, 'multipagesamples', 'simple multipage doc #3')]>>>

XML Parsing

>>> import pyRXP>>> p=pyRXP.Parser()>>> p.parse('<tag1><tag2>content</tag2></tag1>')('tag1', None, [('tag2', None, ['content'], None)], None)>>>

Reading a URL

import urllib conn = urllib.urlopen('http://www.reportlab.com/index.html') print conn.headers data = conn.read()

Similar high level APIs exist for most internet protocols

Integration with C /C++

• Python C API: extend or embed

• SWIG: generate wrappers for C++ code

• CXX and Pyrex - newer

COM Clients

>> import win32com.client>> xl = win32.client.Dispatch("Excel.Application")

COM Servers

class PythonUtilities: _public_methods_ = [ 'SplitString' ] _reg_progid_ = "PythonDemos.Itilities" # Never copy the following ID # use "print pythoncom.CreateGuid()" to make a new one _reg_clsid_ = '{EFC3F30B-FB1B-40BE-8834-00F061BFAA6C}'

def SplitString(self, val, item=None): import string if item != None: item = str(item) return string.split(str(val), item)

# Add code so that when this script is run by# Python.exe, it self registers.if __name__ == '__main__': print "registering COM server...' import win32com.server.register win32com.server.register.UseCommandLine(PythonUtilities)

CORBA

• OmniORB, one of only 3 Corba 2.1 compliant ORBS

• Wrapper classes are statically generated from IDL files, so youget editor auto-completion on your remote objects :-)

CORBA Client

>>>import sys,CORBA,Snake>>>orb =CORBA.ORB_init(sys.argv,CORBA.ORB_ID)>>>adder =orb.string_to_object("corbaname:rir:#adder.obj")>>>adder.accumulate(5)5>>>adder.accumulate(6)11>>>adder.accumulate(42)53>>>adder.reset()>>>adder.accumulate(10)10

CORBA Serverimport sys,CORBA,CosNaming,Snake,Snake__POA

class Adder_i (Snake__POA.Adder): def __init__(self): self.value =0 def accumulate(self,a): self.value =self.value +a return self.value def reset(self): self.value =0orb = CORBA.ORB_init(sys.argv,CORBA.ORB_ID)poa = orb.resolve_initial_references("RootPOA")

adderServant = Adder_i()poa.activate_object(adderServant)adderObjref = adderServant._this()nameRoot = orb.resolve_initial_references("NameService")nameRoot = nameRoot._narrow(CosNaming.NamingContext)name = [CosNaming.NameComponent("adder","obj")]nameRoot.rebind(name,adderObjref)poa._get_the_POAManager().activate()orb.run()27

Java Integration

• Jython - Python running on JVM http://www.jython.org/ http://www.jython.org/applets/index.html

• JPE - Java Python Bridge (Linking at JNI level) http://jpe.sourceforge.net/

Delphi integration

• Complete set of Delphi components which wrap up Python

Part 5

Testing with Python

Testing Tools

• Interactive Prompt

• Doctest

• Unittest

• Crosschecker

Interactive Prompt

• test functions as you write them

DocTest

• comment your code with docstrings showing interactiveprompt sessions

• it replays them and checks the output is as specified

unittest

• same as JUnit and pals.

• Aggregates tests.

Testing Approaches

Two approaches to testing:

• call your APIs to check they work

• use Python's glue capabilities to test systems end to end

Crosschecker

• Commercial product from Secret Labs with an IDE geared towriting and running test cases.

Part 6

Conclusion

Conclusion

Python is the most cost-effective and general way to test systems inother languages - at unit and end-to-end level.

• Python http://www.python.org/

• ReportLab http://www.reportlab.com/

• Jython http://www.jython.org/ http://www.jython.org/applets/index.html

• JPE http://jpe.sourceforge.net/

• Secret Labs http://www.pythonware.com/