introduction to python - dabeazdabeaz.com/python/tutorial/beazley_intro_python/intropy.pdf ·...

An Introduction to Python

David M. Beazley

Department of Computer Science University of Chicago

[email protected]

O’Reilly Open Source Conference

July 17, 2000

O’Reilly OSCON 2000, Introduction to Python, Slide 1July 17, 2000, [email protected]

Overview

A Tour of Python

A Brief Tutorial Lexical Conventions and Syntax Types and Objects Operators and Expressions Control Flow Functions Classes and O-O Programming Modules and Packages I/O Useful library modules

This is mostly a language tutorial

Details of various library modules can be found in the Advanced Tutorial No coverage of third-party extensions (XML, GUI, etc...).


Preliminaries

Audience

Experienced programmers who want to know more about Python. I assume familiarity with fundamental CS concepts. Tutorial is somewhat Unix-centric, but almost everything also applies to Windows/Mac.

Approach

No marketing hype or advocacy. Just the facts.

My Background

I was drawn to Python as a C programmer. Primary interest has been using it as an interpreted interface for C programs. Wrote the "Python Essential Reference" in 1999 (New Riders Publishing). All of the material presented here can be found in that source.


Python

What is it?

A freely available interpreted object-oriented scripting language. Often compared to Tcl and Perl, but it has a much different flavor. And a lot of people think it’s pretty cool.

History

Developed by Guido van Rossum in early 1990’s. Named after Monty Python. Influences include ABC, Modula-2, Lisp, C, and shell scripting.

Availability

http://www.python.org It is included in most Linux distributions. Versions are available for Unix, Windows, and Macintosh. JPython. Python interpreter written in Java (http://www.jpython.org).


A Brief Tour


Starting Python

Chances are, Python is already installed on your machine... unix % python Python 1.5.2 (#1, Sep 19 1999, 16:29:25) [GCC 2.7.2.3] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>>

This starts the interpreter and allows you to type programs interactively.

On Windows and Macintosh

Python is launched as an application. An interpreter window will appear and you will see the prompt.

IDLE

An integrated development environment for Python. Available at www.python.org.


Your First Program

Hello World >>> print "Hello World" Hello World >>>

Well, that was easy enough.

Python as a calculator >>> 3*4.5 + 5 18.5 >>>

Basically, interactive mode is just a simple read-eval loop.

Something more complicated >>> for i in range(0,10): ... print i 0 1 2 ... etc ...


Programs and Files

Programs are generally placed in .py files like this # helloworld.py print "Hello World"

To run a file, give the filename as an argument to the interpreter unix % python helloworld.py Hello World unix %

Or you can use the Unix #! trick #!/usr/local/bin/python print "Hello World"

Or you can just double-click on the program (Windows)


Program Termination

Program Execution

Programs run until there are no more statements to execute.

Usually this is signaled by EOF

Can press Control-D (Unix) or Control-Z (Windows) to exit interactive interpreter

Forcing Termination

Raising an exception:

>>> raise SystemExit

Calling exit manually:

import system system.exit(0)


Variables and Expressions

Expressions

Standard mathematical operators work like other languages:

3 + 5 3 + (5*4) 3 ** 2 ’Hello’ + ’World’

Variable assignment a = 4 << 3 b = a * 4.5 c = (a+b)/2.5 a = "Hello World"

Variables are dynamically typed (No explicit typing, types may change during execution).

Variables are just names for an object. Not tied to a memory location like in C.


Conditionals

if-else # Compute maximum (z) of a and b if a < b: z = b else: z = a

The pass statement if a < b: pass # Do nothing else: z = a

Notes:

Indentation used to denote bodies. pass used to denote an empty body. There is no ’?:’ operator.


Conditionals

elif statement if a == ’+’: op = PLUS elif a == ’-’: op = MINUS elif a == ’*’: op = MULTIPLY else: op = UNKNOWN

Note: There is no switch statement.

Boolean expressions: and, or, not if b >= a and b <= c: print "b is between a and c" if not (b < a or b > c): print "b is still between a and c"

Note: &&, ||, and ! are not used.


Basic Types (Numbers and Strings)

Numbers a = 3 # Integer b = 4.5 # Floating point c = 517288833333L # Long integer (arbitrary precision) d = 4 + 3j # Complex (imaginary) number

Strings a = ’Hello’ # Single quotes b = "World" # Double quotes c = "Bob said ’hey there.’" # A mix of both d = ’’’A triple quoted string can span multiple lines like this’’’ e = """Also works for double quotes"""


Basic Types (Lists)

Lists of arbitrary objects a = [2, 3, 4] # A list of integers b = [2, 7, 3.5, "Hello"] # A mixed list c = [] # An empty list d = [2, [a,b]] # A list containing a list e = a + b # Join two lists

List manipulation x = a[1] # Get 2nd element (0 is first) y = b[1:3] # Return a sublist z = d[1][0][2] # Nested lists b[0] = 42 # Change an element

List methods a.append("foo") # Append an element a.insert(1,"bar") # Insert an element len(a) # Length of the list del a[2] # Delete an element


Basic Types (Tuples)

Tuples f = (2,3,4,5) # A tuple of integers g = (,) # An empty tuple h = (2, [3,4], (10,11,12)) # A tuple containing mixed objects

Tuple Manipulation x = f[1] # Element access. x = 3 y = f[1:3] # Slices. y = (3,4) z = h[1][1] # Nesting. z = 4

Comments

Tuples are like lists, but size is fixed at time of creation. Can’t replace members (said to be "immutable") Why have tuples at all? This is actually a point of much discussion.


Basic Types (Dictionaries)

Dictionaries (Associative Arrays) a = { } # An empty dictionary b = { ’x’: 3, ’y’: 4 } c = { ’uid’: 105, ’login’: ’beazley’, ’name’ : ’David Beazley’ }

Dictionary Access u = c[’uid’] # Get an element c[’shell’] = "/bin/sh" # Set an element if c.has_key("directory"): # Check for presence of an member d = c[’directory’] else: d = None d = c.get("directory",None) # Same thing, more compact k = c.keys() # Get all keys as a list


Loops

The while statement while a < b: # Do something a = a + 1

The for statement (loops over members of a sequence) for i in [3, 4, 10, 25]: print i # Print characters one at a time for c in "Hello World": print c # Loop over a range of numbers for i in range(0,100): print i


Functions

The def statement # Return the remainder of a/b def remainder(a,b): q = a/b r = a - q*b return r # Now use it a = remainder(42,5) # a = 2

Returning multiple values def divide(a,b): q = a/b r = a - q*b return q,r x,y = divide(42,5) # x = 8, y = 2


Classes

The class statement class Account: def __init__(self, initial): self.balance = initial def deposit(self, amt): self.balance = self.balance + amt def withdraw(self,amt): self.balance = self.balance - amt def getbalance(self): return self.balance

Using a class a = Account(1000.00) a.deposit(550.23) a.deposit(100) a.withdraw(50) print a.getbalance()

More details to come...


Exceptions

The try statement try: f = open("foo") except IOError: print "Couldn’t open ’foo’. Sorry."

The raise statement def factorial(n): if n < 0: raise ValueError,"Expected non-negative number" if (n <= 1): return 1 else: return n*factorial(n-1)

Uncaught exceptions >>> factorial(-1) Traceback (innermost last): File "<stdin>", line 1, in ? File "<stdin>", line 3, in factorial ValueError: Expected non-negative number >>>


Files

The open() function f = open("foo","w") # Open a file for writing g = open("bar","r") # Open a file for reading

Reading and writing data f.write("Hello World") data = g.read() # Read all data line = g.readline() # Read a single line lines = g.readlines() # Read data as a list of lines

Formatted I/O

Use the % operator for strings (works like C printf)

for i in range(0,10): f.write("2 times %d = %d\n" % (i, 2*i))


Modules

Large programs can be broken into modules # numbers.py def divide(a,b): q = a/b r = a - q*b return q,r def gcd(x,y): g = y while x > 0: g = x x = y % x y = g return g

The import statement import numbers x,y = numbers.divide(42,5) n = numbers.gcd(7291823, 5683)

import creates a namespace and executes a file.


Python Library

Python is packaged with a large library of standard modules

String processing Operating system interfaces Networking Threads GUI Database Language services Security.

And there are many third party modules

XML Numeric Processing Plotting/Graphics etc.

All of these are accessed using ’import’ import string ... a = string.split(x)


Quick Summary

You have seen about 90% of what you need to know

Python is a relatively simple language. Most novices can pick it up and start doing things right away. The code is readable. When in doubt, experiment interactively.

Rest of this tutorial

More details about the language and built-in objects. A little bit about the standard library.


Syntax and Lexical Conventions


Line Structure

Python statements are delimited by newlines print a+3 b = 4*x

Line continuation (\) can be used for long statements a = math.cos(3*(x-n)) + \ math.sin(3*(y-n))

Triple-quoted strings, lists, tuples, and dictionaries can span multiple lines a = [ 4, 5 ] b = { ’x’: 10, ’y’: 20 } c = foo(x,y,z, w,v)

Short statements can be separated by a semicolon (;) a = 3; b = 10*x; print b


Indentation

Indentation used to denote bodies of functions, conditionals, loops, etc.

Amount of indentation is arbitrary Must be consistent within the same block.

if a: statement1 # Consistent indentation statement2 else: statement3 # Inconsistent indentation statement4 # Error!

Can also place on the same line if a: statement1 else: statement2

Tabs

Expanded into number of spaces needed to move to the next column that is a multiple of 8. Example: Tab in column 11 moves ahead to column 16. Try to avoid tabs if you can (use emacs mode).


Reserved Words and Identifiers

Identifiers

Names used to identify variables, functions, classes, modules, etc. May contain A-Z, a-z, 0-9, and _ Must start with a nonnumeric character. Identifiers starting with _ have special meaning in Python (more later).

Python Reserved Words

and elif global or

assert else if pass

break except import print

class exec in raise

continue finally is return

def for lambda try

del from not while

Can’t use these names as identifiers.


Literals

Numbers 12345 # Integer (same as C long type) 0644 # Integer - Octal 0xfea844 # Integer - Hexadecimal 123.45 # Floating point (64-bit double precision) 1.2345e+02 # Floating point - Exponential notation 123456L # Arbitrary precision integer 12 + 34J # Complex number

Strings ’...’ # Single quoted string "..." # Double quoted strings ’’’...’’’, """...""" # Triple-quoted strings r’...’ # Raw string

Triple quoted strings capture all text verbatim until matching end quotes. Raw strings preserve backslash characters (\). Not interpreted as escape codes. Adjacant strings are concatenated.


Character Escape Codes

Python supports the standard character escape code \ Newline Continuation \\ Backslash \’ Single quote \" Double quote \a Bell \b Backspace \e Escape \n Line feed \v Vertical tab \t Horizontal tab \r Carriage return \0 Null \0XX Octal character value \xXX Hex character value

Note: Escape codes are not interpreted in raw strings

a = r"\n\r"


Operators and Special Symbols

Operator Tokens

+ - * **

/ % << >>

& | ^ ~

< > <= >=

== != <>

Note: C operators such as ++,--, +=, and -= are not supported.

Other valid symbols ( ) [ ] { } , : . ‘ = ; ’ " # \

These symbols may not appear in a program except inside a quoted string @ $ ?


Documentation Strings

First statement of function, class, or module can be a string

This is known as a documentation string Example:

def factorial(n): """This function computes n factorial""" if (n <= 1): return 1 else: return n*factorial(n-1)

Accessing documentation strings

Found by looking at __doc__ attribute.

>>> print factorial.__doc__ This function computes n factorial >>>

Code browsers and IDEs often look at doc-strings to help you out. And it’s considered to be good Python programming style.


Built-in Types


Overview

The Python interpreter provides a number of built-in datatypes

Integers Long integers Floating point numbers Complex numbers Strings Lists Tuples Dictionaries Functions Classes Instances Modules Code Execution frames

Some of these are obvious (e.g., numbers)

Others pertain to the operation of the interpreter (e.g., execution frames)

We’ll discuss the most common ones


The None Type

Python’s version of NULL

Written as "None"

a = None

Doesn’t represent anything (an empty value) Evaluates to false in boolean expressions. Isn’t very exciting in general (even though it’s pretty useful).


Integer Types

Integers a = 42 b = -371 c = 0xffe8ab

Represent values in the range of machine precision. The same as the C long datatype (32 or 64 bits depending on CPU and compiler).

Long Integers d = 42003882399L e = -388298371662776362L

Arbitrary precision integers. Size constrained only by available memory.

Note: All integers are signed


Floats and Complex Numbers

Floating Point a = 42.37 b = -2.3477e+30

Represented as IEEE 754 64-bit double precision (the ’double’ type in C).

Complex Numbers c = 4.6 + 5.2J

Represented as a pair of floating point numbers Available attributes

c.real # Real component c.imag # Imaginary component


Strings

Representation

Strings are sequences of characters May contain embedded nulls and binary data. Strings are immutable

Common Operations s[i] # Return the ith character s[i:j] # Extract a substring len(s) # String length

Examples s = "Hello World" x = s[4] # x = ’o’ y = s[0:6] # y = ’Hello’ print len(s)


Lists

Lists are ordered sequences of arbitrary objects

May contain mixed types. Lists are mutable (can be changed).

List Methods s.append(x) # Append element x to a list s.extend(r) # Appends list r to the end of s s.count(x) # Count occurrences of x in s s.index(x) # Return smallest i where s[i] == x s.insert(i,x) # Insert an element into a list s.pop([i]) # Pops element i from the end of the list. s.remove(x) # Searches for x in the list and removes it s.reverse() # Reverses the items of s in place s.sort([cmpfunc]) # Sorts the items of s in place.

Examples a = [20,45,10,5,3,99] a.append(103) # a = [20,45,10,5,3,99,103] a.extend([1,2,3]) # a = [20,45,10,5,3,99,103,1,2,3] a.count(3) # returns 2


Tuples

Tuples are a fixed sequence of arbitrary objects

Size and contents are fixed at time of creation. Immutable There are no methods, but the following functions are useful

list(s) # Convert a sequence to a list tuple(s) # Convert a sequence to a tuple

Other useful tuple properties

Assignment to multiple values

t = (4,5,6) x,y,z = t # Expands contents into variables

Multiple function return values Can be used as dictionary keys

d = { } d[1,2,3] = "Foo" # Same as d[(1,2,3)] = "Foo" d[1,0,2] = "Bar" # Same as d[(1,0,2)] = "Bar"


Dictionaries

Dictionaries are associative arrays (hashes)

Provide a mapping between keys and objects. Keys may be any immutable object (strings, tuples, numbers) Unordered (unlike sequences)

Methods d.clear() # Remove all items d.copy() # Makes a copy of the dictionary d.has_key(k) # Tests for existence of a key d.items() # Return a list of (key,value) pairs d.keys() # Return a list of keys d.values() # Return a list of values d.update(b) # Adds all objects in dictionary b to d d.get(k [,f]) # Returns d[k] if found. Otherwise, return f.

Example d = { ’name’: ’Dave’, ’uid’:104 } k = d.keys() # Returns [’name’,’uid’] v = d.values() # Returns [’Dave’,104] i = d.items() # Returns [(’name’,’Dave’),(’uid’,104)]


Callable Types

Functions

Defined with the def statement

def foo(a): print "foo", a

Functions are just like any other object a = foo # Variable assignment a(3) # Call a function b = { } b[’bar’] = foo # Place in a dictionary b[’bar’](10) # Call the function

There are a number of callable objects

User defined functions Built-in functions (implemented in C) Class methods. Classes Consult reference for differences


Classes, Instances, and Modules

Classes

Created by the class statement.

class Account:

Class instances

Created by calling a class constructor.

a = Account()

Modules

Created by the import statement

import spam


The __dict__ attribute

Classes, instances, and modules are built using dictionaries

The dictionary is found in the __dict__ attribute

import string print string.__dict__ # Print out the module dictionary print Account.__dict__ # Output the dictionary of a class a = Account() print a.__dict__ # Output the dictionary of an instance

Attribute lookup and dictionaries

All operations of the form obj.name are translated into dictionary operations.

a = obj.name # a = obj.__dict__[name] obj.name = x # obj.__dict__[name] = x

Caveat: Different types may perform additional processing behind the scenes. Example, lookups on a class instance will search through the dictionaries of base classes.


Variable Assignment

Variable assignment is a naming operation a = 42

This creates an integer object with value 42. "a" is a name that refers to the object.

Reference Counting

All objects are reference counted. Variable assignment is nothing more than a reference copy (increases the reference count). Example:

b = a

Now "a" and "b" are both names for the exact same object.


Variable Assignment

Changes to a variable result in new objects a = a + 1

This creates a new object with value a+1. The name "a" is set to refer to the new object (reference count of old object decremented). The value of the old object is not changed! Note: This is different than how it works in a language like C. Variables do not represent fixed memory locations. This is one reason why there are no ++,--,+= operators.

Strange behavior of reference counting a = [3,4,5,6] b = a a[2] = -10 print b

What is the output?

[ 3, 4, -10, 6]

Why? "b" refers to the exact same object as "a".


Reference Counting and Garbage Collection

Reference counts are increased by...

Variable assignment Inclusion of an object in a container (list, tuple, dictionary, etc...).

Reference counts are decreased when...

Local reference goes out of scope (variable name is destroyed). Variable name is bound to another object. Object is removed from a container. The variable name is explicitly destroyed using ’del’ statement

del a

Objects destroyed when reference count reaches zero

Well, unless there are circular references.


Object Identity and Type

Useful functions for finding out information about objects

type(obj) - Returns the type of an object (which itself is a special object)

a = [1,2,3] t = type(a) # t = ListType

id(obj) - Returns the identity of an object (an integer)

i = id(a)

isinstance(obj,type) - Tests the type of an object

import types if isinstance(a,types.ListType): print "a is a list" else: print "a is not a list"


String Conversion

str(obj)

Creates a printable string (same output as produced by print)

repr(obj)

Creates a string representation that can be converted back into the original object Shorthand version is specified with backquotes ‘obj‘

s = repr(obj) a = ‘obj‘ # Same thing

Note: In general, eval(repr(obj)) = obj Often str() and repr() are the same. However, this isn’t guaranteed (e.g., strings).

a = "Hello World\n" print str(a) # Prints "Hello World" (with newline) print repr(a) # Prints "Hello World\012"


Operators and Expressions


Operators and Types

Most Python operations fall into one of these categories

Numbers Sequences Mappings

And these are closely related to the built-in types

Numbers (integers, floats, long integers, complex) Sequences (strings, lists, tuples) Mappings (dictionaries, classes, modules)

User-defined classes can also emulate built-in types

Operator overloading (later)


Operations on Numbers

Operations supported by all numbers x + y Addition x - y Subtraction x * y Multiplication x / y Division x ** y Power x % y Modulo (x mod y) -x Negation +x Unary plus

Integer operations x << y Left shift x >> y Right shift x & y Bitwise and x | y Bitwise or x ^ y Bitwise exclusive or ~x Bitwise negation


Operations on Numbers

Functions applicable to numbers abs(x) Absolute value divmod(x,y) Returns (int(x/y), x % y) pow(x,y [,m]) Returns (x ** y) % m round(x [,n]) Round to n digits of precision (floats only)

Comparisons x < y Less than x > y Greater than x == y Equal to x != y Not equal to (same as <>) x >= y Greater than or equal to x <= y Less than or equal to


Additional Details on Numbers

Type Coercion

Numerical operations only work on numbers of the same type Both operands are converted to the same type if possible. Example: in 3*4.5, "3" is converted to a float before the multiply.

Integer truncation

Operations on integers return integers. Division truncates down. So 7/4 -> 1.

Rounding

The round() function rounds away from 0 for decimals ending in 5. Example: round(0.5) -> 1.0, round(-0.5) -> -1.0


Operations on Sequences

The following operations apply to all sequences

Strings, lists, and tuples.

s + r Concatenation s * n, n * s Makes n copies of s where n is an integer s % d String formatting (s is a string) s[i] Indexing s[i:j] Slicing x in s Membership test x not in s Membership test len(s) Length min(s) Minimum item max(s) Maximum item

Examples a = [1,2,3] b = [5,6] c = a + b # c = [1,2,3,5,6] d = a * 2 # d = [1,2,3,1,2,3] e = c[2:4] # e = [3,5]


Slices

The slicing operator s[i:j]

Extracts all elements s[n] where i <= n < j

a = [0,1,2,3,4,5,6,7,8] b = a[3:6] # b = [3,4,5]

If either index is omitted, beginning or end of sequence is assumed

c = a[:3] # c = [0,1,2] d = a[5:] # d = [5,6,7,8]

Negative index is taken from the end of the sequence

e = a[2:-2] # e = [2,3,4,5,6] f = a[-4:] # f = [5,6,7,8]

No indices just makes a copy (which is sometimes useful)

g = a[:] # g = [0,1,2,3,4,5,6,7,8]


List Operations

The following operations are applicable only to lists s[i] = x Element assignment s[i:j] = r Slice assignment del s[i] Element deletion del s[i:j] Slice deletion

Examples a = [0,1,2,3,4,5,6] a[3] = -10 # a = [0,1,2,-10,4,5,6] a[1:4] = [20,30,40,50] # a = [0,20,30,40,50,4,5,6] a[1:5] = [100] # a = [0,100,4,5,6] del a[1] # a = [0,4,5,6] del a[2:] # a = [0,4]

Note:

Strings and tuples are immutable and can’t be modified.


Sequence Comparison

Comparison Operators s < r, s > r s <= r, s >= r s == r, s != r

Sequences are compared element by element. Result is determined by first mismatch or when end of sequence is reached. A subsequence always evaluates as <

Examples: The following relationships evaluate as true "Hello" < "World" "Hello" < "HelloWorld" [5,4,3,2,1] > [1,2] [1,2,3,4] < [1,2,3,4,5,6]


Dictionary and Attribute Operators

Operations on Dictionaries x = d[k] Lookup by key d[k] = x Assignment del d[k] Delete an item len(d) Number of items in dictionary

Examples

a = d[’name’] d[’email’] = "beazley@cs" del d[’phone’]

Attribute Operator (.)

The dot (.) operator is used to access the attributes of an object

foo.x = 3 print foo.y a = foo.bar(3,4,5) a = foo.bar(3,4,5).spam

Usually this corresponds to a dictionary lookup on __dict__


Type Conversion

The following functions convert between types int(x) Convert x to an integer long(x) Convert x to a long integer float(x) Convert x to a float complex(r,i) Creates a complex number str(x) Create a string repr(x) Create an expression string eval(str) Evaluate a string and return an object tuple(s) Convert sequence s to a tuple list(s) Convert sequence s to a list chr(x) Convert an integer to a character ord(x) Convert single characte to an integer code hex(x) Convert x to a hex string oct(x) Convert x to an octal string

str vs. repr

The str() function creates a string suitable for printing The repr() function creates a string that can be evaluated to produce the original object


Boolean Expressions

Boolean Expressions

x and y x or y not x

Short-circuit behavior

x and y only evaluates y if x is true. x or y only evaluates y if x is false.

Truth values

Non-zero values and non-empty objects evaluate as true. None and zero-length strings, lists, tuples, and dictionaries evaluate as false.


Object Equality and Identity

x == y

Tests the values of x and y for equality Sequences are equal if every element is equal. Dictionaries are equal if they have the same set of keys and all objects with same key have the same value.

x is y

Tests if x and y are exactly the same object.

x is not y

Tests if x and y are different objects

Notes

x == y may be true while x is not y Comparisons between non-compatible types may not generate an error, but the result is arbitrary (although always thesame for the same two objects)


Control Flow and Exceptions


Basic Control Flow Statements (Revisited)

if-elif-else if expr : statements elif expr : statements else: statements

while while expr : statements

for for item in sequence : statements


Break and Continue

The break statement

Exits the enclosing loop

while 1: cmd = raw_input("Enter command > ") if not cmd: break # Process command ...

The continue statement

Returns to the top of the loop (skips the rest of the loop body)

# Print non-negative list elements for item in s: if item < 0: continue print item


Loops and Else

Little known fact...

Loops can have an else statement attached to them. Code executes only if the loop does not exit via break.

Example: Searching for something in a list

# An example lacking in style match = None for i in s: if i == name: match = 1 break if not match: raise NameError, "Bad name"

# Loop-else clause example for i in s: if i == name: break else: raise NameError, "Bad name"

Note: could also just do this

if not name in s: raise NameError, "Bad name"


For Loops and Tuples

The for statement iterates over the elements of a sequence for i in [3,4,5,6,7]: print i

In this case, i is assigned to 3, 4, 5, ...

If the sequence contains tuples of equal size, can assign to multiple names for x,y in [(2,3),(4,5),(5,6),(7,8)]: plot(x,y)

In this case, the loop executes plot(2,3), plot(4,5), plot(5,6), etc... Again, this only works if all elements are tuples of same size.


range and xrange

The range([start,] stop [,stride]) function constructs a list of integers a = range(100) # a = [0,1,2,3,4,..., 99] b = range(10,100) # b = [10,11,12,..., 99] c = range(10,100,5) # c = [10,15,20,..., 95] d = range(100,0,-5) # d = [100,95,90,..., 5]

This is commonly used in looping constructs

for i in range(0,100): statements

But it consumes a lot of memory for large ranges

xrange([start,] stop [,stride]) computes its values a = xrange(0,1000000000,5) b = a[198763233] for i in a: statements

In general, xrange() is better if looping over many values


Exceptions

try-except try: f = open(filename) ... except IOError, e: print "Unable to open",filename,e

Different types of exceptions have names (e.g., "IOError") If an exception of that type occurs in try-block, control passes to the matching except-block. Exceptions have values. Placed into optional variable supplied to except.

raise raise ValueError, "Expected a positive value"

Raises an exception of a given type (e.g., "ValueError"). Accepts an optional value. This value gets passed back to a matching except statement. If exception is uncaught, program terminates with an error.


Built-in Exceptions FloatingPointError IndexError OverflowError KeyError ZeroDivisionError MemoryError AssertionError NameError AttributeError RuntimeError IOError SyntaxError OSError SystemError EOFError SystemExit ImportError TypeError KeyboardInterrupt ValueError

Contained in the ’exceptions’ module. The names are relatively straightforward. Read docs for gory details about each one.


Multiple Exceptions

Catching more than one exception type is easy try: statements ... except IOError,e: # Handle I/O error ... except MemoryError,e: # Handle out of memory error ... except (IndexError, KeyError), e: # Handle either an IndexError or a KeyError ...

Ignoring an exception try: statements except IOError: pass


Specifying Cleanup Actions

finally

Used to supply code that must execute no matter what happens

f = open(filename) try: statements finally: f.close()

finally is not used to catch errors. It only specifies a cleanup action. Executes regardless of whether or not an exception occurs. finally and except can not appear together in the same try statement.

try-else try: statements except IOError,e: statements else: statements

else clause only executes if no exception occurred (and it must appear last).


Exception Hierarchy

Exceptions are really organized into a hierarchy

Exception | StandardError | ... ArithmeticError LookupError FloatingPointError IndexError OverflowError KeyError ZeroDivisionError MemoryError AssertionError NameError AttributeError RuntimeError EnvironmentError SyntaxError IOError SystemError OSError SystemExit EOFError TypeError ImportError ValueError KeyboardInterrupt


Exception Hierarchy

Can catch general categories of errors try: statements except LookupError,e: # Handle IndexError or KeyError


Creating New Exceptions

New exceptions are defined as classes class NetworkError(Exception): def __init__(self,args=None): self.args = args # Raise a user defined exception def lookup(name): ... raise NetworkError, "Bad hostname" # Catch a user defined exception try: statements except NetworkError, e: print e.args

User defined exceptions should inherit from Exception (although not enforced).


User Defined Exception Hierarchies

You can also define exception hierarchies using inheritance class NetworkError(Exception): ... class HostnameError(NetworkError): ... class TimeoutError(NetworkError): ...

Now in your application... try: statements except HostnameError: # Handle a HostnameError try: statements except NetworkError: # Handle any network error


Assertions

The assert statement def divide(a,b): assert b != 0, "Can’t divide by zero" return a/b

Failed assertions result in an AssertionError exception.

__debug__

Assertions are only enabled if the special symbol __debug__ is defined. Internally, assert is expanded into this:

if __debug__: if not test: raise AssertionError, data

By default, __debug__ is set 1. If you run python with -O option, it is set to 0. You can inspect the value of __debug__ for other purposes if you want.


Functions


Functions Revisited

def statement

Defines a new function

def fname ( args ): statements

Example:

def divide(a,b): return a/b

Calling a function x = divide(7,3)

Calling a function with keyword arguments x = divide(b=3, a=7)

Exact same result as before, argument names specified explicitly.


Default Arguments

Function definition with default values def spam(a,b, c=37, d="Guido"): statements ... spam(2,3) spam(2,3,4) spam(2,3,4,"Dave") spam(2,3,d="Dave")

General Comments

No non-optional arguments can follow an optional argument. Can mix non-keyword and keyword arguments together when calling (with care). Value of an optional argument is determined at time of function definition.

x = 10 def spam(a = x): print a x = 20 spam() # Produces ’10’


Variable Length Arguments

A function accepting variable number of arguments def printf(fmt, *args): print fmt % args

Extra arguments are passed as a tuple in args

Accepting an arbitrary set of keyword arguments def foo(**kwargs): print kwargs

kwargs is a dictionary mapping argument names to values.

Accepting both positional and keyword arguments def foo(arg1, *vargs, **kwargs): statements foo(1,2,3,4,name="Guido") # arg1 = 1, vargs = (2,3,4), kwargs= {’name’:’Guido’}


Scoping Rules

Local vs. Global Scope

Functions execute within a local and global scope. Local scope is just the function body. Global scope is the module in which the function was defined. Variable lookups first occur in local scope, then in global scope. All variable assignments are made to the local scope.

x = 4 def foo(): x = x + 1 # Creates a local copy of x foo() print x # Outputs ’4’

The global statement

Declares a variable name to refer to the global scope.

def foo(): global x x = x + 1 # Modifies the global x


Parameter Passing and Return Values

Parameters are passed by reference

This has certain implications for lists, dictionaries, and similar types

def foo(x): x[2] = 10 a = [1,2,3,4,5] foo(a) print a # Outputs ’[1, 2, 10, 4, 5]’

Return values

Values are returned as a tuple of results.

def foo(): ... return (x,y,z) d = foo() # Gets a tuple with three values (a,b,c) = foo() # Assigns values to a, b, and c. a,b,c = foo() # Same thing


The apply Function

The apply(func [,args [,kwargs]]) function can be used to call a function

args is a tuple of positional arguments. kwargs is a dictionary of keyword arguments.

# These two statements are exactly the same foo(3,"x", name="Dave", id=12345) apply(foo, (3,"x"), {’name’:’Dave’, ’id’:12345 })

Why would you use this?

Sometimes it is useful to manipulate arguments and call another function

def fprintf(file,fmt, *args): file.write(fmt % args) def printf(fmt, *args): apply(fprintf, (sys.stdout,fmt)+args)


The lambda operator

In Python, functions are first-class objects

This means you can manipulate functions like all other types (integers, floats, lists, etc...)

def add(a,b): return a+b a = add print a(3,4)

You can also create anonymous functions with lambda a = lambda x,y: x+y ... b = a(3,4)

This is particularly useful for callback functions However, lambda can only be used to specify simple expressions.


The map Function

map(func, s) applies a function to each element of a sequence a = [1,2,3,4,5,6] def foo(x): return 3*x b = map(foo,a) # b = [3,6,9,12,15,18]

Alternatively...

b = map(lambda x: 3*x, a) # b = [3,6,9,12,15,18]

Special cases

map can be applied to multiple lists.

b = map(func, s1, s2, ... sn)

In this case, func must take n arguments. If the function is None, the identity function is assumed.


The reduce Function

reduce(func, s) collects data from a sequence and returns a single value a = [1,2,3,4,5,6] def sum(x,y): return x+y b = reduce(sum, a) # b = (((((1+2)+3)+4)+5)+6) = 21

Alternatively...

b = reduce(lambda x,y: x+y, a)

The function given to reduce must accept two arguments and return a single result (suitable for reuse as one of thearguments).


The filter Function

filter(func, s) filters the elements of a sequence a = [1,2,3,4,5,6] b = filter(lambda x: x < 4, a) # b = [1,2,3]

If func is None, filter() returns all of the elements that evaluate to true.


Exec, Eval, and Execfile

The eval function

Evaluates a string as a Python expression

a = eval("3*math.sin(3.5+x)+7.2")

The exec statement

Executes a string containing arbitrary Python code

a = [3,5,10,13] exec "for i in a: print i"

The execfile function

Executes the contents of a file

execfile("foo.py")

Note: exec is a statement. eval and execfile are functions


Classes and Objects


Preliminaries

The holy pillars of OO

Encapsulation Inheritance Polymorphism

Or, if you prefer

A class is just a data structure with some functions called "methods." You can extend a class by creating a new class that "inherits" stuff defined in the old one. Behavior of an "object" is determined at run-time (depends on what kind of object it is)

Unfortunately, OO has a lot of baggage and terminology

Fortunately, Python is reasonably comprehensible. I’m not going to dwell on terminology or methodology. And for the record, I personally find Python to be infinitely more pleasant than C++. So there.


Classes

The class statement class Account: def __init__(self, initial): self.balance = initial def deposit(self, amt): self.balance = self.balance + amt def withdraw(self,amt): self.balance = self.balance - amt def getbalance(self): return self.balance

A class merely defines a set of attributes and methods for a set of objects. To use it, you have to create some class instances.


Classes Instances

Creating a new instance a = Account(1000.00) b = Account(500.00)

Method invocation a.deposit(500.00) a.deposit(23.45) a.withdraw(78.20) print a.getbalance()

Attribute access print a.balance a.balance = 500000.00

Destruction

Objects destroyed when reference count drops to zero. Usually this is automatic, but you can also do this

del a


More on Class Members

Methods class Account: ... def deposit(self,amt): self.balance = self.balance + amt

First argument to a method always refers to a specific instance (self) Operations on an object must explicitly refer to the self parameter

self.balance = self.balance + amt # This updates the object balance = balance + amt # This is name error (balance unknown)

For the C++ impaired, self is like the ’this’ variable.

Variables class Foo: a = 42

Variable is shared by all instances Like a static member variable in C++


Special Methods

Certain methods have special meaning to the interpreter

These are denoted with leading and trailing underscores (__) There are a few dozen special methods than can be defined.

__init__

Called to initialize a newly created instance.

def __init__(self,initial): self.balance = initial

__del__

Called before an instance is about to be destroyed.

def __del__(self): print "Ay!!!! I’m being killed"

Note: it is rarely necessary to define __del__ Only needed if you need to perform cleanup actions (e.g., close a network connection)


Inheritance

Specification class A: varA = 42 def method1(self): print "Class A: method1" class B(A): # Inherits from class A varB = 37 def method2(self): print "Class B: method2"

Use

It works just like you would expect

b = B() b.method2() # Invokes B.method2() b.method1() # Invokes A.method1() print b.varA # Outputs ’42’


Inheritance

Calling Methods in the Base Class class A: def foo(self): print "Class A: foo" class B(A): # Inherits from class A def foo(self): print "Class B: foo" A.foo(self) # Call foo-method in class A

Most commonly used in object initialization class B(A): def __init__(self,args): # Initialize the base class A.__init__(self) # Initialize myself ...


Multiple Inheritance

Specification class A: def foo(self): print "Class A: foo" class B: def foo(self): print "Class B: foo" def bar(self): print "Class B: bar" class C(A,B): # Inherits from class A and class B def spam(self): print "Class C: spam"

Search for attributes is done using depth-first search in the order of base-classes

c = C() c.foo() # A.foo c.bar() # B.bar


Information Hiding

All attributes of a class are public

No notion of private or protected members.

However, there is a name-mangling trick.

All names in a class that have leading double-underscores are mangled. Identifiers of the form __name become _Classname__name

class A: def __init__(self): self.__X = 3 # self._A__X = 3 class B(A): def __init__(self): self.__X = 42 # self._B__X = 42

This is useful if you need to have private data that is "invisible" to derived classes. Note: can still access if you know the name mangling trick

class B(A): def foo(self): print self._A__X # print A’s private data (shame!)


Operator Overloading

Operators and Special Methods

All operators are mapped to special class methods. You are free to implement these

Example: Make a class behave like a list class A: def __getitem__(self,index): # x = obj[n] ... def __setitem__(self,index,val): # obj[n] = val ... def __delitem__(self,index): # del obj[n] ... def __len__(self): # len(obj) ... def __getslice__(self,i,j): # x = obj[i:j] ... def __setslice__(self,i,j,val): # obj[i:j] = val ... def __delslice__(self,i,j): # del obj[i:j] ...

Consult a reference for gory details


Classes, Types, and Membership Tests

Objects defined by a class are not types

All classes have a type of ClassType All instances have a type of InstanceType Thus, type(a) == type(b) for any two instances of any classes.

Membership tests

isinstance(obj,c). Tests if obj is an instance of class c.

class A: pass class B(A): pass a = A() b = B() isinstance(a,A) # Returns 1 isinstance(b,A) # Returns 1 (B is a subclass of A) isinstance(a,B) # Returns 0

issubclass(c1,c2). Tests if class c1 is a subclass of class c2.

issubclass(B,A) # Returns 1 issubclass(A,B) # Returns 0


Final Words on Classes

Classes and instances are all built using dictionaries

Take a look at the __dict__ attribute

a = Account(100) print a.__dict__ # Modify an object through it’s dictionary a.__dict__[’foo’] = 78

Just about everything revolves around dictionary lookup. Example: attribute lookup, obj.name

1. Look for name in the instance dictionary for obj. 2. Look for name in the class dictionary. 3. Look for name in dictionaries for base-classes.

Separation of types and classes is a topic of debate

There is a move to unify types and classes. Specifically, built in types would become more class-like and classes would define types. This is not yet the case however.


Modules and Packages


Modules

Modules allow you to organize your code

Place related functions into a file such as ’spam.py’

# spam.py a = 37 def foo(): statements def bar(): statements

Use import to load and execute

import spam print spam.a spam.foo()

Comments

A module defines a namespace This namespace is the global namespace for everything in the module. Modules are only loaded once (even for repeated import statements).


Variations on Import

from module import

This imports selective symbols from a module into the current namespace

from spam import foo, bar foo() # No longer need module prefix

Or you can import all of the symbols into the current namespace

from spam import *

Note: But this does not import symbols starting with an underscore (_)

reload module

This reloads a module. For example:

reload spam

Useful for debugging, but somewhat problematic if you aren’t careful. References to classes and other objects in the old module are not updated. Generally doesn’t work with extension modules (written in C).


Module Search Path

To look for modules on import...

The interpreter searches the directories in sys.path

[’’, ’/usr/local/lib/python1.5/’, ’/usr/local/lib/python1.5/plat-sunos5’, ’/usr/local/lib/python1.5/lib-tk’, ’/usr/local/lib/python1.5/lib-dynload’, ’/usr/local/lib/python1.5/site-packages’, ’/usr/local/lib/site-python/’]

New directories can be added to the path as needed

import sys sys.path.append("/home/beazley/python")


Module Loading

On "import spam", the interpreter looks for the following files:

spam.so, spammodule.so, spammodule.sl, or spammodule.dll (compiled extensions) spam.pyo (optimized bytecode, only with -O option) spam.pyc (bytecode) spam.py (source)

Creation of .pyc and .pyo files

Created automatically by the interpreter on module import Contain compiled byte-code for the module. .pyo files are optimized in the sense that they do not contain debugging information. Regenerated if the modification date of the matching .py file is newer. For distribution, only the .pyc or .pyo files are needed (the .py file can be omitted).


Packages

Motivation

Sometimes it is useful to break an application into submodules Example: A Graphics package

Graphics primitives 2d plotting 3d plotting Image file formats

Packages can be used to create a hierarchical module namespace Graphics Graphics.Primitives Graphics.Primitives.lines Graphics.Plot2D.xyplot Graphics.Formats.png Graphics.Formats.jpeg ...


Packages

How to create a package

Organize .py files in a subdirectory structure Include special __init__.py files.

Graphics/ __init__.py Primitives/ __init__.py lines.py fill.py ... Plot2D/ __init__.py xyplot.py logplot.py ... Formats/ __init__.py png.py jpeg.py ...


Packages

Importing from a package

import Graphics.Primitives.fill

Graphics.Primitives.fill.floodfill(img,x,y,color)

from Graphics.Primitives import fill

fill.floodfill(img,x,y,color)

from Graphics.Primitivies.fill import floodfill

floodfill(img,x,y,color)

Note:

On import, the __init__.py files in each subdirectory are executed. Can be used to perform initialization of a subcomponent.


Input/Output


Command Line Options

Reading command line options

Passed as a list in sys.argv

# printopt.py import sys for i in range(0,len(sys.argv)): print "sys.argv[%d] = %s" % (i, sys.argv[i])

Example:

unix % python printopt.py foo bar 34 -p sys.argv[0] = printopt.py sys.argv[1] = foo sys.argv[2] = bar sys.argv[3] = 34 sys.argv[4] = -p

Note: program name is passed in sys.argv[0]


Environment Variables

Reading Environment Variables

Use the os module

import os path = os.environ[’PATH’] user = os.environ[’USER’] editor = os.environ[’EDITOR’] ...

See the Advanced Python Tutorial.


Files

open(filename [, mode])

Opens a file and returns a file object. Mode is one of the following

"r" - Open for reading (the default) "w" - Open for writing (destroys old contents if any) "a" - Open for append "r+" - Open for read/write (updates) "w+" - Open for read/write (destroys old contents first).

An optional "b" may be appended to specify binary data. Example:

f = open("foo") # Open for reading g = open("bar","w") # Open for writing h = open("spam","rb") # Open for reading (binary)


File Methods

The following methods may be applied to a file object f.read([n]) Read at most n bytes f.readline() Read a single line of input f.readlines() Read all input lines f.write(s) Write string s f.writelines(l) Write all strings in list l. f.close() Close the file f.tell() Return the current file pointer f.seek(offset [,where]) Seek to a new position f.isatty() Returns 1 if an interactive terminal f.flush() Flush output buffers f.truncate([n]) Truncate file to at most n bytes f.fileno() Return integer file descriptor

Notes

Line-oriented operations are aware of newline differences (Unix vs. Windows)


Standard Input, Output, Error

Standard Files

Found in the sys module sys.stdin - Standard input sys.stdout - Standard output sys.stderr - Standard error

Many built-in functions use these

print outputs to sys.stdout input() and raw_input() read from sys.stdin

s = raw_input("type a command : ") print "You typed ", s

Error messages and the interactive prompts go to sys.stderr

You can replace these files with other files if you want import sys sys.stdout = open("output","w")


Formatted I/O

The % operator for strings print "%d %f %s" % (42, 3.1415, "Hello") s = "Your name is ’%s %s’" % (firstname, lastname)

Each of the format codes (preceded by %) are replaced by elements of a tuple Works just like the C sprintf function

Format codes d,i Decimal integer u Unsigned integer o Octal integer x Hex integer X Hex integer (uppercase letters) f Floating point as [-]m.dddddd e Floating point as [-]m.dddddde+xx E Floating point as [-]m.ddddddE+xx g,G Use e or f depending on the size of the exponent. s String or any object c Single character % Literal character


Python Library


The Python Library (Recap)

Python is packaged with a large library of modules

String processing Operating system interfaces Networking Threads GUI Database Language services Security.

Discussion of these is beyond scope of this tutorial

But a brief survey of the more popular modules is given here.


Built-in Functions

Contained in the __builtin__ module (always available) abs() eval() intern() open() str() apply() execfile() isinstance() ord() tuple() callable() filter() issubclass() pow() type() chr() float() len() range() vars() cmp() getattr() list() raw_input() xrange() coerce() globals() locals() reduce() compile() hasattr() long() reload() complex() hash() map() repr() delattr() hex() max() round() dir() id() min() setattr() divmod() input() oct() slice()

Most of these have already been covered


The sys module

Functions and variables used by the interpreter sys.argv # Command line arguments sys.path # Module search path sys.maxint # Maximum integer value sys.stdin # Standard input sys.stdout # Standard output sys.stderr # Standard error sys.ps1 # Interactive interpreter prompt (>>>) sys.ps2 # Interactive interpreter prompt (...) sys.exc_info() # Information about last raised exception sys.exit(n) # Exit from the interpreter

In addition, the module provides version and installation information

sys.version # Version string sys.prefix # Installation prefix sys.copyright # Copyright message sys.executable # Name of executable


The string module

Various string processing functions string.atof(s) # Convert to float string.atoi(s) # Convert to integer string.atol(s) # Convert to long string.count(s,pattern) # Count occurrences of pattern in s string.find(s,pattern) # Find pattern in s string.split(s, sep) # String a string string.join(strlist, sep) # Join a list of string string.replace(s,old,new) # Replace occurrences of old with new

Examples s = "Hello World" a = string.split(s) # a = [’Hello’,’World’] b = string.replace(s,"Hello","Goodbye") c = string.join(["foo","bar"],":") # c = "foo:bar"


The math module

All of the usual math operations math.acos(x) math.hypot(x,y) math.asin(x) math.ldexp(x,i) math.atan(x) math.log(x) math.atan2(x,y) math.log10(x) math.ceil(x) math.modf(x) math.cos(x) math.pow(x,y) math.cosh(x) math.sin(x) math.exp(x) math.sinh(x) math.fabs(x) math.sqrt(x) math.floor(x) math.tan(x) math.fmod(x,y) math.tanh(x) math.frexp(x)

Plus two constants math.pi math.e

Note: the cmath module contains similar functions for complex numbers


The re module

Regular Expression Matching re.compile(pattern,flags) # Compile regular expression string re.search(pattern, str) # Search for first match of pattern re.match(pattern,str) # Checks if str matches pattern re.sub(pattern, repl, str) # Replace pattern with repl

Patterns text Matches literal text . Match any character except newline ^ Match the start of a string $ Match the end of a string * Match zero of more repetitions + Match one or more repetitions ? Match zero or one repetitions [...] Match a set of characters (...) Match expression in parenthesis as a group A|B Match A or B


The re module (cont)

Matches

When a regular expression match is found, a special match object is returned Methods

m.group([group1, group2, ...]) # Return one or more subgroup matches m.groups() # Returns a tuple of all subgroups

Examples # Get HTML title m = re.search("<title>(.*)</title>", s, re.IGNORECASE) if m: title = m.group(1) else: title = None

Recommended

Mastering Regular Expressions by Jeffrey Friedl, O’Reilly & Associates


Other Useful Modules

os module

Access to operating system services (files, processes, environment)

socket module

Low-level access to the network.

threading module

Support for threaded programs.

Covered in detail in the advanced tutorial


Conclusions


Conclusions

Final Comments

This tutorial has focused almost exclusively on the language itself Most of Python’s functionality is found in the Python library. This is covered in the Advanced Python Programming tutorial.

Additional Resources

Learning Python. Provides excellent introductory information. Python Essential Reference. (Shameless plug) Online language reference (www.python.org).

Acknowledgements

Guido van Rossum David Ascher Paul Dubois


introduction to python - dabeazdabeaz.com/python/tutorial/beazley_intro_python/intropy.pdf ·...

Documents