Retroweb services made easy

Sébastien Pierre, Datalicious@Montréal Python 7, May. 2009

www.datalicious.ca | github.com/sebastien/retro

Why anotherFramework

Toolkit ?

3

Things have Changed

The 90s“ Golden Spinning Logo Era”

Web = Pages + Animated GIFs

4

Things have Changed

The 00s“ Social Startup AJAX 2.0 Era”

Web = Model +View + Controller

5

Things have Changed

Now“ The Web of Data Era*”

* aka “ semantic linked RDF data 3.0” ;)

Web = Data + Interfaces

6

Shifting our Perspective

Web services are the new interfaces

for the “ web of data”

7

On Retro : Key Concepts

Web services are APIs exposed over HTTP

Web Service = HTTP data bus

8

Retro : An examplePopulation GDP Land Area (hectares)

British Columbia 4,113,487 $164,583.00 91,837,000

Yukon 30,372 $1,452.00 47,890,000

Alberta 3,290,350 $187,493.00 61,400,000

Northwest Territories 41,464 $4,138.00 112,984,000

Saskatchewan 968,157 $39,834.00 58,939,000

Manitoba 1,148,401 $41,662.00 54,029,000

Nunavut 29,474 $1,115.00 192,138,000

Ontario 12,160,282 $536,340.00 88,303,000

Quebec 7,546,131 $265,888.00 132,970,000

New Brunswick 729,997 $23,669.00 7,133,000

Prince Edward Island 135,851 $4,149 564,000

Nova Scotia 913,462 $28,803.00 5,277,000

Newfoundland and Labrador 505,469 $19,696.00 35,498,000

Exposing data throughA web service

9

Step 1: Getting the data

;"Population";"GDP";"Land Area (hectares)""British Columbia";"4,113,487";"$164,583.00";"91,837,000""Yukon";"30,372";"$1,452.00";"47,890,000""Alberta";"3,290,350";"$187,493.00";"61,400,000""Northwest Territories";"41,464";"$4,138.00";"112,984,000""Saskatchewan";"968,157";"$39,834.00";"58,939,000""Manitoba";"1,148,401";"$41,662.00";"54,029,000""Nunavut";"29,474";"$1,115.00";"192,138,000""Ontario";"12,160,282";"$536,340.00";"88,303,000""Quebec";"7,546,131";"$265,888.00";"132,970,000""New Brunswick";"729,997";"$23,669.00";"7,133,000""Prince Edward Island";"135,851";"$4,149";"564,000""Nova Scotia";"913,462";"$28,803.00";"5,277,000""Newfoundland and Labrador";"505,469";"$19,696.00";"35,498,000"

DATA = list(csv.reader(open("provinces.csv"),delimiter=";"))[1:]

10

Step 2: Writing the APIclass ProvincesAPI:

def __init__( self, data ):self.data = data

def list( self ):return list(l[0] for l in self.data)

def province( self, name ):return ([l for l in self.data if l[0] == name] or [None])[0]

def population( self, province ):return self.province(province)[1]

def gdp( self, province ):return self.province(province)[2]

def landArea( self, province ):return self.province(province)[3]

11

Step 3: Testing it

if __name__ == "__main__":api = ProvincesAPI(DATA)for p in api.list():

print "Province:", pprint "­ gdp :", api.gdp(p)print "­ population :", api.population(p)print "­ land area :", api.landArea(p)

Province: British Columbia­ gdp : $164,583.00­ population : 4,113,487­ land area : 91,837,000Province: Yukon­ gdp : $1,452.00­ population : 30,372­ land area : 47,890,000Province: Alberta­ gdp : $187,493.00­ population : 3,290,350­ land area : 61,400,000Province: Northwest Territories­ gdp : $4,138.00­ population : 41,464­ land area : 112,984,000

Province: Saskatchewan­ gdp : $39,834.00­ population : 968,157­ land area : 58,939,000Province: Manitoba­ gdp : $41,662.00­ population : 1,148,401­ land area : 54,029,000Province: Nunavut­ gdp : $1,115.00­ population : 29,474­ land area : 192,138,000Province: Ontario­ gdp : $536,340.00­ population : 12,160,282

Province: New Brunswick­ gdp : $23,669.00­ population : 729,997­ land area : 7,133,000Province: Prince Edward Island­ gdp : $4,149­ population : 135,851­ land area : 564,000Province: Nova Scotia­ gdp : $28,803.00­ population : 913,462­ land area : 5,277,000Province: Newfoundland and Labrador­ gdp : $19,696.00­ population : 505,469

12

Step 4 : Exposing it

from retro import *

class ProvincesAPI(Component):

def __init__( self, data ):Component.__init__(self)self.data = data

@expose(GET="/api/provinces/all")def list( self ):

return list(l[0] for l in self.data)

@expose(GET="/api/province/{name:string}")def province( self, name ):

name = name.lower().replace(" ","")return ([l for l in self.data if l[0].lower().replace(" ","") == name] or [None])[0]

13

Step 4 : Exposing itImport retro

from retro import *

class ProvincesAPI(Component):

def __init__( self, data ):Component.__init__(self)self.data = data

@expose(GET="/api/provinces/all")def list( self ):

return list(l[0] for l in self.data)

@expose(GET="/api/province/{name:string}")def province( self, name ):

name = name.lower().replace(" ","")return ([l for l in self.data if l[0].lower().replace(" ","") == name] or [None])[0]

14

Step 4 : Exposing it

from retro import *

class ProvincesAPI(Component):

def __init__( self, data ):Component.__init__(self)self.data = data

@expose(GET="/api/provinces/all")def list( self ):

return list(l[0] for l in self.data)

@expose(GET="/api/province/{name:string}")def province( self, name ):

name = name.lower().replace(" ","")return ([l for l in self.data if l[0].lower().replace(" ","") == name] or [None])[0]

Decorate the method toexpose it over HTTP

15

Step 4 : Exposing it

from retro import *

class ProvincesAPI(Component):

def __init__( self, data ):Component.__init__(self)self.data = data

@expose(GET="/api/provinces/all")def list( self ):

return list(l[0] for l in self.data)

@expose(GET="/api/province/{name:string}")def province( self, name ):

name = name.lower().replace(" ","")return ([l for l in self.data if l[0].lower().replace(" ","") == name] or [None])[0]

Use HTTP methodsas keyword arguments

16

Step 4 : Exposing it

from retro import *

class ProvincesAPI(Component):

def __init__( self, data ):Component.__init__(self)self.data = data

@expose(GET="/api/provinces/all")def list( self ):

return list(l[0] for l in self.data)

@expose(GET="/api/province/{name:string}")def province( self, name ):

name = name.lower().replace(" ","")return ([l for l in self.data if l[0].lower().replace(" ","") == name] or [None])[0]

Specify URL to respond to

17

Step 4 : Exposing it

from retro import *

class ProvincesAPI(Component):

def __init__( self, data ):Component.__init__(self)self.data = data

@expose(GET="/api/provinces/all")def list( self ):

return list(l[0] for l in self.data)

@expose(GET="/api/province/{name:string}")def province( self, name ):

name = name.lower().replace(" ","")return ([l for l in self.data if l[0].lower().replace(" ","") == name] or [None])[0]

Pass parameters from URLto Python method

18

Step 4 : Exposing it

@expose(GET="/api/province/{province:string}/population")def population( self, province ):

return self.province(province)[1]

@expose(GET="/api/province/{province:string}/gdp")def gdp( self, province ):

return self.province(province)[2]

@expose(GET="/api/province/{province:string}/landArea")def landArea( self, province ):

return self.province(province)[3]

Don't change a line in yourexisting code

19

Step 5 : Running it

curl localhost:8000/api/provinces/all["British Columbia","Yukon","Alberta","Northwest Territories","Saskatchewan","Manitoba","Nunavut","Ontario","Quebec","New Brunswick","Prince Edward Island","Nova Scotia","Newfoundland and Labrador"]

curl localhost:8000/api/province/quebec/gdp"$265,888.00"

curl localhost:8000/api/province/quebec/population"7,546,131"

curl localhost:8000/api/province/quebec/landArea"132,970,000"

if __name__ == "__main__": run(components=[ProvincesAPI(DATA)],port=8000)

20

Step 5 : Running it

curl localhost:8000/api/provinces/all["British Columbia","Yukon","Alberta","Northwest Territories","Saskatchewan","Manitoba","Nunavut","Ontario","Quebec","New Brunswick","Prince Edward Island","Nova Scotia","Newfoundland and Labrador"]

curl localhost:8000/api/province/quebec/gdp"$265,888.00"

curl localhost:8000/api/province/quebec/population"7,546,131"

curl localhost:8000/api/province/quebec/landArea"132,970,000"

if __name__ == "__main__": run(components=[ProvincesAPI(DATA)],port=8000)

Runs the Retro embeddedweb server

if __name__ == "__main__": run(components=[ProvincesAPI(DATA)],port=8000)

21

Step 5 : Running it

curl localhost:8000/api/provinces/all["British Columbia","Yukon","Alberta","Northwest Territories","Saskatchewan","Manitoba","Nunavut","Ontario","Quebec","New Brunswick","Prince Edward Island","Nova Scotia","Newfoundland and Labrador"]

curl localhost:8000/api/province/quebec/gdp"$265,888.00"

curl localhost:8000/api/province/quebec/population"7,546,131"

curl localhost:8000/api/province/quebec/landArea"132,970,000"

if __name__ == "__main__": run(components=[ProvincesAPI(DATA)],port=8000)

if __name__ == "__main__": run(components=[ProvincesAPI(DATA)],port=8000)

Data is returned asJSON

22

Step 6 : Surprise !

if __name__ == "__main__":api = ProvincesAPI(DATA)for p in api.list():

print "Province:", pprint "­ gdp :", api.gdp(p)print "­ population :", api.population(p)print "­ land area :", api.landArea(p)

run(components=[api],port=8000)

Your API object remains usable as it was before in Python

23

Retro in a Nutshell

WSGI ­basedWorks with CGI, FCGI, SCGI, WSGI on Apache, Lighttpd, Nginx

HTTP StreamingUses WSGI generator (yield) support

StandaloneComes with embedded web server and event reactor

LightweightMinimal API, small footprint

The end

Thank you !www.github.com/sebastien/retro

www.datalicious.casebastien@datalicious.ca