writing apps the google-y way (brisbane)
Post on 06-May-2015
2.381 Views
Preview:
DESCRIPTION
TRANSCRIPT
WRITING APPS THE GOOGLE-Y WAYPamela Fox, YOW! Australia 2010 (Brisbane)
Who am I?
twitter.com/pamelafox
pamelafox@gmail.com
pamelafox.org
you get the idea...
Who am I?
Google Maps API Google Wave API
2006 20102008
Google App Engine
Who am I?
wave side projects
92 apps
Who am I?
Java pYthon
What is App Engine?
“Google App Engine enables you to build and host web apps on the same systems that power Google applications.”
http://code.google.com/appengine
What is a “web app”?
Static vs. Dynamic
Anonymous vs. Users
Intranet vs. Internet
~2 billionHundreds - Thousands
What is a “web app”?
Some Google web apps
Some Google App Engine web apps
www.gifttag.comwww.buddypoke.com
Google apps on App Engine
panoramio.com pubsubhubbub.appspot.com
How does App Engine work?
1. You upload application code & resources to Google.
2. Google serves your application from scalable infrastructure.
3. You pay for only the resources that Google used in serving the application.
Demo: Guestbook
awesomest-app.appspot.com
http://code.google.com/p/google-app-engine-samples/source/browse/trunk/guestbook
appengine.google.comlocalhost
build deploy monitor
App Engine architecture
App Engine architecture
user
task
App Engine architecture
App Engine architecture
LIMIT
CPU
LIMIT
Memory
LIMIT
Time
App Engine architecture
hardwareports
globalsfile
system
Groovy, JRuby, Mirah, Clojure, Scala
App Engine architecture
141,241,791 calls1 GB data
$0.15 GB/month
45,000,000 calls
657,000 - 46,000,000 calls
*Always check docs for latest quotas.
192,672,000 calls558 GB data$0.15 GB/month
7,000 - 1,700,000 calls$0.0001 per mail sent
46,000,000 calls 1,046 GB data sent
100,000 - 20,000,000 calls
The tricky bits
Datastore
Entity
PropertiesKey
Entity
Entity
Entity
Entity
Path Kind Name/ID
Example: Speaker Entities
Key Path
Kind ID First Name
Last Name
Speaker1
- Speaker 1 Rod Johnson
Key Path
Kind ID First Name
Last Name
Middle Name Suffix
Speaker2 - Speaker
2 Guy Steele L Jr.
Modeling Speaker Entities
class Speaker(db.model):
firstname = db.StringProperty(required=True)
lastname = db.StringProperty(required=True)
middlename = db.StringProperty()
namesuffix = db.StringProperty()
website = db.StringProperty()
keynote = db.BooleanProperty(default=False)
Saving Speaker Entities
rod = Speaker(firstname="Rod", lastname="Johnson")
guy = Speaker(firstname="Guy", lastname="Steele",
middlename="L", namesuffix="Jr.")
rod.put()
guy.put()
Updating Speaker Entities
rod = Speaker.get_by_id(1)
guy = Speaker.get_by_id(2)
rod.website = "http://www.sexyspring.com"
rod.keynote = True
guy.website = "http://www.lusciouslisp.com"
guy.keynote = True
db.put(rod, guy)
LIMIT!(size/# of batch ops)
Queries & Indexes
Query Index
Index
Index
Index
Query
Query
Query
Query
Query
Queries & Indexes
SELECT * from Speaker ORDER BY lastname
key lastname
Speaker3 Fox
Speaker4 Hohpe
Speaker1 Johnson
Speaker2 Steele
LIMIT!(# of results)
Queries & Indexes
SELECT * from Speaker ORDER by middlename
key middlename
Speaker2 L
Queries & Indexes
SELECT * from Speaker WHERE keynote = True
key keynote
Speaker1 True
Speaker2 True
Speaker3 False
Speaker4 False
Queries & Indexes
SELECT * from Speaker WHERE keynote = False
key keynote
Speaker1 True
Speaker2 True
Speaker3 False
Speaker4 False
Queries
allspeakers = Speaker.all().order('lastname')
for speaker in allspeakers:
print speaker.firstname + '' + speaker.lastname + '' + speaker.website
keynotespeakers = Speaker.all().filter('keynote = ', True)
notspecialspeakers = Speaker.all().filter('keynote = ', False)
LIMIT!(size of results)
Custom Indexes
SELECT * from Speaker ORDER BY lastname, keynote
key lastname keynote
Speaker3 Fox false
Speaker4 Hohpe false
Speaker1 Johnson true
Speaker2 Steele true
speakers = Speaker.all().order('lastname')
.order('keynote')
Custom Indexes
SELECT * from Speaker WHERE lastname > 'Johnson' and keynote = true
key lastname keynote
Speaker3 Fox false
Speaker4 Hohpe false
Speaker1 Johnson true
Speaker2 Steele true
speakers = Speaker.all().order('lastname')
.filter('keynote =', True)
Impossible Indexes
SELECT * from Speaker WHERE lastname < 'Steele' and firstname > 'Gregory'
key lastname firstname
Speaker3 Fox Pamela
Speaker4 Hohpe Gregory
Speaker1 Johnson Rod
Speaker2 Steele Guy
...not in subsequent rows!
Impossible Indexes
SELECT * from Speaker WHERE lastname > 'Fox' ORDER BY firstname
key lastname firstname
Speaker3 Fox Pamela
Speaker4 Hohpe Gregory
Speaker1 Johnson Rod
Speaker2 Steele Guy
...not in the correct order!
Queries with Offset
SELECT * from Speaker LIMIT 2 OFFSET 2
key lastname
Speaker3 Fox
Speaker4 Hohpe
Speaker1 Johnson
Speaker2 Steele
speakers = Speaker.all().fetch(2, 2)
1
2
...slow! LIMIT!(# of offset)
Queries with Cursors
query = db.Query(Speaker)
speakers = q.fetch(1000)
cursor = q.cursor()
memcache.set('speaker_cursor', cursor)
...
last_cursor = memcache.get('speaker_cursor')
q.with_cursor(last_cursor)
speakers = q.fetch(1000)
More Properties
class Talk(db.Model):
title = db.StringProperty(required=True)
abstract = db.TextProperty(required=True)
speaker = db.ReferenceProperty(Speaker)
tags = db.StringListProperty()
pamela = Speaker.all().filter('firstname = ', 'Pamela').get()
talk = Talk('Writing Apps the Googley Way', 'Bla bla bla',
pamela, ['App Engine', 'Python'])
talk.put()
talk = Talk('Wonders of the Onesie', 'Bluh bluh bluh',
pamela, ['Pajamas', 'Onesies'])
talk.put()
Back-References
pamela = Speaker.all().filter('firstname = ', 'Pamela').get()
for talk in pamela.talk_set:
print talk.title
key speaker
Talk6 Speaker2
Talk1 Speaker3
Talk2 Speaker3
Talk5 Speaker4
SELECT * from Talk WHERE speaker = Speaker3
Searching List Properties
talks = Talk.all().filter('tags = ', 'python')
.fetch(10)
SELECT * from Talk WHERE tags = 'Python'
key lastname
Talk1 App Engine
Talk2 Pajamas
Talk1 Python
Talk2 Onesies
LIMIT!(# of index rows)
Update Transactions
commitjournal apply entities
apply indexes
A B
Entity Groups
pamela = Speaker.all().filter('firstname = ', 'Pamela').get()
talk1 = Talk('Writing Apps the Googley Way', 'Bla bla bla',
pamela, ['App Engine', 'Python'],
parent=pamela)
talk2 = Talk('Wonders of the Onesie', 'Bluh bluh bluh',
pamela, ['Pajamas', 'Onesies'],
parent=pamela)
db.put(talk1, talk2)
def update_talks():
talk1.title = 'Writing Apps the Microsoft Way'
talk2.title = 'Wonders of the Windows'
db.put(talk1, talk2)
db.run_in_transaction(update_talks)
Common Features
Counters
1 2 3 4 5people have done something.
RageTube: Global Stats
ragetube.net
http://github.com/pamelafox/ragetube
RageTube: Global Stats
SongStat
yaycountviewcount
title artist
Key
Path KindName(song)
naycount
mehcount
RageTube: Global Stats
viewcount viewcount viewcount
datastore memcache
RageTube: Global Stats
class Song(db.Model): viewcount = db.IntegerProperty(default=0) title = db.StringProperty() artist = db.StringProperty()
def get_viewcount(self): viewcount = self.viewcount cached_viewcount = memcache.get('viewcount-' + self.key().name(), self.key().kind()) if cached_viewcount: viewcount += cached_viewcount return viewcount
@classmethod def flush_viewcount(cls, name): song = cls.get_by_key_name(name) value = memcache.get('viewcount-' + name, cls.kind()) memcache.decr('viewcount-' + name, value, cls.kind()) song.viewcount += value song.put()
@classmethod def incr_viewcount(cls, name, interval=5, value=1): memcache.incr('viewcount-' + name, value, cls.kind()) interval_num = get_interval_number(datetime.now(), interval) task_name = '-'.join([cls.kind(), name.replace(' ', '-'), 'viewcount', str(interval), str(interval_num)]) deferred.defer(cls.flush_viewcount, name, _name=task_name)
LIMIT!(# of tasks)
Ratings
Rated by 500 users.
App Gallery: Ratings
google.com/analytics/apps/
App Gallery: Ratings
Comment Application
total_ratings
sum_ratings
avg_ratingrated_inde
x
comment_count
rating
App Gallery: Ratings
def UpdateAppCommentData(self, rating, operation): def UpdateCommentData(self, rating, operation): self.comment_count += 1 * operation self.sum_ratings += rating * operation self.total_ratings += 1 * operation self.avg_rating = int(round(self.sum_ratings / self.total_ratings)) self.rated_index = '%d:%d:%d' % (self.avg_rating, self.total_ratings, self.index) self.put()
db.run_in_transaction(UpdateCommentData, self, rating, operation)
app.UpdateAppCommentData(rating, db_models.Comment.ADD)comment = db_models.Comment()comment.application = appcomment.rating = ratingcomment.put()
query.order('-avg_rating').order('-rated_index')
Geospatial Queries
City-Go-Round: Agencies
citygoround.org
https://github.com/walkscore/City-Go-Round
City-Go-Round: Geo Queries
AgencyGeoModel
location (GeoPt)
location_geocells (StringListProper
ty)
City-Go-Round: Geo Queries
def fetch_agencies_near(lat, long, bbox_side_in_miles): query = Agency.all() bbox = bbox_centered_at(lat, long, bbox_side_in_miles) return Agency.bounding_box_fetch(query, bbox, max_results = 50)
def bounding_box_fetch(query, bbox, max_results=1000,): results = [] query_geocells = geocell.best_bbox_search_cells(bbox)
for entity in query.filter('location_geocells IN', query_geocells): if len(results) == max_results: break if (entity.location.lat >= bbox.south and entity.location.lat <= bbox.north and entity.location.lon >= bbox.west and entity.location.lon <= bbox.east): results.append(entity) return results
Full Text Search
pizza Search
ThingyIt's like pizza, but in the cloud.
Other ThingyThis will make you smell as delicious as pizza.
Disclosed.ca: Search
https://github.com/nurey/disclosed
disclosed.ca
Disclosed.ca: Search
Contract
agency_name vendor_name
description comments
uri
Disclosed.ca: Search
from search.core import SearchIndexProperty, porter_stemmer
class Contract(db.Model): uri = db.StringProperty(required=True) agency_name = db.StringProperty(required=True) vendor_name = db.StringProperty(required=True) description = db.StringProperty() comments = db.TextProperty() search_index = SearchIndexProperty(('agency_name', 'vendor_name', 'description', 'comments'), indexer=porter_stemmer)
results = Contract.search_index.search(sheep').fetch(20)
Disclosed.ca: Search
Contract
agency_name vendor_name
description comments
uri
search_index(StringListProper
ty)
SearchIndex
Disclosed.ca: Search
key search_index
ContractSearch1 charter
ContractSearch1 june
ContractSearch1 sheep
ContractSearch2 sheep
ContractSearch1 wood
SELECT FROM ContractSearch WHERE search_index = "sheep"
More Learning
http://ae-book.appspot.com
http://code.google.com/appengine
http://blog.notdot.net/
AppEngine: Now & Later
"Run your web apps on Google's infrastructure.Easy to build, easy to maintain, easy to scale."
Roadmap:
App Engine (Standard):• MapReduce• Bulk Import/Export• Channel API• Datastore Options
App Engine for Business:• SQL• SLA + Support
Thanks for coming!
top related