Introducing App Engine's Latest FeaturesChris SchalkDeveloper Advocate

Agenda

● What is App Engine?

● Part I: App Engine Production Updates ○ Growth trajectory○ App Engine Success so far○ A Big Production Update: App Engine Leaving

Preview

● Part II: App Engine Technical Updates○ Go Runtime, Backends, Channel API, FTS,

Mapreduce

● Easy to build● Easy to maintain● Easy to scale

What is App Engine?

Focus on building your app, let us wear the pagers!

Build and run your web apps on Google’s infrastructure

IaaS

PaaS

SaaS

Source: Gartner AADI Summit Dec 2009

Cloud landscape as Gartner Sees It

What is App Engine?

● Downloadable SDK● Application runtimes

○ Java, Python, Go● Local development tools

○ Eclipse plugin● Specialized api services ● Cloud based dashboard● Automatically scales● Built in fault tolerance, load

balancing● Continuously evolving...

PaaS Development Platform in a Box

Language Runtime Options

GO JavaExperimental

App Engine Services

BlobstoreImages

Mail XMPP Task Queue

Memcache Datastore URL Fetch

User Service

Agenda

● What is App Engine?

● Part I: App Engine Production Updates ○ Growth trajectory○ App Engine Success so far○ A Big Production Update: App Engine Leaving

Preview

● Part II: App Engine Technical Updates○ Go Runtime, Backends, Channel API, FTS,

Mapreduce

App Engine Evolution Through The Years

App Engine LaunchPython

DatastoreMemcachelogs export

2008 2009 2010

JavaDB Import

cron

Batch write/readHttps

Status-Dashboard

Task QueuesXMPP

incoming email

MultitenancyInstance Console

Always Onhi-perf imag

10 min tasks

BlobstoreAppstats

cursorsMapper

2011

Hi-ReplicationDatastore

Channel APIFiles API

Remote APIProsp Search

100,000+ Active developers per Month

200,000+Active apps per week

1.5B+Pageviews per day

App Engine - By the Numbers

Notable App Engine Customers

Royal Wedding - Scalability Success

Official Royal Wedding blog on App Engine

On Wedding day, served: ● Up to 32k requests per second● 37.7 million pageviews ● 13.7 million visitors

http://goo.gl/F1SGc

Enterprise Success on App Engine

Best Buy Applications on App Engine

WebFilings Financial SaaS on App Engine

http://www.webfilings.com/customers/valued_customers

Enterprise customers:

● Procter & Gamble● Delta Airlines● Valero Energy● Sprint/Nextel● eBay ● United Airlines● Dish Network● ...

Recently signed:● One of the world's largest

banks● The world's largest retailer● One of the largest US

auto manufacturers

App Engine: Certified Security

App Engine is now fully SAS70/SSAE16/ISAE 3402 Certified!

● http://googleappengine.blogspot.com/2011/05/year-ahead-for-google-app-engine.html

One Big Production Update...

.... App Engine is coming out of preview!

Production App Engine Features

● Full Google Product - 3 year deprecation support

● 99.95% SLA for all paid users

● Operational and Developer Support available

● Billing via invoice

● Ongoing feature delivery: custom domain SSL, MapReduce, etc..

App Engine Leaving Preview in 2011!

What Does Leaving Preview Mean?

● Long-term committment by Google to App Engine

● Long-term committment by App Engine to users!

● It's a real product now, Enterprises can trust in it!

● http://googleappengine.blogspot.com/2011/05/year-ahead-for-google-app-engine.html

Production App Engine Pricing

● Usage types:○ Free - similar to today○ Paid - $9/app/mo (+ usage) including SLA○ Premier - $500/account/mo (+ usage) including operational

support

● Add-on fee for special services○ For example: SSL for custom domains will be available for

monthly fee

Production App Engine Further Details

● Instance based charging○ CPU-hours replaced by Instance Hours (IH)○ 1 instance running for 1 hour○ Similar to instances in admin console today

● APIs○ Currently charged per CPU-hour○ Will be charged per operation

● FAQ○ https://groups.google.com/group/google-

appengine/browse_thread/thread/a1bfa432e0c002a7○ https://groups.google.com/group/google-

appengine/browse_thread/thread/1e5ba8835fdade7e

Agenda

● What is App Engine?

● Part I: App Engine Production Updates ○ Growth trajectory○ App Engine Success so far○ A Big Production Update: App Engine Leaving

Preview

● Part II: App Engine Technical Updates○ Go Runtime, Backends, Pull Queues, Channel

API, FTS, MapReduce

App Engine 1.5 Updates

● Introduced at Google IO, May 2011

'Go' added as third language runtime

● Released SDK for Mac & Linux○ Windows port in the works

● Uses tweaked Python SDK tools● Launched for production 1.5.2 (Jul 2011)● http://blog.golang.org/2011/05/go-and-google-app-engine.html

High Replication Datastore now default

● Master/Slave (M/S) original App Engine datastore● High Replication Datastore (HRD) uses Paxos algorithm● Was originally 3x the cost of M/S*● Slightly slower (distributed writes)● Queries eventually consistent● Well-received

○ http://googleappengine.blogspot.com/2011/03/high-replication-datastore-solid-choice.html

HRD as default - cont.

● *Price reduction to discourage M/S (no longer 3x)● Has Delivered ~99.999% uptime since launch in January● M/S price increases when App Engine leaves Preview● Working on better tools

○ To make migration easier○ To estimate updated cost

Backends feature introduced

● Long-running servers● No more 30s or 10m deadline

○ Fully configurable instances with options:■ RAM: 128MB to 1GB■ CPU: 600MHz to 4.8GHz no request deadlines

○ Can run indefinitely ○ Resident or Dynamic - Public or Private○ Automatic restarts ○ Easy to configure (backends.yaml, backends.xml)

● (Demo)● http://www.google.com/events/io/2011/sessions/app-engine-

backends.html

"Pull Queues" introduced

● A new alternative to previous Task Queues, now referred to as "Push Queues"

● Lets users decide when to execute tasks● Uses a REST API● Allows external access

Recalling "Push Queues"

"Pull Queues" introduced

● A new alternative to previous Task Queues, now referred to as "Push Queues"

● Task is merely data, no longer a web request● Workers can pull (lease) tasks from queue● Uses a REST API

○ https://www.googleapis.com/taskqueue/myappid/taskqueues/myqueue

● Can configure ACLs for external access

● http://www.google.com/events/io/2011/sessions/putting-task-queues-to-work.html

Pull Queues vs. (Push)Task Queues

● Push Task Queues take & execute work○ Tasks are "pushed" to application○ Execute at a given rate

● Pull queues offer more flexibility○ Workers can pull (lease) tasks when

appropriate

Other App Engine 1.5.(0) Features

● Mail API updates○ Reduced free recipients from 2000 to 100 for new apps○ ...

● Source code downloading○ Now Owner and Uploader can download

● http://googleappengine.blogspot.com/2011/05/app-engine-150-release.html

App Engine 1.5.1 Features

● Geolocation header○ X-AppEngine-country - can customize web pages based on

location

● Presence support in Channel API○ Can take action when user connects/disconnects (demo)

● WebP support in Images API○ WebP 39% smaller than JPEG!

● Create RPC services with ProtoRPC○ Now part of App Engine

● http://googleappengine.blogspot.com/2011/06/app-engine-151-release.html

App Engine 1.5.2 Features

● Production: adjustable scheduler parameters○ Able to set the minimum pending latency and maximum

number of idle instances

● Namespaced datastore statistics● New option to query datastore stats per namespace

● Tasks○ New Task Queue details page○ Pull Queue task size bump to 1MB○ Pull Queue task lease extensions

● http://googleappengine.blogspot.com/2011/07/app-engine-152-sdk-released.html

App Engine 1.5.3 Features

● Blobstore file uploads no longer limited○ Allows you to serve much larger files: images, videos etc.

● Datastore Admin now easily enable via 1-click○ No longer have to update app.yaml, and java developers

don't have to upload a python version of their app

● New HRD migration tool trusted tester program● Download App

○ Can now download any files that were uploaded into your war directory

● http://googleappengine.blogspot.com/2011/08/app-engine-153-sdk-released.html

Experimental feature updates

● Introduced at Google I/O 2011 talks● Full-text Search

○ Full-text search service coming to App Engine○ Other queries: numeric, geo, date search capabilities, etc.○ http://www.google.com/events/io/2011/sessions/full-text-

search.html

● MapReduce○ Part 1: Mapper has been available for awhile○ Work is ongoing for shuffle and reduce○ http://www.google.com/events/io/2011/sessions/app-engine-

mapreduce.html○ http://www.youtube.com/watch?v=EIxelKcyCC0

Q&A