cloudcamp athens 2011 presenting heroku
TRANSCRIPT
HerokuThe platform for your next awesome app.
1
Monday, October 3, 11
In a nutshellPlatform as ServiceRuby on Rails & lately NodeJS, Java, Clojure on top of AWS.
Founded 2007 - Bought in 2010 from Salesforce ($212M)
194.000 apps NOW.~30% from mobile applications. RESTful services. Special deal with Facebook for FB app hosting.
Jul 2011, Matz (Yukihiro Matsumoto) joined heroku as Chief Architect, Ruby [1]Remember: MRI stands for Matz’s Ruby Interpreter
2CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting HerokuMonday, October 3, 11
Summary - Index
InfrastructureHardware - Parking.
Network InterfaceRouting Mesh = Magic
DatabasePostgresql, NoSQL solutions
Process Model & DynoZooming in the heart of Heroku
Stacks & Frameworks
Add onsA sweet surprise
DeployingGit - Rollback - Migrating.
ScalingEasier than ever.
MonitoringLogging - Exception Tracking
3CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
Effective Cloud HostingTaking advantage of Pricing / process model
AutoScale web processes
AutoScale background jobs
Monday, October 3, 11
Infrastructure
Build atop of Amazon Web Services
Cedar Stack: on High Memory Double Extra Large Instances [2]
35 GB Ram
13 EC2 Compute Units.
maybe ~ 120 Dynos per Instance
Database on EC2 - EB storage mhhh (see April’s outage incident)
4CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting HerokuMonday, October 3, 11
Infrastructure
5CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
Not in Cedar Stack
Monday, October 3, 11
Network Interface
DNSSeems to be 6 nginx reverse proxies + any dedicated IP/proxy ($100 / month) (proxy.heroku.com)
Routing MeshRouting software written in Erlang. (seems to have some secret sauce)
If no live dyno is found for this app’s route, a new dyno is spawned.
HTTP Caching< Stack Cedar: Varnish Reverse Proxy Caching via HTTP headers, HTTP 1.0 compatible.
>= Cedar: No Reverse Proxy Caching, HTTP 1.1 Compatible, Routes directly to dynos:
Non-blocking servers (Goliath / EventMaching) - Long Polling6CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
Monday, October 3, 11
Heroku Dyno
A dyno is a single (isolated) process of any type, running on the Heroku platformMongrel, Thin, Delayed Job, Resque, Workers...
Dyno Manifold manages the resources.Any number of dynos / application w/ the latest code compiled and deployed.
Crashed dynos are closed and restarted automatically.
Dynos are isolated: LXC & Chroot for isolated user space / resources / filesystem. [4]
512 Mb Ram.
Cost per Dyno (dyno-hour)1 Dyno for Free.
$ 0.05 / hour = $ 35 / month / dyno --- 6 minutes = 0.1 dyno-hour = $ 0.005
7CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting HerokuMonday, October 3, 11
Dyno’s Process Model of Cedar
Dyno Manifold does the Process Management Starts / Stops / Restarts / Idles the processes / Sets config Vars / Applies Code Releases
All the possible Process Types (servers, workers,..) are described in a procfile.
Procfile is a file on the root of your app and describes any needed process that you want to run (and scale)
Foreman is a ruby Gem handling profiles for the Development Environment.
Scale Preview
8CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
web: bundle exec rails server -p $PORTworker: bundle exec rake jobs:workscraper: bundle exec ruby scraper.rb
$ heroku scale web=3 worker=2 scraper=2
Monday, October 3, 11
Database - SQL solutions.
PostgreSQLShared database PostgreSQL 8.3 - Dedicated database uses PostgreSQL 9.
Continues Backups [3]
Storing on EBS volumes of AmazonWS (“With this information in hand, we'll be taking a hard look on how to reduce our dependence on EBS”) [3]
External Connections are NOT allowed
Goodies Backups - psqldump
Database Sync with TAPS
Is currently a DaaS for Heroku - encouraging polyglot
9CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting HerokuMonday, October 3, 11
Database - noSQL solutions
DaaS is (maybe) the future of complex web applications
10CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
Heroku Supports encourages the polyglot conceptRedis for your Ad Service / counters etc.
Hadoop for your big data tables
S3 for your Assets
Memcache for transient data such as Sessions
CouchDB for data that need to be replicated (your separate mobile app database)
Neo4j for your super clever recommendation engine
Currently most of theses DaaS offer an add-on for HEROKU.
Monday, October 3, 11
Stacks & Frameworks
Zooming in Cedar StackOS: Ubuntu Server 10.04 (Lucid Lynx) - Ruby: MRI 1.9.2 - Bundler: 1.1pre5 - Node.js: 0.4.7 - NPM: 1.0.6 - JVM: OpenJDK 6
Migration between stacks is allowed
Rails 3, NodeJs, JAVA & Spring, Python (private beta), Clojure
11CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
>> heroku stack:migrate bamboo-mri-1.9.2
Monday, October 3, 11
Add ons
Services Implement Heroku’s Add-ons API for automate provisioning
12CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
MYSQL_URL=mysql://user:[email protected]/database
Dyno Manifold will Restart all the running Dynos, including the new ENV variables.
Monday, October 3, 11
Deploying.
Based on Git.Heroku is a git remote for your code.
When receiving a push (git push heroku master) starts the deployment process
bundle install
Remove unused files, including the .git directory, anything in log and tmp, .gem files in the bundled gems directory, and anything specified in .slugignore.
Write the environment into the slug. (The slug is rebuilt any time a config var or add-on is changed)
Package into the final slug archive.
Slug Size Matters: Larger Slug == slower distribution
13CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
$ git remote add heroku [email protected]:myapp.git
< 30 MBMonday, October 3, 11
Scaling
Concept: More Dynos = Better Concurrency
Unlimited Number of Dynos / Application.Command Line
14CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
$ heroku scale web=3 worker=2 scraper=2
Monday, October 3, 11
Scaling
Concept: More Dynos = Better Concurrency
Unlimited Number of Dynos / Application.Command Line
Web Interface
15CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting HerokuMonday, October 3, 11
Scaling
Concept: More Dynos = Better Concurrency
Unlimited Number of Dynos / Application.Command Line
Web Interface
Ruby Script (wait for it...)
16CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting HerokuMonday, October 3, 11
Scaling
Concept: More Dynos = Better Concurrency
Unlimited Number of Dynos / Application.Command Line
Web Interface
Ruby Script (wait for it...)
Mobile App (Nazumi App)
17CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
$ heroku scale web=3 worker=2 scraper=2
Monday, October 3, 11
Scaling
Concept: More Dynos = Better Concurrency
Unlimited Number of Dynos / Application.Command Line
Web Interface
Ruby Script (wait for it...)
Mobile App (Nazumi App)
18CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
$ heroku scale web=3 worker=2 scraper=2
Monday, October 3, 11
Scaling
Concept: More Dynos = Better Concurrency
Unlimited Number of Dynos / Application.Command Line
Web Interface
Ruby Script (wait for it...)
Mobile App (Nazumi App)
Database ScalingPostgreSQL 9
Amazon RDS addon
NoSQL solutions
19CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
Ika: $800/month, 50 connections, 7.5 GB of RAM, 4 compute unitsBaku: $3200/month, 128 connections, 34 GB of RAM, 13 cuMecha: $6400/month, 256 connections, 68 GB of RAM, 26 cu
Monday, October 3, 11
Monitoring
Heroku Logsheroku logs everything
NginX - Routing Mesh - Every Dyno (web, worker, custom)
Add-ons New Relic
Airbrake (HopToad)
StillAlive
20CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
$ heroku logs --tail$ heroku logs --source heroku --ps router$ heroku logs -s app -p worker.1$ heroku logs --ps postgres Only for dedicated Database
Monday, October 3, 11
Heroku Billing Model
100% Based on it’s process modelDynos are billed per second at a rate of 0.05$ / hour
First Dyno is FREE (Watch out: single Dyno Idling Policy can cause delays)
Add-ons are billed per second.
Databases are billed per hour.
1 web server + 1 worker + Shared Database (20GB) = 1x36 + 1x36 + 15 = 87$ /month
21CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting HerokuMonday, October 3, 11
Dyno #1Dyno #2Dyno #3Dyno #4Dyno #5Dyno #6Dyno #7Dyno #8Dyno #9Dyno #10
Heroku can be even more elastic
22CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
1LargeElastic
~4.7 GB
ram
Monday, October 3, 11
Dyno #1Dyno #2Dyno #3Dyno #4Dyno #5Dyno #6Dyno #7Dyno #8Dyno #9Dyno #10
Heroku can be even more elastic
23CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
1LargeElastic
~4.7 GB
ram
Monday, October 3, 11
Dyno #1Dyno #2Dyno #3Dyno #4Dyno #5Dyno #6Dyno #7Dyno #8Dyno #9Dyno #10
Heroku can be even more elastic
24CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
1LargeElastic
~4.7 GB
ram
Monday, October 3, 11
Dyno #1Dyno #2Dyno #3Dyno #4Dyno #5Dyno #6Dyno #7Dyno #8Dyno #9Dyno #10
Heroku can be even more elastic
25CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
1LargeElastic
~4.7 GB
ram
10 Dynos => $ 0.5/h 1 LARGE => $ 0.35/h
HEROKU = 42% more expensive
Monday, October 3, 11
Dyno #1Dyno #2Dyno #3Dyno #4Dyno #5Dyno #6Dyno #7Dyno #8Dyno #9Dyno #10
Heroku can be even more elastic
26CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting Heroku
1LargeElastic
~4.7 GB
ram
10 Dynos => $ 0.5/h 1 LARGE => $ 0.35/h
HEROKU = 42% more expensive
0.33*1/2+0.33*1/5+0.33*1 = 56% dynamic usage
10 DYNAMIC Dynos => $ 0.28/h 1 LARGE => $ 0.35/h
HEROKU = 25% cheaper
Monday, October 3, 11
Use Heroku because:It’s freeYou can have a look at almost everything without spending a penny.
It’s fastRapid prototyping meets heroku
It support open source solutions
Scaling needs can show up any day
You are better developer than administrator
27CloudCamp.org/Athens | 2011 | Savvas Georgiou | presenting HerokuMonday, October 3, 11
CodeFor dynamic Dyno management
28
Monday, October 3, 11
Reference[1] http://blog.heroku.com/archives/2011/7/12/matz_joins_heroku/[2] http://cedar-versions.herokuapp.com/evil[3] https://status.heroku.com/incident/151[4] http://devcenter.heroku.com/articles/dyno-isolation[5[ http://devcenter.heroku.com/articles/process-model[6] http://blog.heroku.com/archives/2010/7/20/nosql/[7] https://addons.heroku.com/provider/resources/technical/how/provision/3[8] http://blog.heroku.com/archives/2010/7/20/nosql/
30
Monday, October 3, 11