hpca2012 facebook keynote
DESCRIPTION
TRANSCRIPT
![Page 1: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/1.jpg)
![Page 2: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/2.jpg)
Social Networking at Scale
Sanjeev Kumar Facebook
![Page 3: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/3.jpg)
1 What makes scaling Facebook challenging?
2 Evolution of Software Architecture
3 Evolution of Datacenter Architecture
Outline
![Page 4: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/4.jpg)
700B minutes spent
on the site every month
2.5M sites using
social plugins
30B pieces of content
shared each month
500M daily active users
!""# !""$ !""% !""& !"'"
845M users worldwide
![Page 5: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/5.jpg)
What makes scaling Facebook challenging? ▪ Massive scale
▪ Social Graph is central to everything on the site
▪ Rapidly evolving product
▪ Complex Infrastructure
![Page 6: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/6.jpg)
Traditional websites
Bob
Bob’s data
Julie
Julie’s data
Dan
Dan’s data
Beth
Beth’s data
Sue
Sue’s data
Erin
Erin’s data
Bob
Bob’s data
Horizontally scalable
![Page 7: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/7.jpg)
People are only one dimension of the social graph
Social Graph
![Page 8: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/8.jpg)
Facebook: The data is interconnected Common operation: Query the social graph
Bob Erin Beth
Servers
![Page 9: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/9.jpg)
Social Graph Cont’d
▪ Highly connected
▪ 4.74 average degree-of-separation between users on Facebook
▪ Made denser by our connections to places, interests, etc.
▪ Examples of Queries on Social Graph
▪ What are the most interesting updates from my connections?
▪ Who are my connections in real-life who I am not connected to on Facebook?
▪ What are the most relevant events tonight near me and related to my interests? Or that my friends are going to?
![Page 10: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/10.jpg)
Social Graph Cont’d
▪ System Implications of Social Graph
▪ Expensive to query
▪ Difficult to partition
▪ Highly customized for each user
▪ Large working sets (Fat tail)
![Page 11: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/11.jpg)
What makes scaling Facebook challenging? ▪ Massive scale
▪ Social Graph: Querying is expensive at every level
▪ Rapidly evolving product
▪ Complex Infrastructure
![Page 12: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/12.jpg)
!M
"!!M
#!!M
$!!M
%!!M
&!!M 800M
New Apps February 2004
Sign Up NewsFeed
2006
Platform launch 2007
Translations 2008
The Stream 2009
Open Graph 2010
</> Social Plugins 2010
Photos Update 2010
Places 2010
Mobile Event 2010
Groups 2010
Messages 2010
New Profile 2010
Questions 2011 ? Unified Mobile
Sites Product Launches
2011 2004
New Apps 2004/2005
Timeline Music Video Calling
iPad App
![Page 13: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/13.jpg)
Rapidly evolving product
▪ Facebook is a platform
▪ External developers are innovating as well
▪ One integrated product
▪ Changes in one part have major implications on other parts
▪ For e.g. Timeline surfaces some of the older photos
▪ System Implications
▪ Build for flexibility (avoid premature optimizations)
▪ Revisit design tradeoffs (they might have changed)
![Page 14: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/14.jpg)
What makes scaling Facebook challenging? ▪ Massive scale
▪ Social Graph: Querying is expensive at every level
▪ Rapidly evolving product
▪ Complex Infrastructure
![Page 15: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/15.jpg)
Complex infrastructure
▪ Large number of Software components
▪ Multiple Storage systems
▪ Multiple Caching Systems
▪ 100s of specialized services
▪ Often deploy cutting-edge hardware
▪ At our scale, we are early adopters of new hardware
▪ Failure is routine
▪ Systems implications
▪ Keep things as simple as possible
![Page 16: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/16.jpg)
1 What makes scaling Facebook challenging?
2 Evolution of Software Architecture
3 Evolution of Datacenter Architecture
Outline
![Page 17: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/17.jpg)
Cache Tier
Web Tier
Storage Tier
Services Tier
Evolution of the Software Architecture Evolution of each of these 4 tiers
![Page 18: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/18.jpg)
Cache Tier
Web Tier
Storage Tier
Services Tier
Evolution of the Software Architecture Evolution of Web Tier
![Page 19: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/19.jpg)
Web Tier
▪ Stateless request processing
▪ Gather Data: from storage tiers
▪ Transform: Ranking (for Relevance) and Filtering (for Privacy)
▪ Presentation: Generate HTML
▪ Runs PHP code
▪ Widely used for web development
▪ Dynamically typed scripting language
▪ Integrated product è One single source tree for all the entire code
▪ Same “binary” on every web tier box
▪ Scalability: Efficiently process each request
![Page 20: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/20.jpg)
0 5 10 15 20 25 30 35 40 45
PHP Zend Python
Ruby Ocaml
C# Java C++
Generation 1: Zend Interpreter for PHP
▪ Reasonably fast (for an interpreter)
▪ Rapid development
▪ Don’t have to recompile during testing
▪ But: at scale, performance matters
Relative Execution Time
![Page 21: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/21.jpg)
Generation 2: HipHop Compiler for PHP
▪ Technically challenging, Impressive gains, Still room for improvement
▪ But: takes time to compile (slows down development)
▪ Solution: HipHop interpreter
▪ But: Interpreter and compiler sometimes disagree
▪ Performance Gains are slowing. Can we improve performance further?
0 5 10 15 20 25 30 35 40 45
PHP HipHop PHP Zend
Python Ruby
Ocaml C#
Java C++
Relative Execution Time
![Page 22: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/22.jpg)
Generation 3: HipHop Virtual Machine
▪ Best of both worlds
▪ Common path, well-specified bytecode semantics
▪ Potential performance upside from dynamic specialization
▪ Work-In-Progress
PHP AST
Optimizer
Parser
Bytecode
Bytecode Generator
HHVM Interpreter
HHVM JIT
![Page 23: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/23.jpg)
Web Tier Facts ▪ Execution time only a small factor in user-perceived performance
▪ Can potentially use less powerful processors
▪ Throughput matters more than latency (True for other tiers as well)
▪ Memory management (allocation/free) is a significant remaining cost
▪ Copy-on-Write in HipHop implementation
▪ Poor Instruction Cache Performance
▪ Partly due to the one massive binary
▪ Web load predictable in aggregate
▪ Can use less dynamic techniques to save power
▪ Potentially even turn off machines. Failure rates is an open question?
![Page 24: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/24.jpg)
Cache Tier
Web Tier
Storage Tier
Services Tier
Evolution of the Software Architecture Evolution of Storage Tier
Storage Tier
![Page 25: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/25.jpg)
Evolution of a Storage Tier
▪ Multiple storage systems at Facebook
▪ MySQL
▪ HBase (NoSQL)
▪ Haystack (for BLOBS) ç
▪ Case Study: BLOB storage
▪ BLOB: Binary Large Objects (Photos, Videos, Email attachments, etc.)
▪ Large files, No updates/appends, Sequential reads
▪ More than 100 petabytes
▪ 250 million photos uploaded per day
![Page 26: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/26.jpg)
Generation 1: Commercial Filers
▪ New Photos Product
▪ First build it the easy way
▪ Commercial Storage Tier + HTTP server
▪ Each Photo is stored as a separate file
▪ Quickly up and running
▪ Reliably Store and Serve Photos
▪ But: Inefficient
▪ Limited by IO rate and not storage density
▪ Average 10 IOs to serve each photo
▪ Wasted IO to traverse the directory structure
NFS Storage
![Page 27: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/27.jpg)
Generation 2: Gen 1 Optimized
▪ Optimization Example:
▪ Cache NFS handles to reduce wasted IO operations
▪ Reduce the number of IO operations per photo by 3X
▪ But:
▪ Still expensive: High end storage boxes
▪ Still inefficient: Still IO bound and wasting IOs
directory inode • owner info • size • timestamps • blocks
directory data • inode # • filename
file inode • owner info • size • timestamps • blocks
data
NFS Storage Optimized
![Page 28: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/28.jpg)
Generation 3: Haystack [OSDI’10]
▪ Custom Solution
▪ Commodity Storage Hardware
▪ Optimized for 1 IO operation per request
▪ File system on top of a file system
▪ Compact Index in memory
▪ Metadata and data laid out contiguously
▪ Efficient from IO perspective
▪ But:
▪ Problem has changed now
Superblock
Needle 1
Needle 2
Needle 3
Magic No
Key
Flags
Photo
Checksum
Single Disk IO to read/write a photo
![Page 29: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/29.jpg)
Generation 4: Tiered Storage
▪ Usage characteristics
▪ Fat tail of accesses: everyone has friends J
▪ A large fraction of the tier is no longer IO limited (new)
▪ Storing efficiency matters much more than serving efficiency
▪ Approach: Tiered Storage
▪ Last layer optimized for storage efficiency and durability
▪ Fronted by caching tier optimized for serving efficiency
▪ Working-In-Progress
![Page 30: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/30.jpg)
BLOB Storage Facts
▪ Hot and Warm data. Little cold data.
▪ Low CPU utilization
▪ Single digit percentages
▪ Fixed memory need
▪ Enough for the index
▪ Little use for anything more
▪ Next generation will use denser storage systems
▪ Do we even bother with hardware raid?
▪ Details to be publicly released soon
![Page 31: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/31.jpg)
Cache Tier
Web Tier
Storage Tier
Services Tier
Evolution of the Software Architecture Evolution of Cache Tier
Storage Tier
![Page 32: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/32.jpg)
Cache Tier: Memcache
Web Tier
Storage Tier
First few Generations: Memcache
Storage Tier
Look-Aside Cache Key-Value Store Does one thing very well Does little else Improved performance by 10X
![Page 33: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/33.jpg)
Memcache limitations ▪ “Values” are opaque
▪ End up moving huge amounts of data across the network
▪ Storage hierarchy exposed to web tier
▪ Harder to explore alternative storage solutions
▪ Harder to keep consistent
▪ Harder to protect the storage tier from thundering herds
![Page 34: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/34.jpg)
Web Tier
Storage Tier
Storage Tier
Cache Tier: Tao
Alternative Caching Tier: Tao
1. Has a data model 2. Write-Through Cache 3. Abstracts the storage tier
![Page 35: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/35.jpg)
Tao Cont’d
▪ Data Model
▪ Objects (Nodes)
▪ Associations (edges)
▪ Have “type” and data
▪ Simple graph operations on them
▪ Efficient: Content-aware
▪ Can be performed on the caching tier
▪ In production for a couple of years
▪ Serving a big portion of data accesses
![Page 36: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/36.jpg)
Tao opens up possibilities
▪ Alternate storage systems
▪ Multiple storage systems
▪ To accommodate different use case (access patterns)
▪ Even more powerful Graph operations
▪ Multi-Tiered caching
![Page 37: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/37.jpg)
Cache Tier Facts
▪ Memcache
▪ Low CPU utilization
▪ Little use for Flash since it is bottlenecked on network
▪ Tao
▪ Much higher CPU load
▪ Will continue to increase as it supports more complex operations
▪ Could use Flash in a multi-tiered cache hierarchy
![Page 38: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/38.jpg)
Cache Tier
Web Tier
Storage Tier
Services Tier
Evolution of the Software Architecture Evolution of Services Tier
Storage Tier
![Page 39: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/39.jpg)
Cache Tier
Web Tier
Storage Tier
Life before Services Example: Wish your friend a Happy Birthday
Storage Tier
Inefficient and Messy • Potentially access hundreds of machines • Solution: Nightly cron jobs • Issues with corner cases What about more complex problems? Solution: Build Specialized Services
![Page 40: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/40.jpg)
A more complex service: News Feed Aggregation of your friends’ activity
One of many (100s) services at Facebook
![Page 41: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/41.jpg)
News Feed Product characteristics ▪ Real-time distribution
▪ Along edges on the Social Graph
▪ Writer can potentially broadcast to very large audience
▪ Reader wants different & dynamic ways to filter data
▪ Average user has 1000s of stories per day from friends/pages
▪ Friend list, Recency, Aggregation, Ranking, etc.
![Page 42: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/42.jpg)
Service: News Feed
News Feed Service
▪ Build and maintain an index: Distributed
▪ Rank: Multiple ranking algorithms
Query [ Read ]
User Update [ Write ]
![Page 43: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/43.jpg)
Two approaches: Push vs. Pull
▪ Push approach
▪ Distribute actions by reader
▪ Write broadcasts, read one location
▪ Pull approach
▪ Distribute actions by writer
▪ Write one location, read gathers
▪ Pull model is preferred because
▪ More dynamic: Easier to iterate
▪ “In a social graph, the number of incoming edges is much smaller than the outgoing ones.”
9,000,000
621
![Page 44: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/44.jpg)
Service: News Feed
Leafs
Aggregators
News Feed Service: Big Picture Query
[ Read ] User Update
[ Write ]
▪ Pull Model
▪ Leafs: One copy of the entire index. Stored in memory (Soft state)
▪ Aggregators: Aggregate results on the read path (Stateless)
![Page 45: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/45.jpg)
Service: News Feed
Leafs
Aggregators
News Feed Service: Writes
▪ On User update (Write)
▪ Index sharded by Writer
▪ Need to update one leaf
Query [ Read ]
User Update [ Write ]
![Page 46: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/46.jpg)
Service: News Feed
Leafs
Aggregators
News Feed Service: Reads
▪ On Query (Read)
▪ Query all leafs
▪ Then do aggregation/ranking
Query [ Read ]
User Update [ Write ]
![Page 47: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/47.jpg)
Service: News Feed
Leafs
Aggregators
News Feed Service: Scalability
▪ 1000s of machines
▪ Leafs: Multiple sets. Each set (10s of machines) has the entire index
▪ Aggregators: Stateless. Scale with load.
Query [ Read ]
User Update [ Write ]
![Page 48: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/48.jpg)
News Feed Service: Reliability ▪ Dealing with (daily) failures
▪ Large number of failure types
▪ Hardware/software
▪ Servers/Networks
▪ Intermittent/Permanent
▪ Local/Global
▪ Keep the software architecture simple
▪ Stateless components are a plus
▪ For example, on read requests:
▪ If a leaf is inaccessible, failover the request to a different set
▪ If an aggregator is inaccessible, just pick another
![Page 49: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/49.jpg)
New Feed Service Facts
▪ Number of leafs dominate the number of aggregators
▪ Reads are more expensive than writes
▪ Every read (query) involves one aggregator and every leaf in the set
▪ Very high network load between aggregator and leafs
▪ Important to keep a full leaf set within a single rack on machines
▪ Uses Flash on leafs to ensure this
![Page 50: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/50.jpg)
Cache Tier
Web Tier
Storage Tier
Services Tier
Evolution of the Software Architecture Summary
Memcache & Tao
HipHop Compiler & VM
New Feed
BLOB Storage
![Page 51: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/51.jpg)
1 What makes scaling Facebook challenging?
2 Evolution of Software Architecture
3 Evolution of Datacenter Architecture
Outline
![Page 52: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/52.jpg)
Recall: Characteristics of Facebook
▪ Massive Scale
▪ Social Graph
▪ Expensive to query
▪ Hard to partition
▪ Large working set (Fat tail)
▪ Product is rapidly evolving
▪ Hardware failures are routine
![Page 53: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/53.jpg)
Implications ▪ On Datacenters
▪ Small number of massive datacenters (currently 4)
▪ On Servers
▪ Minimize the “classes” (single digit) of machines deployed
▪ Web Tier, Cache Tier, Storage Tier, and a couple of special configurations
▪ Started with
▪ Leased datacenters + Standard server configurations from vendors
▪ Moving to
▪ Custom built datacenters + custom servers
▪ Continue to rely on a small number of machine “classes”
![Page 54: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/54.jpg)
Servers
Data Center
AMDMotherboard
Intel Motherboard
Server Chassis
Battery Cabinet
Triplet Rack
Power Supply
Electrical Mechanical
![Page 55: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/55.jpg)
![Page 56: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/56.jpg)
Evaporative cooling system
![Page 57: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/57.jpg)
Open Compute
▪ Custom datacenters & servers
▪ Minimizes power loss
▪ POE of 1.07
▪ Vanity Free design
▪ Designed for ease of operations
▪ Designs are open-sourced
▪ More on the way
![Page 58: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/58.jpg)
1 What makes scaling Facebook challenging?
2 Evolution of Software Architecture
3 Evolution of Datacenter Architecture
Outline
Questions?
![Page 59: Hpca2012 facebook keynote](https://reader034.vdocuments.net/reader034/viewer/2022042606/540e28cf8d7f728d7e8b4bde/html5/thumbnails/59.jpg)
(c) 2009 Facebook, Inc. or its licensors. "Facebook" is a registered trademark of Facebook, Inc.. All rights reserved. 1.0