api strategy evolution at netflix
DESCRIPTION
Developing an API strategy should be considered a journey, not a project with a predetermined outcome. This presentation describes Netflix's journey to discover a winning API strategy as well as future directions for the API.TRANSCRIPT
API Strategy Evolution at
Michael Hart (@michaelhart)Director of Engineering, Social Systems(Former Director of Engineering, API)
Daniel Jacobson (@daniel_jacobson)Director of Engineering, API
Presentation is designed for presenting vs. reading.Please review speaker notes if reading online.
(disclaimer)
YMMV
A Thousand Flowers
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 460%
2%
4%
6%
8%
10%
12%
14%
16%
18%
20%
Per App % of Total Monthly Active API-enabled App Users
A Thousand Flowers?
Hundreds of Devices
Agility vs. Stability
Tons of Tests
Recap:A Thousand FlowersHundreds of DevicesTons of TestsMore to come…
?
Be prepared to evolve
Developing an effective API strategy is a journey- not just a single project
The Future of the Netflix APITo Infinity and Beyond…
Growth of the Netflix API
Over 10 BILLION requests in November(Peaks at about 10,000 requests per second)
It is time to get back to thinking about the future…
First Step: Back to the Drawing Board…
Some of our Key Goals• Resiliency, Reliability and Stability
– Maximize uptime
• Scalability– System can grow without needing frequent refactoring
• Simplicity– Easier integration for API consumers
• Agility– Architecture will support rapid development without compromising the core of the
system
Netflix API
XBox
PS3
Wii
Google TV
Apple TV
iPad App
iPhoneApp
Roku
TiVo
Netflix API XBox
PS3
Wii
Google TV
Apple TV
iPad App
iPhoneApp
Roku
TiVo
Future Architecture needs to support key audience first with a trickle down of features to the public audience
Hide Our Internal Business Dependencies from API Consumers
(Some) API Interfaces and Their DependenciesAuto-
Complete Similars QueuesLIstsRental History
Agg Lists
SEARCH SIMILARS SYSTEM
RECOMMENDATION ENGINE QUEUERENTAL
HISTORY
REST Interface API Dependency
Title Details Previews Ratings RecommendationsTitle
States
CONTENT MANAGEMENT
SYSTEMTITLE STATES CINEMATCH
TITLESTATE
SIMILARSLIST
SEARCHLIST
RENTAL HISTORY
LIST
RATINGRECOMMENDATIONTITLE DETAILS
TITLE LIST
QUEUELIST
AUTOCOMPLETE
LISTETC…
TITLE INFORMATION
UNIFIED LIST/TITLE RESPONSEWITH PARTIAL RESPONSE BUILT IN
INTERNET REQUEST
Recommend-ation
Ratings
Rental History
User Account
Title Catalog
Queue
Normal-ization
And Resiiency
Ratings Service
Recommend-ation
Service
Rental History Service
LIST API
USER API
TITLE API
AUTHAPI
iPhone Wrapper
Wii Wrapper
Xbox Wrapper
PS3 Wrapper
Roku Wrapper
AppleTV Wrapper
iPad Wrapper
PC / Mac Wrapper
TiVo Wrapper
Source Data Layer
API Repository Layer API Layer
Wrapper Layer App Layer
Web Service Layer
SHARED
API
INTER-FACES
Shared Layer
Recommend-ation
Ratings
Rental History
User Account
Title Catalog
Queue
Normal-ization
And Resiiency
Ratings Service
Recommend-ation
Service
Rental History Service
LIST API
USER API
TITLE API
AUTHAPI
iPhone Wrapper
Wii Wrapper
Xbox Wrapper
PS3 Wrapper
Roku Wrapper
AppleTV Wrapper
iPad Wrapper
PC / Mac Wrapper
TiVo Wrapper
Source Data Layer
API Repository Layer API Layer
Wrapper Layer App Layer
Web Service Layer
SHARED
API
INTER-FACES
Shared Layer
THESE ARE SOME OF THE DATA SOURCES
THAT PROVIDE CONTENT TO THE API
Recommend-ation
Ratings
Rental History
User Account
Title Catalog
Queue
Normal-ization
And Resiiency
Ratings Service
Recommend-ation
Service
Rental History Service
LIST API
USER API
TITLE API
AUTHAPI
iPhone Wrapper
Wii Wrapper
Xbox Wrapper
PS3 Wrapper
Roku Wrapper
AppleTV Wrapper
iPad Wrapper
PC / Mac Wrapper
TiVo Wrapper
Source Data Layer
API Repository Layer API Layer
Wrapper Layer App Layer
Web Service Layer
SHARED
API
INTER-FACES
Shared Layer
THESE ARE SOME OF THE SERVICES PROVIDED BY
SOME OF THE API DEPENDENCIES THROUGH WHICH CONTENT IS MADE
AVAILABLE TO THE API
Recommend-ation
Ratings
Rental History
User Account
Title Catalog
Queue
Normal-ization
And Resiiency
Ratings Service
Recommend-ation
Service
Rental History Service
LIST API
USER API
TITLE API
AUTHAPI
iPhone Wrapper
Wii Wrapper
Xbox Wrapper
PS3 Wrapper
Roku Wrapper
AppleTV Wrapper
iPad Wrapper
PC / Mac Wrapper
TiVo Wrapper
Source Data Layer
API Repository Layer API Layer
Wrapper Layer App Layer
Web Service Layer
SHARED
API
INTER-FACES
Shared Layer
THIS IS THE LAYER WHERE THE API CAN NORMALIZE THE DATA TO PREPARE IT FOR CONSUMPTION BY API CLIENTS. IT COULD
ALSO BE WHERE THE API CAN APPLY RESILIENCY MEASURES TO PROTECT AGAINST FAILURES FROM THE API
DEPENDENCIES.
ALTHOUGH THE DIAGRAM REPRESENTS THIS AS A DISCRETE API LAYER, IT COULD JUST AS EASILY BE A DISTRIBUTED MODEL
ACROSS THE DEPENDENCIES.
Recommend-ation
Ratings
Rental History
User Account
Title Catalog
Queue
Normal-ization
And Resiiency
Ratings Service
Recommend-ation
Service
Rental History Service
LIST API
USER API
TITLE API
AUTHAPI
iPhone Wrapper
Wii Wrapper
Xbox Wrapper
PS3 Wrapper
Roku Wrapper
AppleTV Wrapper
iPad Wrapper
PC / Mac Wrapper
TiVo Wrapper
Source Data Layer
API Repository Layer API Layer
Wrapper Layer App Layer
Web Service Layer
SHARED
API
INTER-FACES
Shared Layer
MAKING PURE APIs WILL IMPROVE THE CONSISTENCY OF THE API CODE AND
THOSE THAT CONSUME IT. THIS WOULD HELP PRODUCT MANAGERS AS WELL
SINCE THE CODE FOR THEIR PRODUCTS CAN ALSO BECOME MORE UNIFORM.
Recommend-ation
Ratings
Rental History
User Account
Title Catalog
Queue
Normal-ization
And Resiiency
Ratings Service
Recommend-ation
Service
Rental History Service
LIST API
USER API
TITLE API
AUTHAPI
iPhone Wrapper
Wii Wrapper
Xbox Wrapper
PS3 Wrapper
Roku Wrapper
AppleTV Wrapper
iPad Wrapper
PC / Mac Wrapper
TiVo Wrapper
Source Data Layer
API Repository Layer API Layer
Wrapper Layer App Layer
Web Service Layer
SHARED
API
INTER-FACES
Shared Layer
THE SHARED API INTERFACE LAYER ALLOWS THE VARIOUS CLIENTS TO MERGE RESULTS FROM THE PURE APIs AND PREPARE THEM
IN WAYS THAT ARE CONDUCIVE TO THE GOALS OF THEIR APP. MEANWHILE, ANY OF THESE INTERFACES CAN BE SHARED ACROSS
THE CLIENTS SINCE MANY OF THESE INTERFACES WILL BE COMMON ACROSS THE
PLATFORMS.
Recommend-ation
Ratings
Rental History
User Account
Title Catalog
Queue
Normal-ization
And Resiiency
Ratings Service
Recommend-ation
Service
Rental History Service
LIST API
USER API
TITLE API
AUTHAPI
iPhone Wrapper
Wii Wrapper
Xbox Wrapper
PS3 Wrapper
Roku Wrapper
AppleTV Wrapper
iPad Wrapper
PC / Mac Wrapper
TiVo Wrapper
Source Data Layer
API Repository Layer API Layer
Wrapper Layer App Layer
Web Service Layer
SHARED
API
INTER-FACES
Shared Layer
THE WRAPPER LAYER IS DESIGNED TO CONSUME THE PURE APIS AND/OR THE SHARED INTERFACES WHILE ALLOWING
DEVICE/APP-SPECIFIC IMPLEMENTATIONS. SOME OF THESE WRAPPERS ARE
THEORETICAL WHILE OTHERS MAY ACTUALLY TAKE ADVANTAGE OF THIS TIER.
Recommend-ation
Ratings
Rental History
User Account
Title Catalog
Queue
Normal-ization
And Resiiency
Ratings Service
Recommend-ation
Service
Rental History Service
LIST API
USER API
TITLE API
AUTHAPI
iPhone Wrapper
Wii Wrapper
Xbox Wrapper
PS3 Wrapper
Roku Wrapper
AppleTV Wrapper
iPad Wrapper
PC / Mac Wrapper
TiVo Wrapper
Source Data Layer
API Repository Layer API Layer
Wrapper Layer App Layer
Web Service Layer
SHARED
API
INTER-FACES
Shared Layer
THESE ARE SOME OF THE ACTUAL INTERNAL API
CUSTOMERS TODAY. THIS LIST WILL GROW OVER TIME, BUT
THIS ARCHITECTURE WILL SCALE ACCORDINGLY.
Flexible API RequestsAs flexible as SQL
SELECT person_first_name, person_last_name, dept_name
FROM person, department
WHERE UPPER(dept_name) LIKE ‘%Engineering%’
AND person_added_date BETWEEN ‘2002-01-01’ AND sysdate
ORDER BY person_last_name ASC
Manages Partial Response
Determines the Selected API
Restricts Output
Sorts Results
GOAL: API should be able to handle any request that you can do in SQL
The Problem with Versioning1.0
1.5
2.0
Today
3.0?
4.0?
5.0?
Maybe we don’t need versioning?1.0
1.5
2.0
Future APIs
Today
Branching can always be done later…1.0
1.5
2.0
New API
Today
XML vs. JSON?Or both? Or other?
xml
REST…as long as it makes sense for the business
Questions?
API Management• Engineering Manager – Web API Systems: http://bit.ly/9O3Hic• API Integration Systems Analyst: http://bit.ly/hyPxCh
API Engineering• Senior Software Engineer/Architect – Web API Platform: http://bit.ly/e0AKw4
API Test Engineering• Senior Software Engineer in Test – API: http://bit.ly/9zHPtJ• Senior Software Engineer in Test – API Platform: http://bit.ly/eD3tLn• Lead Software Engineer in Test – API: http://bit.ly/fqxBYC
Social Engineering• Social Systems – Engineer Architect: http://bit.ly/dYq6C8
Michael HartDirector of Engineering – Social@michaelhart
Daniel JacobsonDirector of Engineering – API@daniel_jacobson