faбrik - timesopen: sockets and streams - sept. 2012
TRANSCRIPT
![Page 1: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/1.jpg)
NYT faбrik
TimesOpen – 12 September 2012
![Page 2: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/2.jpg)
2
Who
Architect Infrastructure Group Previous
– US Army– Harvard– Tech companies– United Nations
![Page 3: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/3.jpg)
3
What we’ll cover
A story faбrik overview Code and demo
![Page 4: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/4.jpg)
4
Takeaways
Application Developer– Demand ‘events’ (no polling, no thanks, been there, done that)– Demand infrastructure that scales and you don’t have to worry about
Infrastructure Engineer– Decouple, flatten, simplify– Outsource complexity
![Page 5: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/5.jpg)
5
The Story: Harvard U, 1986+
Bad: – No Internet (pre-web)– data locked in mainframe– large central clerical staff– monolithic central systems
Good:– Vision of an Information Utility– desire to innovate– lots of desktop computers (30,000)– email everywhere although over diverse networks and technologies
![Page 6: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/6.jpg)
6
The Story: Harvard U, 1986+
Solution: – Relational database (decouple data from application)– Email backbone (decouple producers from consumers)– Event-driven desktop applications (flatten)– Identical code on mainframe (simplify)
Result:– Data warehouse unlocked (before the term was coined)– Central clerical staff functions upgraded/dispersed– Old central systems replaceable and, ultimately, replaced– Happy users!
![Page 7: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/7.jpg)
7
faбrik – Why?
?
![Page 8: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/8.jpg)
8
NYT Mission
Enhance society by creating, collecting and distributing high quality news, information and entertainment
- Distributing: publish / subscribe- Collecting: gather / analyze- High Quality: fast, reliable, accurate
![Page 9: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/9.jpg)
9
faбrik
Asynchronous Messaging Framework
For client devices as well as our apps
Enabled by:– Websockets– Robust message handling software– Amazon Web Services
Focusing on simple, common services
![Page 10: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/10.jpg)
10
faбrik – solve the problem!
App
![Page 11: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/11.jpg)
11
faбrik – basic
App
App
App
Message Broker
![Page 12: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/12.jpg)
12
faбrik – basic
App
App
App
Message Broker
Amazon Web Services
• EC2• S3• Identity &
Access Mgt• DynamoDB• Route 53…
![Page 13: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/13.jpg)
13
faбrik – basic++
OtherApp
AppBuddy
Service Buddy
Message Broker
Message Broker
Service Buddy
“Retail”
“Wholesale”
![Page 14: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/14.jpg)
14
faбrik: Current Implementation
Open source– Erlang/OTP– RabbitMQ– Nodejs – Sockjs (websockets +)– Python (gevent, select/epoll, tornado, futures, …)– ZeroMQ (new)
Automated build/deployment
![Page 15: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/15.jpg)
15
faбrik – active/active cluster
Service Buddy
‘a’
Service Buddy
‘b’
Zone ‘a’ Zone ‘b’
Region Wherever
![Page 16: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/16.jpg)
16
faбrik – active/active cluster
Service Buddy
‘a’
Service Buddy
‘b’
Zone ‘a’ Zone ‘b’
Region Wherever
Got your back!
Got yours too!
![Page 17: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/17.jpg)
17
faбrik
![Page 18: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/18.jpg)
18
faбrik
![Page 19: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/19.jpg)
19
faбrik
![Page 20: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/20.jpg)
20
faбrik – Layers
![Page 21: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/21.jpg)
21
faбrik – Layers
![Page 22: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/22.jpg)
22
faбrik – Layers
![Page 23: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/23.jpg)
23
faбrik – Layers
![Page 24: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/24.jpg)
24
![Page 25: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/25.jpg)
25
![Page 26: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/26.jpg)
26
Pub / Sub
![Page 27: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/27.jpg)
27
Pub / Sub
![Page 28: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/28.jpg)
28
Pub / Sub
![Page 29: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/29.jpg)
29
Pub / Sub
![Page 30: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/30.jpg)
30
Pub / Sub
![Page 31: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/31.jpg)
31
Gather / Analyze
![Page 32: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/32.jpg)
32
Gather / Analyze
![Page 33: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/33.jpg)
33
Gather / Analyze
![Page 34: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/34.jpg)
34
Gather / Analyze
![Page 35: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/35.jpg)
35
Gather / Analyze
![Page 36: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/36.jpg)
36
Gather / Analyze
![Page 37: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/37.jpg)
37
Gather / Analyze
![Page 38: Faбrik - TimesOpen: Sockets and Streams - Sept. 2012](https://reader038.vdocuments.net/reader038/viewer/2022110121/5587dc00d8b42a0f638b4584/html5/thumbnails/38.jpg)
38
faбrik
Demo
Code
App