so we're running apache zookeeper. now what? by camille fournier
DESCRIPTION
The ZooKeeper framework was originally built at Yahoo! to make it easy for the company’s applications to access configuration information in a robust and easy-to-understand way, but it has since grown to offer a lot of features that help coordinate work across distributed clusters. Apache Zookeeper became a de-facto standard for coordination service and used by Storm, Hadoop, HBase, ElasticSearch and other distributed computing frameworks.TRANSCRIPT
![Page 1: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/1.jpg)
So we’re running ZooKeeper. Now What?Camille Fournier, Rent the Runway
@skamille
![Page 2: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/2.jpg)
Big Data. Big Systems.
![Page 3: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/3.jpg)
Common Challenges
• Outages• Coordination • Operational Complexity
![Page 4: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/4.jpg)
Common Deficiency
• Consistency guarantees
![Page 5: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/5.jpg)
ZooKeeper in Storm
Storm uses Zookeeper for coordinating the cluster. Zookeeper is not used for message passing, so the load Storm places on Zookeeper is quite low
![Page 6: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/6.jpg)
ZooKeeper
A centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services
• Distributed, Consistent Data Store• Highly Available• High performance• Strictly ordered access
![Page 7: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/7.jpg)
Highly Available
• Tolerates the loss of a minority ((n/2) – 1) of ensemble members and still function
![Page 8: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/8.jpg)
High Performance
• All data is stored in memory• Performance measured around 50,000
operations/second• Particularly fast for read performance, built for read-
dominant workloads
![Page 9: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/9.jpg)
Strictly Ordered Access
• Atomic Writes• In the order you sent them• Changes always seen in the order they occurred
• Reliable, no writes acked will be dropped
![Page 10: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/10.jpg)
leader
followerfollower
Basics: Cluster Interactions
cli
cli
clicli
cli
cli
![Page 11: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/11.jpg)
Basics: Cluster Interactions
cli
cli
clicli
cli
cli
leader
followerfollower
![Page 12: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/12.jpg)
Basics: Cluster Interactions
cli
cli
clicli
cli
cli
leader
follower
![Page 13: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/13.jpg)
Data Structure
/a
/a/b/a/b/d
/a/c /a/c/e000001
cli
/a/b/myNode
![Page 14: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/14.jpg)
Basics: Data Structure
• Nodes can contain data, have children, or both• Ephemeral nodes are associated with the session that
created them• They cannot have children, and disappear when that
session ends• Sequential nodes have an ever-increasing number
attached to them
![Page 15: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/15.jpg)
Watches
client
client
client
leader
follower
follower
getData “/foo” true
return “mydata”
![Page 16: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/16.jpg)
Watches
client
client
client
leader
follower
followersetData “/foo” “bar”
![Page 17: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/17.jpg)
Watches
client
client
client
leader
follower
follower
NOTIFICATION
![Page 18: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/18.jpg)
Watches
client
client
client
leader
follower
follower
getData “/foo” true
return “bar”
![Page 19: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/19.jpg)
Basics: Watches
• Set against data or path changes• Ordered with respect to other events, other watches, and
asynchronous replies. • A client will see a watch event for a node it is watching
before seeing the new data that corresponds to that node.• The order of watch events corresponds to the order of the
updates as seen by the ZooKeeper service• One time notifications; must be reset, changes can be
missed between notification and reset of the watch
![Page 20: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/20.jpg)
Basics: Creation API
• create• delete• setData
![Page 21: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/21.jpg)
Basics: Get/Watch API
• exists• getData• getChildren
![Page 22: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/22.jpg)
Basics: API
• multi * new in 3.4• sync
![Page 23: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/23.jpg)
Common Uses
Service ManagementDistributed Locking
![Page 24: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/24.jpg)
Coordination
• In Storm, ZooKeeper is the source of communication between Nimbus and Supervisors • Nimbus finds Supervisors via ZooKeeper
![Page 25: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/25.jpg)
Discovery (Naming)
Find servers doing job “Products”
Encode as path in ZooKeeper:
/servers/productsServers register as ephemeral nodes under this path with details about location, other connection info
![Page 26: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/26.jpg)
Configuration
Read config from nodes
Watch nodes for config changes
![Page 27: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/27.jpg)
Locking
Shared LocksBarriers and LatchesLeader ElectionTwo-Phase Commit
![Page 28: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/28.jpg)
And Now, The Scary Part
![Page 29: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/29.jpg)
The State Machine
![Page 30: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/30.jpg)
NOPE
NOPE
![Page 31: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/31.jpg)
Recommended Clients
• Curator (Java)• Kazoo (Python)• Twitter Commons for Discovery
![Page 32: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/32.jpg)
Be Aware
ZooKeeper Owns Your Availability(maybe)
![Page 33: So we're running Apache ZooKeeper. Now What? By Camille Fournier](https://reader036.vdocuments.net/reader036/viewer/2022070315/554f4563b4c905423f8b48a7/html5/thumbnails/33.jpg)
Credits and Contact
• Thank you to @zaa for the format of the slide on watches• Tweet me! @skamille• Email me! [email protected]• Kazoo: http://kazoo.readthedocs.org/en/latest/• Curator: http://curator.incubator.apache.org/• Twitter commons: http://twitter.github.io/commons/