accumulo summit 2015: zookeeper, accumulo, and you [internals]

of 38/38
Securely explore your data ZOOKEEPER, ACCUMULO AND YOU Michael Allen Architect Sqrrl Data, Inc.

Post on 15-Jul-2015

128 views

Category:

Technology

2 download

Embed Size (px)

TRANSCRIPT

PowerPoint Presentation

Zookeeper,Accumulo and youMichael AllenArchitectSqrrl Data, Inc.Securely explore your dataThe case of the dead tserverWhy when I close my laptop does my tablet server die?ACCUMULO CLUSTERWorkerHadoop DataNodeAccumulo TserverWorkerHadoop DataNodeAccumulo TserverAccumulo MasterZookeeperClusterHadoop NameNodeZookeeper ClusterZookeeperClusterZookeeper ClusterZookeeperClusterZookeeper ClusterZookeeperNodeZookeeperNodeZookeeperNodeZookeeper ClusterZookeeperNodeZookeeperNodeZookeeperNodeZookeeperNodeZookeeperNodeWhat's zookeeper good at?Zab was introduced in ZK 3.4.08Zookeeper DATAZookeeper NodeZookeeper DATAZookeeper Node//accumulo/accumulo/instancesZookeeper DATAZookeeper Node//accumulo/accumulo/instances834c234-cd2731This is a lie, each Accumulo instance has a name under /accumulo/instances and those have the UUIDs.11Zookeeper CLIENTsZookeeper Node

ZookeeperClient2181/tcpRandom long client ID and random passwordKeep alive pingConnections are stateful within the cluster. Cluster manages the session state, clients are along for the ride, for the most part. Clients will send a keep alive ping to let the ZK server know they're still there. If ZK clients get partitioned away from client, they will go into disconnected then expired state.12Ephemeral NOdesZookeeper Node

Ephemeral nodeEphemeral nodes exist only as long as the session with that client exists. Accumulo takes advantage of this feature for listing available tablet servers within a cluster.13Sequential NOdesZookeeper Node5

Sequential nodes6

Sequential nodes are a feature of ZK. You can request to make one from the client. ZK guarantees that nodes are created with monotonically increasing values in name and that all clients see a consistent view of who owns which nodes. You can use this to make a simple mutex for things like master server ownership within a cluster.14WATCHESZookeeper Node

56watchClients may "watch" a node to wait for updates, deletes. Watches respond one time and then need to be reset. Curator framework from Facebook takes a lot of heavy lifting out of setting up local caches of ZooKeeper nodes that are kept up to date behind the scenes.15Accumulo data in Zookeeper/accumulocf7c5ecd-0c3c-4d5c-aed2-36d9ca076976usermonitorroot_tabletgctable_lockshdfs_reservationsnamespacesrecoveryfatetserverstablesnext_filetracersconfigdeadbulk_failed_copyqmastersinstancestserverszlock-0000000000server1.companyco.com:9997Tserverszlock-0000000000server2.compnyco.com:9997configgc.cycle.delayconfiggeneral.rpc.timeoutgeneral.server.message.size.maxtables+rTABLES!01234stateconfflush-idcompact-idcompact-cancel-idnamenamespaceroot_tabletlast_locationROOT_TABLETlocationdirwalogsLocation has a server name (like tservers) plus a ZooKeeper client session ID. TabletLocationCache will consult this information when it finds it doesn't know where the root tablet is or the root tablet has moved servers.20usersrootUSERSmallenjvinesafuchs?21USERS

22USERS

What the frak?23usersrootZOOKEEPER ACLSmallenjvinesafuchs'digest,'accumulo:SkvnZlrIQ19GNd7eLDXGKg0Esgw=: cdrwaDigest is the authentication scheme, more on that one in a minute. Can also be "auth", meaning anyone that did any kind of auth, "host" which is hostname (or suffix), "ip" which can be specific IP or subnet. Create, delete, read, write, ACL setting24DIGEst Scheme really means passwords

25UH oh...I forgot the password I used!

But...I do have access to zkServer.shRooting your zookeeperCreate an identityZk-digest.sh#!/bin/bash

if [ -z ${ZOOKEEPER_HOME} ]; then echo "Set \$ZOOKEEPER_HOME before running this script" exit 4747fi

if [ -z ${JAVA_HOME} ]; then echo "Set \$JAVA_HOME before running this script" exit 4747fi

if [ $# -eq 0 ]; then echo "usage: zk-digest.sh " echo "" echo " Utility to produce authentication digests, such as you might see in ZooKeeper node ACL entries" echo "" echo " Example: zk-digest.sh sqrrl:secret" exit 4747fi

ZK_CLASSPATH="\${ZOOKEEPER_HOME}/build/classes:\${ZOOKEEPER_HOME}/build/lib/*.jar:\${ZOOKEEPER_HOME}/lib/slf4j-log4j12-1.6.1.jar:\${ZOOKEEPER_HOME}/lib/slf4j-api-1.6.1.jar:\${ZOOKEEPER_HOME}/lib/netty-3.2.2.Final.jar:\${ZOOKEEPER_HOME}/lib/log4j-1.2.15.jar:\${ZOOKEEPER_HOME}/lib/jline-0.9.94.jar:\${ZOOKEEPER_HOME}/zookeeper-3.4.5.jar:\${ZOOKEEPER_HOME}/src/java/lib/*.jar:\${ZOOKEEPER_HOME}/conf\"

${JAVA_HOME}/bin/java -Dzookeeper.log.dir="." \-Dzookeeper.root.logger="INFO,CONSOLE" \-cp "${ZK_CLASSPATH}" \-Dcom.sun.management.jmxremote \-Dcom.sun.management.jmxremote.local.only=false \org.apache.zookeeper.server.auth.DigestAuthenticationProvider $*Rooting your zookeeperCreate an identityzk-digest.sh super:secretsuper:secret->super:lK75jTNcA+U9vtVEw5vB51mj/w4=

Rooting your zookeeperCreate an identityEdit zk-server.sh

Rooting your zookeeperCreate an identityEdit zk-server.shnohup $JAVA "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" \-cp "$CLASSPATH" $JVMFLAGS $ZOOMAIN "$ZOOCFG" > "$_ZOO_DAEMON_OUT" 2>&1 < /dev/null &

Rooting your zookeeperCreate an identityEdit zk-server.shnohup $JAVA "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" \"-Dzookeeper.DigestAuthenticationProvider.superDigest=super:lK75jTNcA+U9vtVEw5vB51mj/w4= \"-cp "$CLASSPATH" $JVMFLAGS $ZOOMAIN "$ZOOCFG" > "$_ZOO_DAEMON_OUT" 2>&1 < /dev/null &

Rooting your zookeeperCreate an identityEdit zk-server.shReboot ZookeeperRooting your zookeeper

The case of the dead tserverWhy when I close my laptop does my tablet server die?The case of the dead tservertserverszlock-0000000000server1.companyco.com:9997The case of the dead tservertserverszlock-0000000000server1.companyco.com:9997XQUESTIONS?