introduction to apache accumulo

How to use this presentation

• Covered topics: Accumulo architecture, operational maintenance, fault handling

• Intended Audience: Developers, supporters, PMs who are conversant in multi-component systems, i.e. involved in web services.

• Presumes familiarity with RDBMS

• Expected running time: 40 - 60 minutes

• License: CC-BY-SA 2.0

• Please let me know if you find it useful and what it could use: busbey@cloudera.com

Introduction to Apache AccumuloScaling a web application made easier

Sean Busbey // Software Engineer

Let’s talk about Apache Accumulo…

But in the context of a specific use case

•I really like technology that solves a problem.

•Keep in mind that this won’t be exhaustive.

•YMMV, proof-of-concepts with metrics are better than slides.

Who am I?

• Apache Accumulo PMC

• Apache HBase committer

• Software Engineer on Cloudera’s storage team

That is to say, I work for a vendor and no longer have operational scale problems of my own.

We’ll focus on an application that enables conversations centered on cute cats.

Simple sharing model built with privacy

controls

•User defines a group that may see their posting

•User posts a picture to a given group

•Members of the group may write short messages

Straight forward web architecture

Relational Data Model

Will map user names to identifiers used elsewhere.

Will track ownership and descriptive name.

Will allow users to add and remove members.

User table Group table Group membership table

Relational Data Model

Tracks distribution group, owner, and topical image.

Individual comments from users.

Topic table Comment table

First growth: robustness

Second growth: application scale out

Scaling reads: what goes into this page?

Database reads eventually become a

bottleneck

Scale by de-normalizing in favor of reads

Change to writes - original

Change to writes – de-normalized

Generally known

as the fan-out

pattern.

The trick is to not get crushed by the writes

•Each poster now does a write for each member of the group a post goes to.

•Removing access is now a much larger delete query.

•Most databases are geared toward few writes and many reads; are we screwed?

Recall our access pattern

Basically one of

these consumer

boxes.

Lines up very well with sharding

•Divide the query space up by e.g. a hash of user id into n shards.

•Store a copy of table on each shard, but just for user ids that hash to that shard.

•Reads and writes are spread across instances.

Database shards Layout

What were the nice-to-haves for the RDBMS

again?

• No longer leveraging relational data model.

• Now running, backing up, and failing over num shards number of database instances.

• Robustness in a shard has to be managed.

• Sharding is essentially static; adding more resources with growth still painful.

Now we have some context for Accumulo.Our goal is to end up with less operational overhead.

“The Apache Accumulo™ sorted, distributed key/value store is a robust, scalable, high performance data storage and retrieval system.”Accumulo PMC via https://accumulo.apache.org/

Accumulo-based App Layout

In Accumulo, you address cells rather than

records

Key Valu

Keys are multi-dimensional

Key Valu

Column Tim

Keys are multi-dimensional

Key Valu

Column Tim

eFamily Qualifier Visibility

Accumulo doesn’t assume a schema

•All key and value components, save time, are byte[]

•The application is responsible for serialization

•Common to use different serialization for the values in different columns.

Mapping records to cells

•Treat a row as a database

• Essentially each column is a record field

•Treat each cell as a database record

• Need to uniquely identify each record

• Useful if you generally need the whole row and not a subset of columns

• Can then treat each row as a shard of database records.

Let’s use a concrete example.

Already know our reads are within a shard.

Mapping our data into cells

Key Value

Row Column Family Column Qualifier Visibility author, image url,

and commentreader id discussion id comment order group id

We end up with something close to our

original.

Note the use of visibility

Visibility enforcement

•At scan time, our application will pass in the groups for the current user.

•Accumulo will filter any cells that don’t match those groups.

• Group removal is a simple update in the group management system again.

Sparse column storage

•We are creating lots of columns: per discussion per group member.

•Accumulo only stores columns that exist in a given row.

All cells sorted according to key

• Total ordering based on lex-sort of raw byte arraysof key components.

• Time is sorted most-recent-first

• Reads are done on a contiguous range of cells.

When sorted our data looks like this….

And the scan for a page is roughly…

Lexicoders

• Turning different kinds of data into sortable bytes is painful

• Accumulo ships implementations for several common Java types

• Also for e.g. reversing the sort order and building compound keys.

Inefficiencies in our data model

Key Value

Row Column Family Column Qualifier Visibility author, image url,

and commentreader id discussion id comment order group id

Two categories of data

Key Value

Row Column Family Column Qualifier Visibility author, image url

reader id discussion id image group id

Key Value

Row Column Family Column Qualifier Visibility author, comment

reader id discussion id text group id

And now our data looks like this

And the scan for a page covers less data

“The Apache Accumulo™ sorted, distributedkey/value store is a robust, scalable, high performance data storage and retrieval system.”Accumulo PMC via https://accumulo.apache.org/

Our simplified diagram

Slightly less simplified

Back to the data model

Key Valu

Column Tim

Back to the data model

Key Valu

Column Tim

Rows are grouped into Tablets

• Tablet is defined by a start and end row

• All cells for a given row must be in the same Tablet.

Tablets are assigned to Tablet Servers

• At any given point in time, a Tablet is serviced by a single Tablet Server

Slightly less simplified

Tablets are assigned to Tablet Servers

• At any given point in time, a Tablet is serviced by a single Tablet Server

• That server is responsible for client reads and writes to all hosted Tablets

• Finding the proper server is handled by the Accumulo libraries

• Proper key design means io load gets spread across multiple machines

Tablet assignment is not static

• Assignment tend to have steady state

• But can move in the event of new resources or failure

Remember our RDBMS scaling?

New RDBMS shard

1. Provision hardware for service

2. Rewrite data under new sharding

3. Update application services

• Doing this without an outage is hard work (and well paid if you can get it)

New Accumulo Tablet Server

1. Provision hardware for service

2. Add server to cluster

3. Tablets automatically migrate from busier nodes to new node

• No outage from client perspective.

All distributed systems have communication

failures

In the face of such a failure you can either

• remain available on remaining nodes to all clients

• provide a consistent view of updates to a subset of clients

Now you know the basics of CAPRemember that you can’t give up partition tolerance

Remember our RDBMS robustness?

Accumulo is a CP system

• Tablet Servers ensure that updates have been written to a distributed write-ahead-log before acknowledging

• Tablet Server failures are automatically detected

• Newly assigned hosts for recovered Tablets then replay edits up until last ack before serving new requests

Client write

Write goals

• Low latency ack

• Don’t lose acked writes in face of node failure

Client write

Recovery timing

• Tunable time to detection – increases network load

• Size of outstanding write ahead logs

Client write

Accumulo-based App Layout

What’s the catch?

• Still requires application updates to use API – no interactive SQL bindings*

• No Disaster Recovery – coming in next minor release

Thank you.

introduction to apache accumulo

Technology

apache accumulo and cloudera

compaction and splitting in apache accumulo

accumulo summit 2015: performance models for apache...

accumulo summit 2015: using d4m for rapid prototyping of...

accumulo summit 2014: data-center replication with apache...

accumulo summit 2015: real-time distributed and reactive...

apache accumulo user manual version 1 · contents 1...

real-time distributed and reactive systems with apache kafka...

developing apache spark applications · apache spark...

apache accumulo installaon guide - cloudera · 2019. 7....

apache accumulo 1.4 & 1.5 features - the apache software...

apache uima introduction

introduction to apache accumulo

accumulo summit 2015: alternatives to apache accumulo's java...

introduction to apache drill - nyc apache drill meetup

accumulo summit 2014: monitoring apache accumulo

oct 2012 hug: apache accumulo: unlocking the power of big...

accumulo summit 2014: accumulo on yarn

benchmarking the apache accumulo distributed key–value...

accumulo extensions to google's bigtable...