1

Headline Goes HereSpeaker Name or Subhead Goes Here

DO NOT USE PUBLICLY PRIOR TO 10/23/12HBase and Metrics2

Elliott Clark | Engineer January 2013

2

HBase Metrics Circa 0.90 to 0.94

• Spread around lots of places• Using old deprecated Metrics• Using Lots of ConcurrentHashMaps• Some metrics stored two places• Inconsistent Naming

3

Lets Fix It

4

Lets Fix it

• Replace everything with metrics2• Rename everything•Make things faster• Remove un-needed metrics

5

Problems

• Reflection would be hard and slow• Have to support both Hadoop 1 and Hadoop 2• Need to add functionality to MetricsRegistry

Hadoop 1 Hadoop 2

MetricMutableGauge MutableGauge

MetricCollector MetricBuilder

Registry uses String Registry uses Interns.info

DefaultMetricsSystem.INSTANCE DefaultMetricsSystem.instance()

6

Solution

• Two implementations• One for Hadoop 1 -- hbase-hadoop1-compat• One for Hadoop 2 – hbase-hadoop2-compat• Layer connecting hbase-server to correct compat• Use ServiceLoader so there’s no compile dependency

7

Jar Dependencies

hbase-hadoop-compat

hbase-server

hbase-hadoop1-compat hbase-hadoop2-compat

8

Yeah but why do I really care about this ?

9

Performance

• Performance improvements• 9% on scans.• ~5% on gets• < 1% on all calls.

10

Naming

•Metrics have changed names• All Camel Cased• Count used as suffix to denote a counter• Metrics have descriptions

11

Better Sink Support

• Can filter metrics at the source• Some sources can take per region metrics• Others only need small subset

12

What else Does the HBase user see ?

•More Histograms meaning more data•Metrics should be MUCH easier to find in JMX• Ability to Filter what metrics are reported where• OpenTSDB will need a patch to work

13

Wrap Up

• Trunk (soon to be 0.96) now on Metrics2• Still needs more documentation

14