big data dc - benchpress

13
Palomino Labs, Inc. palominolabs.com Distributed Benchmarking with BenchPress

Upload: drew-stephens

Post on 22-Jan-2015

379 views

Category:

Technology


3 download

DESCRIPTION

Talk at Big Data DC (http://www.meetup.com/bigdatadc/) about BenchPress (https://github.com/palominolabs/benchpress)

TRANSCRIPT

Page 1: Big Data DC - BenchPress

Palomino Labs, Inc. palominolabs.com

Distributed Benchmarking with BenchPress

Page 2: Big Data DC - BenchPress

BenchPress

• High performance benchmarking

• Distributed workers

• Flexible

• SQL databases

• NoSQL datastores

• DDOS

• Anything (no, but most things)‽

Page 3: Big Data DC - BenchPress

Y!CSB, The Grinder, JMeter

• Limited workload configuration

• For storage systems only (Y!CSB)

• Single host only

Page 4: Big Data DC - BenchPress

BenchPress Communication

• Service Discovery

• Apache ZooKeeper

• Apache Curator

Page 5: Big Data DC - BenchPress

BenchPress Communication

• Service Discovery

• Apache ZooKeeper

• Apache Curator

• Job Control

• Jetty

• Jersey

• Jackson

Page 6: Big Data DC - BenchPress

BenchPress Job Definition

Page 7: Big Data DC - BenchPress

MongoDBTaskFactory

Page 8: Big Data DC - BenchPress

MongoDBRunnable

Page 9: Big Data DC - BenchPress

• Submitting job• curl -X POST -H "Content-Type: application/json"\• -d @job.json http://benchpress.foo:7000/controller/job

• Job status & completion• curl http://benchpress.foo:7000/controller/job/<job-id>

Page 10: Big Data DC - BenchPress

Advanced Reporting

Page 11: Big Data DC - BenchPress

Advanced Reporting

Page 12: Big Data DC - BenchPress

See Also

• Service Discovery with Curatorhttp://blog.palominolabs.com/2012/08/14/using-netflix-curator-for-service-discovery/

• Apache (née Netflix) Curatorhttp://curator.incubator.apache.org/

• Yahoo! Cloud System Benchmarkhttps://github.com/brianfrankcooper/YCSB/

• Code highlighting byhttp://markup.su/highlighter/

Page 13: Big Data DC - BenchPress

Palomino Labs, Inc. palominolabs.com

Drew Stephens

[email protected]

@dinomite

http://www.slideshare.net/dinomite/big-data-dc-benchpress