hug bigtop

7/31/2019 HUG Bigtop

1/37

What's stacking?

Or:How I Learned to Stop Worrying

and Love the BigTop

Dr. Konstantin BoudnikApache Hadoop committer since 2009

Co-author of iTest (BigTop's kernel)

Senior architect @[email protected]


2/37

Distributed under CC3.0https://creativecommons.org/licenses/by/3.0/


3/37

Disclaimer!

All characters appearing in this work are fictitious. Anyresemblance to real creatures, living or dead, is purely

coincidental


4/37

Can you see the forrest behind the trees?


5/37

Not a technical talk...

What's the real intrinsic value of BigTopWhat is our philosophical proposition


6/37

Why Hadoop data stack is important?


7/37

The elephant is yesterday's news...An article over at Forbes:

...it is not a database, it is not an analyticsenvironment, and it is not a visualization tool.

By itself, it is not a solution for helpingbusinesses make better decisions.Some self-proclaimed hot-shot dude


8/37

Sorry, I am allergic to bullshit...


9/37

There are limits, all right...

Maximum size of the file system High availability Recoverability


10/37

But there is a lot to pick from...

Do you want to experiment with CEPH

Cassandra Ganglia Puppet


11/37

Why I am wasting your time?

You are an ASP and need to ship a stack


12/37


You are an ASP and need to ship a stack You need a higher level abstraction for MR


13/37



You are in need of virtual deployment


14/37



You are in need of virtual deployment

Hey!

I am gonna just pick up a distro from B or S 1)

1) B - Big Ego DistributorS - Small Ego Distributor


15/37

Yeah, right! (case study)

Commercial vendor BS packs Hive 0.7.1 intoa release


16/37

We know how it is full of it


17/37

Good news!

Our next release comes with Hive 0.8.1BS' VP of Product


18/37

But may be they just forgot this?..

public int executeUpdate(String sql, String[]columnNames) throws SQLException {

throw new SQLException("Method notsupported");

}


19/37

So, no worries! We can deliver!


20/37

But can they?


21/37

Do you REALLY have to be at mercy

of a distribution vendors?


22/37

I have no resource to do it myself

DIY: and there's always but...


23/37

Maybe ASP or ISV business isn't for you?


24/37

Or maybe BigTop stack: take it and go...

Modify a stack BOM Build Deploy Configure with Puppet (included) Test (scenarios are provided or easy to add) Grab a virtual appliance if short of hardware


25/37

Hit a problem?

Just ri nse and repeat (C)


26/37

Many thanks to the good folks from Hadoop community!

Many components Gazillions of versions

Lot of patches if you like it hot and dirty


27/37

Where is the philosophy?

WAIT!


28/37

Competition is great! Competition enabling open platform Diversity of stacks No more vendors lock-ups Don't be afraid to try Be your own Sprechstallmeister


29/37

May be something wrong with this?


30/37

News And Updates


31/37

5 releases of BigTop!

Three are outTwo more in the work


32/37

Strong community and growing!

20 committers and many contributorsWe are looking for more!


33/37

And sorry: no more binaries for you

BigTop won't be publishing conveniencebinary artifacts anymore


34/37

Might as well use this expertise:http://www.magnatempusgroup.net


35/37

Come and join us!

Powered By Bigtop:ClouderaEMC

HortonWorksMagnaTempusGroupTrend Micro

https://incubator.apache.org/bigtop/

https://blogs.apache.org/bigtop/


36/37

May be one day...


37/37

Q & A

hug bigtop

Documents