#geodesummit keynote: creating the future of big data through 'the apache way"
TRANSCRIPT
![Page 1: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/1.jpg)
>>>>>>>>>>>>>>>>>>>>>
CREATING THE FUTURE OF BIG DATA THROUGH "THE APACHE WAY” WHY THIS MATTERS TO THE COMMUNITY
Dr. Justin R. Erenkrantz, Bloomberg LP [email protected] / @jerenkrantz
![Page 2: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/2.jpg)
WHY SHOULD I PAY ATTENTION?
» MentortoApacheGeodeandHAWQ» Commi5ertoApacheHTTPServer,APR,Subversion,Serf» FormerPresidentandDirectorofTheApacheSoBware
FoundaDon» Ph.D.fromUniversityofCalifornia,Irvine» DissertaDon:"ComputaDonalREST:ANewModelfor
Decentralized,Internet-ScaleApplicaDons”» HeadofComputeArchitectureatBloombergLP» ~50billionDcksDAILYflowthroughoursystems
2
![Page 3: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/3.jpg)
TECH @ BLOOMBERG: OPEN SOURCE
3
» ThecoreofourBloombergProfessionalpla\ormhasevolvedawayfromproprietarycode
» FoundaDonsofournext-generaDoninfrastructure-OpenStack,Ceph,Hadoop,Spark,Solr,Chromium,Chef-areallopen-source
» Nolongercanvendorstellusthattheywon’tfixacriDcalbug» Placesalotofpressureonourpartnerstocollaborateopenly
» Givingbacktothecommunity-h"ps://github.com/bloomberg/
» Allowsustoinnovateatthehigherlevels–helpingourcustomersmakesenseofthefirehoseofinformaDonthatisavailabletothem
![Page 4: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/4.jpg)
TECH @ BLOOMBERG: OPEN CAN BE HARDWARE TOO!
4
![Page 5: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/5.jpg)
HISTORY LESSON…
5
» StartedasApacheGroupwith8membersinFeb1995resumingworkonNCSAh5pd» UIUCplacedtheservercodeinpublicdomain» MostoftheUIUCteamleBtojoinNetscape» WebmastersleBinthelurchandjoinedtogether
» TheApacheSoBwareFoundaDonincorporatedin1999» Today,thereareover350communiDesaffiliatedwithApacheperformingover16,000codecommits/month
Why?
![Page 6: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/6.jpg)
PHILOSOPHY OF THE APACHE SOFTWARE FOUNDATION
6
» Letthecontributorsdowhattheydobest:contribute.FoundaDonexiststodotherest.
» DoesnotpayforcontribuDons» Manyaresponsoredbyathird-party» StaffASFhasarefocusedoninfrastructure/PR/etc
» Doesnotpick“winners”or“losers”» “CompeDDon”betweenASFprojectsperfectlyacceptableaslongastherearehealthycommuniDes…thinkGeodeandIgnite(!)
![Page 7: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/7.jpg)
ANTI-PHILOSOPHY
7
» “TheApacheWay”isnot…» DumpingyourcodeonGitHub» Single-sponsorcontribuDons» RunningaBenevolentDictatorship(BDFL)
» TheApacheSoBwareFoundaDonmaynotbebestforallprojects...that’sperfectlyOK.
» IfyouwishtobepartofApache,youneedtoadheretosocialconstructsandnorms
» Technicaldecisionsareuptothecommunitytodecide
![Page 8: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/8.jpg)
ROLE OF APACHE INCUBATOR
8
» Eachproject(TLP)isrunrelaDvelyautonomously» ProjectkarmadoesnotautomaDcallycarryover
» IfIcancommittoGeode,itdoesn’tmeanIcancommittoIgnite!(But,Icouldlikelyearniteasily!)
» Incubatorwasformedin2003aswewerestrugglingtoscalethefoundaDonandrepeatthemodel.Itworked.
» Ifapodlingdoesnothaveahealthycommunity,it’llnevergraduate.That’sOK.IfthepodlingdoesbecomeaTLP,butlaterlosesitscommunity,it’llendupintheArc.That’sOK,too.
![Page 9: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/9.jpg)
TRANSPARENCY & MERITOCRACY
9
» Roy’s Mantra: "If it's not on the list, it didn't happen.” » Apache in the age of GitHub, JIRA, ReviewBoard, etc. » Is the mailing list doomed?
» Generation gap may mean email isn’t preferred » Tools are always secondary to process » Transparency is the aim: allows others to have a voice » The tools and process are never about prohibiting face-to-
face contact - but, ensuring that there is equal access for participation and permitting asynchronous decision making
» Making decisions in a synchronous echo chamber (Slack, IRC, etc.) is not conducive to transparency
![Page 10: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/10.jpg)
MAKING DECISIONS
10
» Voting is the way contributors are (and feel) empowered » “Binding” votes from recognized contributors (PMC)
» Vote on code, ideas, and, most importantly, releases » Minimum acceptable quorum: 3 voters » Minimum acceptable time frame: 72 hours » The power of the dreaded “-1” (veto)
» Code can be vetoed, but not releases » Veto should be cast as a last resort; used to foster
discussion
![Page 11: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/11.jpg)
GROWING COMMUNITY
11
» ContribuDonscancomefromanywhere » Reliesuponcorecontributorsbeingopentoideas
» Yet,thereoBenisasetofagreeduponprinciples» GoingtoGeodecommunityandsaythatyoushouldremoveallconsistencycodeisanon-starter
» Thisisthepowerofthemythical"TheApacheWay”» Meritocracy:accessbasedondemonstratedskills
» MichaelYoung'sTheRiseoftheMeritocracy(1958)–negaDveconnotaDonsacrossanenDresociety
![Page 12: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/12.jpg)
GROWING COMMUNITY
12
» AsadownstreamconsumerofApacheprojects,willtherebesomeonewhoismaintainingthecodebase?CanIhelpvolunteertomaintainit?
» Acodebasebyitselfisinert» Codeisneverperfect,butahealthyandinclusivecommunitywillbeimprovingthecodeconstantlybaseduponfeedbackandothers
» “CommunityoverCode”
![Page 13: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/13.jpg)
ROLES IN INCUBATOR
13
» Thinkofapodlingasbeingprovidedasetoftrainingwheelsastheylearntherulesoftheroad.
» RequiredquarterlyreporDngisoneofthefewmechanismsthattheBoardimposestoallprojectstoensurethatthecommunityishealthy.» Ifnoonesubmitsthereport,noonemaybehome!
» MentorsarearoundtoanswerquesDons,shareknowledge,andbestpracDces.Mentorsarenottheretocontributecode–though,oBenwecould;but,thatroleisdisDnct.
![Page 14: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/14.jpg)
NORMS OF THE COMMUNITY
14
» Overtheyears,mostdisputesIhaveseencomedowntonormsthatwerenotagreeduponordocumented
» ForminganexplicitconsensusonreleaseversioningandcompaDbilityrulesup-frontissoincrediblyhelpful.» Projectsalwayshaveatensionbetween“newfeatures”andcompaDbility.Decidewherethecommunitywantstobeearlyon.
» TheGeodewikisecDonisgreat.Keepitup!
![Page 15: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/15.jpg)
EXPECTATIONS FOR CONTRIBUTORS
15
» ExplicitlycommunicaDngtocontributorswhoarenotyetinPMCwhattheexpectaDonsareforreceivingcommitaccess(vote)toaprojectisextremelyhelpful.
» It’spainfultoseecontributorswhodonotfeelempoweredbythecommunity.It’sahugeredflag.
» Eachprojectcanandshouldsetitsownbar.» MygutfeelingnowistoerronthesideofinclusivenessandgivecommitrightsearlierthanIdid.It’sallunderversioncontrolanyway.Worstcase,revokethatperson’sbit.
![Page 16: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/16.jpg)
GRADUATION
16
» WhenwillApacheGeodegraduatefromIncubator?» “Whenit'sready”istheonlyhonestanswer.» Geodecommunityneedstodemonstratethatitcangovernitselfandbeinclusiveandtransparent
» Itdoesn’thavetobeperfect–nocommunityis.» ThisiswheretheBoardcanbeextremelyhelpful.
» IamextremelyhappytoseetheprogressthatGeodehasmadesofarandwishittheverybestonitspath.
![Page 17: #GeodeSummit Keynote: Creating the Future of Big Data Through 'The Apache Way"](https://reader034.vdocuments.net/reader034/viewer/2022052117/5882b76b1a28abd75a8b76e7/html5/thumbnails/17.jpg)
Join the Apache Geode Community!
• Check out: http://geode.incubator.apache.org
• Subscribe: [email protected]
• Download: http://geode.incubator.apache.org/releases/