Community Development at the Apache Software Foundation

BOSC 2010, Boston, July 2010Open Bioinformatics Foundation

Ross Gardler@rgardler

rgardler@apache.orgVice President, Community Development

The Apache Software Foundation

● History of The Apache Software Foundation● Accident or design?

● The Apache Way● Flexibility and invariants

● Scaling the foundation● From 8 to 2500+

● Surviving the future● A collection of projects or collaboration between

experts?

History of the ASF

Founding of Apache● Started as “Apache Group” (8 members)● Resumed work on NCSA httpd in Feb 1995

● UIUC put httpd in public domain, but essentially abandoned it

● Chose permissive licensing● Informal corporate structure, until...

Creation of the Foundation● Incorporated with 21 members in 1999

● ~2,500 committers, 291 members, and 53 emritus members today

● Membership-based organisation● 501(c)3 publish charity status

Apache's mission

The Apache Software Foundation provides support for the Apache community of open source

software projects. The Apache projects are characterised by a collaborative, consensus based development process, an open and

pragmatic software licence, and a desire to create high quality software that leads the way in its field.

Apache's tagline

We are more than a group of projects sharing a server, we are a community of developers and users

Or to put it another way...● Let developers focus on what they do best

● Code● Document (if only...)

● Foundation exists to do the rest● “The Apache Way”

● Open development vs. open source● All technical decisions about a project are made in

public (mailing lists)

Indirect financial support● Apache does not pay for development

● Voluntary contributions only● Many (not all) developers are paid by a third-

party to work on the project● Foundation bears indirect support costs

● Infrastructure, legal, publicity, etc.

The Apache Way

We're a meritocracy● Everyone has a voice

● Do stuff and get a vote● Merit is not transferable between projects● Merit across multiple projects or at foundation

level earn you membership

Decision Making● Who makes decisions?

● Community consensus via debate and code● Occasionally a vote is required

● +1, +0, -0, -1● Everyone in the community has a vote

● “binding” vote given to committers● All technical votes cast on public lists

Rule of Lazy Consensus● Trusted parties can just get on with it

● They know when a change will be controversial● State what you intend to do and why● Start doing it● If no-one objects to the idea commit the code

● There's always code review

Scaling the Foundation

Growth of the Foundation● Started with just HTTP Server in 1995● The model “felt” repeatable● Today, we have over 70 top-level projects

● >30 in Incubator● It took over 15 years to get there...

● … but it wasn't smooth ...

Jakarta “Foundation”● Jakarta: “umbrella” for all Java efforts

● Successful as a brand in its own right● Tomcat, Ant, Struts, etc.

● Started to copy the foundation org structure● “mini”-board... but problems arose...● who was responsible?

Importance of Oversight● Jakarta issues led to a lot of navel gazing● Ultimately agreed upon an extremely flat

organisational structure● Very short communication lines● Umbrella projects are bad!

● We killed jakarta: all projects became top-level

Board Oversight● Projects submit quarterly board reports

● By far the most important board activity● Looking for repeating community anti-patterns

● Emerging umbrella projects● Undue commercial influence● Stagnating community● Etc.

● NOT concerned with technical issues

Foundation Structure● Top-Level Projects

● Users● Contributors● Committers● PMC

– Technical management

● Foundation● Members

– Cross-project community

● Board– Oversight

● Foundation projects

What makes a good Apache project?

Community● Diversity

● Minimum of 3 committers ● unrelated to one another outside the project

● Fully audited code● CLAs on file● No incompatible licences

● Practice The Apache Way–

Generic and Reusable● Don't (just) open source a complete application● Open source the component parts

independently● Encourage communities from outside your

domain● Get enhancements to your libraries from others

● Or maybe, you can contribute to someone elses

CC- BY-NC-SA © 2010 Bertrand Delacrataz

CC- BY-NC-SA © 2010 Bertrand Delacrataz

Surviving the Future

Can the ASF scale further?● More projects mean

● Small overhead on foundation● More volunteers at foundation level

● But...● more opinions on what “The Apache Way” is● more potential for abuse of ASF brand

Managing growing pains● Flat structures give power to “newbies”

● Low barriers to entry● No entrance “exam”● Can result in the “blind leading the blind”

● But, hierarchy is inefficient● Higher barrier to entry creates a hierarchy

● So how do we ensure “The Apache Way” survives?● Without stagnating

The Apache Way has the answer● Peer review is core to The Apache Way

● “Just get on with it, we'll object if necessary”● Someone is watching● Objections are supported by debate

– That's how we learn● Mentoring is a small step from peer review

The Incubator● Incubates new projects

● More specifically new project communities● Mentors guide the project team

● Teach The Apache Way

Community Development Projecthttp//community.apache.org

● Google Summer of Code● Code only● Summer only● Students only

● Apache Mentoring Project● Any contribution● Any time● Anyone

Why do this in foundation projects?● Let developers do what they do best: code● But some volunteers want to help newcomers

● Good for recruiting● Good for training● Good for community● Good for getting stuff done

Summary

Separation of Concerns● Foundation

● Brand● Infrastructure● Legal● Cross project community

● Projects● Technical issues● Project community

No leaders, just doers● Give decision making power to the doers● Use lazy consensus to avoid “management by

committee”● Anyone with sufficient merit can veto

● But they should never need to● Consensus through discussion

Generalise● Don't just open source a complete application● Open source components of the application● Most applications address a niche

● You are not unusual in this

Educate● Apache projects are successful because of

“The Apache Way”● There are other ways

● But not in Apache● Better to ensure newcomers understand

● Incubation for project communities● Mentoring for individuals

Questions