coordination and productivity issues in free software: the role of brooks' law
DESCRIPTION
BCS Open Source Specialist GroupTRANSCRIPT
13/11/2013 BCS Open Source Specialist Group
Measuring Open Source CommunitiesDr. Paul J. Adams & Dr Andrea Capiluppi
What’s This All About?
✤ How to measure?
✤ When to measure?
✤ When not to measure?
✤ Why measure?
Process is at the heart of all software engineering; Free Software is no different. This first part shows how we can measure process and establish how important it really is.
Hint: Very.
Preamble
Who is this dude?
✤ How do we identify him?
✤ What does he do?
✤ Who does he do it with?
✤ Is he productive?
✤ Is he any good?
What Does Open Source Look Like?
How To Measure
Err... What Can We Measure?
✤ VCS Metadata
✤ Code
✤ Communication
✤ Bug Tracking
✤ Documentation
✤ ...
Qt for Android - The Crocodile
Qt Webkit - The Ski Slope
VCS Meta-Data
✤ Developers check code into repository and associated meta-data in stored:
✤ Committer identity
✤ Date/Time
✤ Message
✤ Resources touched
A Commit: Good
commit 5567d08d87e0bd83acd677fb1577a4db76172c0aAuthor: Paul Adams <[email protected]>Date: Tue May 27 08:47:47 2008 +0000 * No hard-coded absolute URLs please, relatives work There's no need to use "localhost:8080..." type urls in the system. We can simply put "updater?...." or whatever. This fixed a problem in which I couldn't poke the updater from my laptop whilst Alitheia core was running on a different machine.
A Commit: Bad
commit b0748c6ddd294ec8e6bbf39fb5ef5dd8128d0dbfAuthor: Paul Adams <[email protected]>Date: Mon Dec 3 13:37:47 2007 +0000 * So this is what I consider a decent ascii-art sheep (~)o " " * The other day, however, I was entertained to: (^^)'> " " I still prefer my own. Cast your votes now. Feel free to produce further alternatives.
Who? & When?
Amarok: Before
Amarok: After
Rekonq
What Have We Exposed?
✤ Who is the core team?
✤ Where are the granular tasks?
✤ What is developer turnover like?
✤ What is developer uptake like?
✤ But what do we not see?
✤ Isolation? Structure?
Who & Who? Loose.
Who & Who? Tight.
Cohesion: How Tight Is Tight?
✤ Create cohesion graph (for some time period).
✤ Find shortest paths between all nodes.
✤ Take average.
✤ Err...
✤ That’s it!
Who? & What?
Coordination and ProductivityIssues in Free Software:The Role of Brooks' LawThe Role of Brooks' LawThe Role of Brooks' Law
Rationale for the study✤ “Adding manpower to a late project makes it later.”
✤ Fred Brooks, Jnr.
Context of the study✤ Open Source processes are
fundamentally different from traditional counterparts
✤ Does the Brooks law still hold for Open Source systems?
Brooks: two primary premises
1) Developer’s ability to become productive when joining a new team;
✤ 2) Quality of coordination as the team grows
Testing Brooks laws
✤ Brooks’ Law can be hardly transcended
✤ Does it apply to
✤ * the core team?
✤ * the larger cohort of contributors?
Measuring Brooks' Law
✤ Issue of communication: does communication and coordination quality deteriorate as a Free Software project grows?
✤ Issue of early productivity: how long does it take to fully
train a new contributor?
✤ Measured on KDE, Plone, Evince
Communication paths+graphs
✤ Given a number n of developers, there are n2 − n communication paths
✤ Comm. path: did they work on the same artefact?
✤ Weight: number of artifacts that a pair has jointly worked on
Graph processing
✤ 1) pairwise cohesion (PwC): edge weight between 2 nodes (ex: PwC4→5 = 4)
✤ 2) path cohesion (PaC) = weight of the shortest path between 2 nodes (ex: PaCo1→5 = 5)
✤ 3) coordination cohesion (CCo): average weight of all the shortest paths between the nodes (ex: CCo = 2.67)
Coordination paths: Plone and Evince
Coordination paths in KDE
✤ 1) Few active developers, high cohesion
✤ 2) increasing number of developers, quasi-constant cohesion
✤ 3) Very large number of developers, increasing cohesion
Issues of productivity
✤ Hypothesis: a contributor requires a “ramp up” period before they are fully effective.
✤ first three weeks of contribution indicative of their future effort
Partially engaged: < 1 commit per week in first three weeks
Fully engaged: committing in each
of their first three weeks
Early developers
✤ KDE better at engaging their new contributors (39% fully engaged)
✤ Plone 14%
✤ Evince 9%
Contributors becoming developers
high commit rates both before and after the ramp-up (engaged developers)
high commit rate before butlow commit rate after the
ramp-up (unsuccessful engagement
of developers)
low commit rate before and high commit rate after
the ramp-up (successful engagement
of developers)
low commit rates both beforeand after the ramp-up (sporadic developers)
KDE productivity
✤ KDE effective at obtaining new contributors
✤ Not as effective at keeping those contributors productive over time
Plone productivity
✤ Only 32 of the 131 contributors show improved productivity
✤ Not as capable of bringing new developers as KDE
✤ Less effective at converting new contributors into productive members
Evince productivity
✤ Evince not effective at bringing in new developers
✤ Even less effective at converting contributors into productive team members.
Conclusion/Summary
✤ Two aspects considered:
✤ Communication/Coordination: are OSS projects facing issues as they grow larger?
✤ Productivity: how effective are OSS projects in converting “contributors” to “developers”
Conclusion/Summary
Communication/Coordination:
✤ Phase 1: large coordination effort required to manage few developers (Brooks’ Law applies to Free Software)
✤ Phase 2: coordination effort is diluted as new developers join in
✤ Phase 3 (for KDE): coordination effort ramps up again (due to the KDE internal characteristics)
Conclusion/Summary
✤ Productivity:
✤ Issue of attracting and retaining new developers:
✤ 1) KDE can attract and successfully retain new developers
✤ 2) Evince is successful at attracting new developers, but not as good in keeping them attached
✤ 3) for both KDE and Plone, developers become effective committers within a 30 weeks’ ramp up
Questions?