coordination and productivity issues in free software: the role of brooks' law

13/11/2013 BCS Open Source Specialist Group

Measuring Open Source CommunitiesDr. Paul J. Adams & Dr Andrea Capiluppi

What’s This All About?

✤ How to measure?

✤ When to measure?

✤ When not to measure?

✤ Why measure?

Process is at the heart of all software engineering; Free Software is no different. This first part shows how we can measure process and establish how important it really is.

Hint: Very.

Preamble

Who is this dude?

✤ How do we identify him?

✤ What does he do?

✤ Who does he do it with?

✤ Is he productive?

✤ Is he any good?

What Does Open Source Look Like?

How To Measure

Err... What Can We Measure?

✤ VCS Metadata

✤ Code

✤ Communication

✤ Bug Tracking

✤ Documentation

✤ ...

Qt for Android - The Crocodile

Qt Webkit - The Ski Slope

VCS Meta-Data

✤ Developers check code into repository and associated meta-data in stored:

✤ Committer identity

✤ Date/Time

✤ Message

✤ Resources touched

A Commit: Good

commit 5567d08d87e0bd83acd677fb1577a4db76172c0aAuthor: Paul Adams <paul.adams@siriusit.co.uk>Date: Tue May 27 08:47:47 2008 +0000 * No hard-coded absolute URLs please, relatives work There's no need to use "localhost:8080..." type urls in the system. We can simply put "updater?...." or whatever. This fixed a problem in which I couldn't poke the updater from my laptop whilst Alitheia core was running on a different machine.

A Commit: Bad

commit b0748c6ddd294ec8e6bbf39fb5ef5dd8128d0dbfAuthor: Paul Adams <paul.adams@siriusit.co.uk>Date: Mon Dec 3 13:37:47 2007 +0000 * So this is what I consider a decent ascii-art sheep (~)o " " * The other day, however, I was entertained to: (^^)'> " " I still prefer my own. Cast your votes now. Feel free to produce further alternatives.

Who? & When?

Amarok: Before

Amarok: After

Rekonq

What Have We Exposed?

✤ Who is the core team?

✤ Where are the granular tasks?

✤ What is developer turnover like?

✤ What is developer uptake like?

✤ But what do we not see?

✤ Isolation? Structure?

Who & Who? Loose.

Who & Who? Tight.

Cohesion: How Tight Is Tight?

✤ Create cohesion graph (for some time period).

✤ Find shortest paths between all nodes.

✤ Take average.

✤ Err...

✤ That’s it!

Who? & What?

Coordination and ProductivityIssues in Free Software:The Role of Brooks' LawThe Role of Brooks' LawThe Role of Brooks' Law

Rationale for the study✤ “Adding manpower to a late project makes it later.”

✤ Fred Brooks, Jnr.

Context of the study✤ Open Source processes are

fundamentally different from traditional counterparts

✤ Does the Brooks law still hold for Open Source systems?

Brooks: two primary premises

1) Developer’s ability to become productive when joining a new team;

✤ 2) Quality of coordination as the team grows

Testing Brooks laws

✤ Brooks’ Law can be hardly transcended

✤ Does it apply to

✤ * the core team?

✤ * the larger cohort of contributors?

Measuring Brooks' Law

✤ Issue of communication: does communication and coordination quality deteriorate as a Free Software project grows?

✤ Issue of early productivity: how long does it take to fully

train a new contributor?

✤ Measured on KDE, Plone, Evince

Communication paths+graphs

✤ Given a number n of developers, there are n2 − n communication paths

✤ Comm. path: did they work on the same artefact?

✤ Weight: number of artifacts that a pair has jointly worked on

Graph processing

✤ 1) pairwise cohesion (PwC): edge weight between 2 nodes (ex: PwC4→5 = 4)

✤ 2) path cohesion (PaC) = weight of the shortest path between 2 nodes (ex: PaCo1→5 = 5)

✤ 3) coordination cohesion (CCo): average weight of all the shortest paths between the nodes (ex: CCo = 2.67)

Coordination paths: Plone and Evince

Coordination paths in KDE

✤ 1) Few active developers, high cohesion

✤ 2) increasing number of developers, quasi-constant cohesion

✤ 3) Very large number of developers, increasing cohesion

Issues of productivity

✤ Hypothesis: a contributor requires a “ramp up” period before they are fully effective.

✤ first three weeks of contribution indicative of their future effort

Partially engaged: < 1 commit per week in first three weeks

Fully engaged: committing in each

of their first three weeks

Early developers

✤ KDE better at engaging their new contributors (39% fully engaged)

✤ Plone 14%

✤ Evince 9%

Contributors becoming developers

high commit rates both before and after the ramp-up (engaged developers)

high commit rate before butlow commit rate after the

ramp-up (unsuccessful engagement

of developers)

low commit rate before and high commit rate after

the ramp-up (successful engagement

of developers)

low commit rates both beforeand after the ramp-up (sporadic developers)

KDE productivity

✤ KDE effective at obtaining new contributors

✤ Not as effective at keeping those contributors productive over time

Plone productivity

✤ Only 32 of the 131 contributors show improved productivity

✤ Not as capable of bringing new developers as KDE

✤ Less effective at converting new contributors into productive members

Evince productivity

✤ Evince not effective at bringing in new developers

✤ Even less effective at converting contributors into productive team members.

Conclusion/Summary

✤ Two aspects considered:

✤ Communication/Coordination: are OSS projects facing issues as they grow larger?

✤ Productivity: how effective are OSS projects in converting “contributors” to “developers”

Conclusion/Summary

Communication/Coordination:

✤ Phase 1: large coordination effort required to manage few developers (Brooks’ Law applies to Free Software)

✤ Phase 2: coordination effort is diluted as new developers join in

✤ Phase 3 (for KDE): coordination effort ramps up again (due to the KDE internal characteristics)

Conclusion/Summary

✤ Productivity:

✤ Issue of attracting and retaining new developers:

✤ 1) KDE can attract and successfully retain new developers

✤ 2) Evince is successful at attracting new developers, but not as good in keeping them attached

✤ 3) for both KDE and Plone, developers become effective committers within a 30 weeks’ ramp up

Questions?