Where's my monitoring at?

André Klapper <ak-47@gmx.net>GUADEC 2007, Birmingham

Some random figures about a software project called GNOME(and a pretty untechy talk, compared to the rest here)

Monitoring

● Measurement & numbers for decision making support & a more optimal use of resources

● continuous process on various levels, can end up in defining and better documenting of processes and assigning of responsibilities

What is this about, dude?

● GNOME = not a company, but nevertheless professional?

● identifying problems with the help of some automatization – some examples what we have, and what we don't have

● end up in better overall quality and happier end customers by clearer internal processes?

„Where are we, where do we wanna be?“

GNOME: Structure

Here:● Release Team● Docs● GTP/I18N● Bugsquad● („Hacking“ only

w.r.t maintainers)

Source: https://twiki.softwarelivre.org/bin/viewfile/Main/ VicenteAguiar?rev=1;filename=OrganogramaGNOME2.4_.png

by Vicente Aguiar

Release Team (RT)

● Aim: Test & publish perfect(TM) releases [partially helped by GARNOME and the BuildBrigade (-> Thursday 19th, 16:00)]

● Tasks: Release Engineering, Schedule Planning, Nagging, module inclusion final decisions, assigning Release Notes

● Resources: 8 persons, see http://live.gnome.org/ReleasePlanning/Membership

● Stats: no, what for? :-)● Problems: response times, manpower/slacking.

RT: Break request amounts and response times

● Rules: http://live.gnome.org/ReleasePlanning/ RequestingFreezeBreaks

● ABI/API UI String Hardcode2.13 Amount 2 2 24 21

>1 >1, >3

2.17 Amount 1 1 5 10>1 >1 >1

ø Resp.Time (in days)

18× >1, 3× >2

ø Resp.Time (in days)

2× >1, 2× >2, 1× >4, 1× >9

RT: Wrong/missing announcements of maintainers

● miss of informing any of release-team, desktop-devel-list, gnome-i18n and gnome-doc-list (http://live.gnome.org/MaintainersCorner)

●

● Some modules do not have strings, therefore no announcements were sent to gnome-i18n@. Table results by searching for „branch“ in subject of the four mailing lists)

total correct missed...r-t i18n d-d-ldoc all

2005/09/05 – 2006/03/12 (2.13) 41 23 12 1 3 6 02006/03/13 – 2006/09/03 (2.15) 46 20 19 2 12 11 02006/09/04 – 2007/03/11 (2.17) 54 26 19 2 16 8 1

Maintainers' quick guide

ActionBranching % always inform inform inform informString Change trunk Announcement Period inform inform

trunk Freeze period request

stable inform informstable ...else: inform request

UI Change trunk Announcement Period informtrunk Freeze period request requeststable always request request

Code Change trunk Hardcode Freeze request

Code- base

Time http://live.gnome.org/Schedule

(accidentially forgotten to mark)

release-team@

gnome-i18n@

gnome-doc-list@

desktop-devel-list@

● Resources: pure luck?● Problems: last cycles: always new writers;

different lenghts & styles => no consistency● 2.18 release notes written in the last days, no

volunteers● 2.20: volunteers available, better planning and

embedded into the schedule (timeframe: august 6th – September 10th) => looks good ;-)

RT: Release notes

User Documentation

● http://live.gnome.org/DocumentationProject● Aim: Provide documentation for applications● Tasks: Write and update documentation● Resources, Stats: ?● Problems: not always up-to-date (only a few

weeks between freeze and release) => manpower (?); no way of tracking actuality

Documentation: How up to date?● Outdated if UI changes a lot => developers

have to inform the doc writers● Docs that haven't been updated for a long time:

svn  log ­ ­limi t 1 h ttp:/ /akla pper@ svn.g nome. org/ svn/ $modu le/tr unk/{d oc,he lp,do cs}/C /$mod ule.x ml 

(excluded build fixes, migration to g-d-u)● => oldest docs: bug-buddy; gnome-keyring-manager

(2005-03-25), fast-user-switch-applet, zenity, gnome-applets\battstat (2005-12-11)

● http://l10n.gnome.org/module/: add „last change“ info?

I18N/GTP

● Aim: Translate UI and Documentation to other languages

● Resources: 2 GTP spokepersons (http://live.gnome.org/TranslationProject/SpokesPersons), >100 language teams (http://developer.gnome.org/projects/gtp/ teams.html), each one with one coordinator and an unknown number of translators

I18N/GTP

● Stats: damned-lies at http://l10n.gnome.org per language, module, team, and release set. Powerful.

● Tasks of GTP: String Freeze Break Approvals, Language Team Maintainership changes, new languages approval: gnome-18n@ for entry at l10n.gnome.org, bugzilla team for creation of L10N component, and later on accounts-team for svn permissions (only 3 1/2 people on accounts team)

I18N Teams: Maintenance

● http://l10n.gnome.org/languages/ lists 133 langs●

● (note: partially buggy stats due to http://bugzilla.gnome.org/416009)

Strings: 2.14 2.18UI ~35000 36916Docs ~18100 24094

I18N Teams: UI Translation changes within the last 12 months

Positive (>10%) Negative (>-7%)Lang 2.14 2.18 diff Lang 2.14 2.18 diffBengali Indian 9 80 71 Nepali 97 77 -20Malayalam 19 76 57 Albanian 98 84 -14Slovenian 40 89 49 Indonesian 96 84 -12Oriya 18 61 43 Czech 98 87 -11Marathi 9 52 43 Croatian 65 54 -11Malagasy 8 50 42 Romanian 93 83 -10Arabic 57 99 42 Norwegian Nynorsk 80 70 -10Tamil 67 85 18 Serbian Latvian 98 89 -9

Serbian 98 89 -9Welsh 97 88 -9Canadian English 96 87 -9Latvian 97 89 -8Georgian 66 58 -8Slovak 61 54 -7

GNOME 2.18: 63 langs with UI translation >50%

Source: Stats as per http://l10n.gnome.org/releases/gnome-2-1* on 2007-07-10, 02:00 UTC

I18N Teams: Docu Translation changes within the last 12 months

Docs with >10% status ComparisonLang 2.14 2.18 diff Lang UI DocsSpanish 71 88 17 Arabic 99 0French 34 85 51 Dzongkha 99 0Swedish 24 70 46 Macedonian 99 0Italian 41 44 3 Hungarian 99 2Russian 14 40 26 Catalan 99 2British English 0 34 34 Finnish 99 2Ukrainian 33 32 -1 Lithuanian 99 0Brazilian Portuguese25 27 2 Danish 99 0Punjabi 5 18 13 Japanese 99 1Portuguese 0 16 16 Vietnamese 99 0Korean 0 14 14German 0 12 12Bulgarian 11 12 1 63 langs with UI >50%, but

13 langs with Docs >10%?

Source: Stats as per http://l10n.gnome.org/releases/gnome-2-1* on 2007-07-10, 02:00 UTC

I18N Teams: Maintainer changes and other non-“freeze requests“

2.13 9 1 82.15 12 6 72.17 9 0 6

Maintainership change or drop actively by maintainer himself, or after a complaint with positive reaction of maintainer to change

Complaint about unresponsiveness, and NO response by the team leader => maintainership change

Granting for SVN Access; webpage or email address change, or wrong maintainer in b.g.o

● data taken from gnome-18n@ mailinglist

● response times?

I18N Teams: Maintainer changes etc: Response times of GTP

●

● some requests perhaps answered off-list, or really unanswered...

Version2.13 17 16 1 7,062.15 23 20 2,5 7,752.17 13 8 0 2,38

Number of requests

„Trackable“ requests

Median [in days]

Average [in days]

GTP: Announcement amounts

●

● not significant

2.14 (6 weeks) 2.18 (8 weeks)Total 163 69Top 3 evolution (58) gedit (8)

gnome-utils (13) eog (6)gnome-panel (12) evolution (6)

Bugsquad

● Aim: Quality Assurance (http://live.gnome.org/Bugsquad)

● Tasks: Keeping track of bugs, make major bugs not go unnoticed, user help and feedback.

● Resources: Technical bugmasters and varying number of triagers (2-8 „hardcore“ triagers, plus maintainers and some „normal“ triagers)

● Existing stats: b.g.o/weekly-bug-summary.cgi, b.g.o/duplicates.cgi, Patches: b.g.o/reports/patch-diligence-report.cgi, b.g.o/reports/patch-report.cgi (general)

Bugsquad: Workload

2005 2006 2007Amount of Days: 365 365 193

18888 26942 30691Opened in that period: 37845 67543 64952

+78%+81%

Closed: 34196 59006 62413

3211 8371 6788

Current open reports, excludes enhance. req.:

#Bugs closed by top bug closer

All-time: 15842

Sources: <http://developer.gnome.org/projects/bugsquad/statistics/2005/> and /2006/>, <http://bugzilla.gnome.org/reports/weekly-bug-summary.cgi?days=194&products=0>, <http://mail.gnome.org/archives/desktop-devel-list/2006-January/msg00004.html>, Olav

Top 10 Bug closers in 2007

Bugs without a responseProductEvolution 4259 1045 24,54%nautilus 2057 665 32,33%gtk+ 2015 406 20,15%doxygen 650 344 52,92%rhythmbox 818 100 12,22%gnome-vfs 456 120 26,32%epiphany 612 23 3,76%GnuCash 514 38 7,39%gnome-panel 436 69 15,83%gnome-applets 377 124 32,89%gimmie 296 233 78,72%Gnumeric 486 55 11,32%Evolution Exchange 288 117 40,63%GtkHtml 319 65 20,38%Gstreamer 446 56 12,56%

Open bugs (incl. enhanc.)

Bugs without response

Source: top 15 modules of the last 365 days (http://bugzilla.gnome.org/reports/weekly-bug-summary.cgi?days=365&products=30) as per 2007/07/13 UTC 02:45

Graphical Reports?

Source: http://bugzilla.ximian.com/reports.cgi?product=Evolution& output=show_chart&datasets=NEW%3A&banner=1&quip=0

Bugsquad

● Problems: Amount of untriaged bugs, response rate and time; understaffed products (bugzilla mail ignored by maintainers and only up to triagers), quality of stacktraces and installing debug packages, hacked up Auto-reject feature, manpower, Patch nagging (-> Patchsquad BoF on Saturday 21st, 10:30)

● 2.20: Breakpad Support (see Fernando Herrera's „Managing bugs and crashes in GNOME“, Sa 21st July, 14:00)

● => Major aim: reduce workloud

Bugsquad: Flood me, baby!

Bugsquad: Auto-reject

● http://live.gnome.org/Bugsquad/AutoReject●

Bugsquad: Auto-reject

Date 2007/05/08 2007/06/10 2007/07/10Nautilus 8925 9347 9603Evolution 10604 11190 11524TOTAL 26236 27571 28619

Dead modules in SVN

● Problems: Identify dead modules (bit rott) => translators waste time; or new maintainership needed?

● "svn log --limit 5" to find modules without recent changes, then send emails to maintainers: http://live.gnome.org/ThomasAndersen/CvsCleanup, but only average response rate (note to myself: nag again)

● waiting for bkor to set up an SVN archive, should happen within thre next months(TM)

Other stuff

● Bugzilla: Response times (confirmed -> new; first comment of !reporter) and rates

● Accounts Team (http://live.gnome.org/AccountsTeam) needs more people (complaints about waiting weeks to get SVN access at gnome-i18n@)

Summary

● We have some nice snapshot stats, but we miss stats showing changes of a period of time => easier to identify dangerous developments

● sometimes communication fails (even if a process is well-documented)

● We could probably be more successful and happier with better organization... conclusions?

Thanks for your interest!