Etiquette in Wikipedia: Weening New Editors into

Productive Ones

Wikimedia FoundationRyan Faulkner, Maryana Pinchuk, Steven Walling

Making Wikipedia more welcoming to new editors with templates

Warning Templates in Wikipedia

https://meta.wikimedia.org/wiki/File:First_msg_new_users_proportional_highres.png

Why are warning templates interesting

● 4K new user accounts daily

● 1K go on to make at least one edit

● 20K new users receive at least a first warning per month

● 80% of all first messages are delivered by automated or

semi-automated tools

● Huggle was responsible for 10-40% of first messages

from 2008 to mid 2011 in any given month

How does Huggle work?

http://en.wikipedia.org/wiki/File:Huggle.png

Examples of Issue Specific Warnings

test edits - addition of an edit as a test (not content)

spamming - addition of an external link to the body of an article

unsourced content - new content added to an article without a clear source

deletion - removal of a portion of the content of an article without explanation or a clear reason

Method: The Hypothesis & Experimental Treatments

Friendlier and clearer templates generated by vandal fighting tools can increase the productivity of new editors

Method: Our Eligible Users

In order to be considered for measurement:

1. The warning received must be a first warning2. The editor must not go on to be blocked after the

warning3. The editor must be registered (not an anonymous user)

Experiments ran Nov. 8th, 2011 to Dec. 9th, 2011 inclusive

Method: Our DataFor each editor in the experiment we measure:

1. Timestamp of the warning template event2. User ID and user name3. The number of revisions in all namespaces over the editors lifetime

before and in the three day period after the template event4. The number of warnings in all namespaces over the editors lifetime

before and in the three day period after the template event5. The number of blocks in all namespaces over the editors lifetime

before and in the three day period after the template event

Method: Derived Metricsjafter(u):Edits for editor u in the three day period after the warning

jbefore(u):Edits for editor u in their lifetime before the warning

Gn:

Editor group defined by all editors making a minimum of n edits

m(u) (normalized edit difference):(jbefore(u) - jafter(u)) / jbefore(u)

This is a metric used to incorporate information about an editor's past experience

logit(template(u)) = ß0 + ß1 * m(u)The regression model used to evaluate editor productivity

E1: The editor is delivered a "shortened" message warning message.

E2: The editor is delivered a ``personalized'' message, that is, uses the active voice, explicitly acknowledges that the edit was one made in good faith, and invites discussion on the reverter's talk page.

E3: Combines both a "personalized" and "shortened" warning, and also tests issue specific warnings (This experiment only involved editors receiving warning types:test, delete, spam, unsourced)

Method: Our Experiments

Results: Logistic Regression

Experiment test sample

control sample

Editor Group

ß1 error p-value AIC

E1: short 26 44 G5 -2.151 0.9864 0.0315 88.418

E2: personalized 29 35 G5 -1.496 0.5577 0.135 85.815

E3: mixture w/ specific warnings

32 32 G5 -1.4906 0.5964 0.0124 89.088

Results: Plots

Conclusions

1. We observed a positive effect on normalized edit

difference for the experimental templates. There is clear

merit integrating this approach with existing automated

and semi-automated warning tools

2. We reported our results to the community and instituted

new warnings based on our findings.

3. Follow-up: test a wider set of treatments, try to push

editors to productivity threshold

The End

Thanks.