automatic authorship identification diana michalek, ross t. sowell, paul kantor, alex genkin, david...

Automatic Authorship Identification

Diana Michalek, Ross T. Sowell, Paul Kantor, Alex Genkin, David Madigan, Fred Roberts,

and David D. Lewis

Acknowledgements

• Support– U.S. National Science Foundation

• Knowledge Discovery and Dissemination Program

• Disclaimer– The views expressed in this talk are those of the

authors, and not of any other individuals or organizations.

The Authorship Problem

• Given:– A piece of text with unknown author– A list of possible authors– A sample of their writing

• Problem:– Can we automatically determine which person

wrote the text?

The Authorship Problem

• Given:– A piece of text

– A list of possible authors

– A sample of their writing

• Problem:– Can we automatically determine which person wrote

the text?

• Approach:– Use style markers to identify the author

Motivation and Applications

• Forensics

• Arts


• Forensics– Unabomber

• Arts


• Forensics– Unabomber

• Arts– Shakespeare


• History

• E-mail


• History– Federalist Papers

• E-mail


• History– Federalist Papers

• 85 Total• 12 Disputed

• E-mail


• Counter-Terrorism


• Counter-Terrorism– Osama Bin Laden

Previous Work: Mosteller and Wallace (1984)

• Function Words


• Function Words

Upon Also An

By Of On

There This To

Although Both Enough

While Whilst Always

Though Commonly Consequently

Considerable(ly) According Apt

Direction Innovation(s) Language

Vigor(ous) Kind Matter(s)

Particularly Probability Work(s)


• Function Words

Upon Also An

By Of On

There This To

Although Both Enough

While Whilst Always

Though Commonly Consequently

Considerable(ly) According Apt

Direction Innovation(s) Language

Vigor(ous) Kind Matter(s)

Particularly Probability Work(s)

wk = number times word k appears in text

T = (w1, w2, …, w30)


• Bayesian Inference


• Bayesian Inference

Odds(1, 2 | x) = (p1/p2)[f1(x)/f2(x)]

Final odds = (initial odds)(likelihood ratio)


• Experiment– Use 18 Hamilton and 14 Madison papers to

gather information

• Results


• Experiment– Use 18 Hamilton and 14 Madison papers to

gather information – Test: known Hamilton papers, disputed papers

• Results


• Experiment– Use 18 Hamilton and 14 Madison papers to gather

information – Test: known Hamilton papers, disputed papers

• Results – Strong odds in favor of Hamilton for other known

Hamilton papers

– Strong odds in favor of Madison for all disputed papers

Previous Work: Corney (2003)

• Analyzed email data to determine:– minimum message length – minimum number of messages needed to model

an authors’ style– which stylometric features can be used to

determine authorship


• Stylometric features– Proportion of white-space– Punctuation patterns– Function word frequencies– Frequency of 2-grams– Email-specific features

• Greetings, signatures, html tags


• Conclusions:– Authorship attribution can be successfully

performed– 200-250 words is enough– 20 data points is enough for training– Best feature: function words– Not so great: 2-grams

Our Work: Trials with the Federalist Papers

• Wrote scripts in Perl and Python to compute– Sentence length frequencies– Word length frequencies– Ratios of 3-letter words to 2-letter words

• Analyzed our data with graphing and statistics software.

Sentence Length Frequencies

• Step 1: Parsing the text– What constitutes a sentence?

“Mrs. Jones is has been working on her Ph.D. for 8.5 years.”

“I said no.”“Take the no. 7 bus downtown.”

“What are you talking about ?!?!?!?!!”

“Sometimes….I just feel…anxious.”


• Step 2: Obtain sentence length datai M H

1 1 0

2 0 0

3 0 0

4 1 0

5 9 2

6 6 6

7 14 7

8 22 6

9 16 14

i M H

10 19 21

11 15 20

… … …

30 26 21

31 28 16

32 26 28

… … …

173 0 1

201 1 0

i - sentence lengthM - Number of length-i sentences in

known Madison papers (1139 sentences)

H - Number of length-i sentences inknown Hamilton papers(1142 sentences)


• Step 3: Graph the data

Sentence Length Distributions

• Step 4: Does the data show a difference between Madison and Hamilton?– View sentence lengths as sample data taken

from two distributions– Apply the Kolmogorov-Smirnov test

Kolmogorov-Smirnov Test

• Input:– Two vectors of data values, taken from a

continuous distribution.

• Method:– Examines maximal vertical distance

between empirical cumulative distribution curves

• Output:– p-value

A B

1 4

4 6

3 2

8 7

5 1

A B

1 4

5 10

8 12

16 19

21 20


• Results of step 4:– p-value for sentence length

frequency data is…

0.5121


• Results of step 4:– p-value for sentence length frequency

data is…

• Not too helpful…but there is hope!– Try more features– Try different features

0.5121

Future Work

• Examine email data

• Build our own authorship-identification tool

• Test new stylometric features for distinguishing ability

automatic authorship identification diana michalek, ross t. sowell, paul kantor, alex genkin, david...

Documents