using data mining techniques to improve efficiency in police intelligence

Using Data Mining Techniques to Improve Efficiency in Police

Intelligence

Dr Rick Adderley

Recent Riots

• London

Recent Riots

• Birmingham

Recent Riots

• Manchester

Recent Riots – Policing Cuts

The chair of South Wales Police Federation has warned if "savage" cuts go ahead police will not be able to respond effectively to future riots.

The Mayor of London Boris Johnson has warned the government against cutting police numbers. Mr Johnson said the case for cuts had been "substantially weakened" by the riots and that he opposed the Home Secretary's plans to reduce forces' budgets.

Policing Cuts

Views wanted on £134m police cuts in Greater Manchester

Home secretary defends cuts to policing budget

Crime concerns over Surrey Police cutsClaims that crime will rise in Surrey because of cuts to police funding have been rebutted by the force.

Welsh police voice fears over budget cuts

South Yorkshire Police chief warns of crime rise

• How can the provision of intelligence be strengthened and improved in the light of severe cuts?

Providing Operational Intelligence

• Retired Police Inspector– FLINTS

• Business Intelligence Company– 2003– Policing and Security– EU Research Projects

• PhD – Offender Profiling and Crime Trend Analysis

Introduction

• Validated Examples1. Interview Lists2. Early Detection of Crime Series3. Modelling Forensic Recovery4. Automatic Identification of Priority & Prolific

Offenders

Introduction

• IBM SPSS Modeller (Clementine)• SAS Enterprise Miner• Insightful Miner…or…

Data Mining Tools in PolicingAll of the Validated Examples can be accomplished by using:

Data Mining Tools in Policing

• To automatically interrogate one or more data sets with a view to providing information that will save time, reduce crime, deter offending and enhance dynamic business processes.

Data Mining

• Offenders Learn from Offenders

Additional Complication

• Problem Outline– Analysts Time Constraints– Examine Index Crime & Compare with:• Keyword Type Search• Personal Memory

– Produce a List “Matching” Index Crime• 1 to 2 Hours• 10% to 15% Accurate

– Interviewing Officers Refine List

1. Interview Lists

• MLP– Training Set– Testing Set

1. Interview Lists

1. Interview Lists

Billy Smith

Crime

BCU

Beat

PostCode

• Results of Modelling– MLP– Takes Into Account Whole Range of Criminality– Improved Accuracy• 75% to 85%

– Independent Validation• Intelligence Unit Sergeants• Intelligence Unit Analyst

1. Interview Lists

• Spate of Burglaries/Robberies in an Area– Are They Linked– Who May Be Responsible

2. Early Detection of Crime Series

• Self Organising Map

2. Early Detection of Crime Series

2. Early Detection of Crime Series

2. Early Detection of Crime Series

2. Early Detection of Crime Series

Each Cluster willContain Crimes That Are Similar

• Model Current Offenders• Overlay Onto Crime Map

2. Early Detection of Crime Series

2. Early Detection of Crime Series SOMKey ENTRY FEATURE FEATURE_TYPE METHOD ROOMS_SEARCHED SEARCH_TYPE CR_NO LASTNAME FIRSTNAME DOBK6-3 Rear Window Fixed Climbed All Tidy ZZ/1686/05 K6-3 Side Window Fixed Smash All Tidy ZZ/2139/05 K6-3 Rear Window Fixed Smash All Tidy ZZ/2493/05 K6-3 Rear Window Fixed Forced All Tidy ZZ/4371/05 K6-3 Rear Window Fixed Forced All Tidy ZZ/1681/05 KHAN VIJAY 21/04/1977K7-9 Front Door Fixed Insecure Down Tidy ZZ/1862/05 K7-9 Front Door Roller Insecure One Tidy ZZ/1056/05 K7-9 Front Door Fixed Insecure Down Tidy ZZ/1520/05 K7-9 Front Door Fixed Insecure Down Tidy ZZ/1578/05 K7-9 Front Door Fixed Insecure One Tidy ZZ/2056/05 K7-9 Front Door Fixed Insecure Down Tidy ZZ/2256/05 K7-9 Front Door Fixed Insecure One Tidy ZZ/3937/05 K7-9 Front Door Fixed Insecure Many Tidy ZZ/694/05 BULLEN IAN 11/11/1982K7-9 Front Door Fixed Insecure One Tidy ZZ/3332/05 SINGH JARNAIL 29/08/1979K9-2 Rear Window Casement Smash All UnTidy ZZ/305/05 K9-2 Rear Window Transom Forced All UnTidy ZZ/457/05 K9-2 Rear Window Casement Smash All UnTidy ZZ/788/05 K9-2 Rear Window Casement Smash All UnTidy ZZ/1587/05 K9-2 Rear Window Casement Smash All UnTidy ZZ/1865/05 K9-2 Rear Window Casement Smash All UnTidy ZZ/1979/05 K9-2 Rear Window Transom Smash All UnTidy ZZ/1980/05 K9-2 Rear Window Transom Forced All UnTidy ZZ/3309/05 K9-2 Rear Window Casement Smash All UnTidy ZZ/1668/05 BURTON ALAN 09/06/1979

• Northamptonshire Forensic Science Department– Dr John Bond

• Motivation:-

3. Modelling Forensic Recovery

3. Modelling Forensic Recovery

• Northamptonshire Forensic Science Department– Dr John Bond

• Motivation:-– Which Crimes Should be Attended First?– Which Crime Give Best Opportunity of Forensic

Recovery?

3. Modelling Forensic Recovery

• CRISP-DM– www.the-modeling-agency.com/crisp-dm.pdf

– Naïve Bayes Algorithm– Q-Prop Neural Network Algorithm

3. Modelling Forensic Recovery

3. Modelling Forensic Recovery

• Algorithm Results– 10 Fold Cross Validation on 28,490 Volume Crime

Records

Basic Variables Basic + Temporal Basic + MO vars

Bayes 68.59 69.2 74.5

Q-Prop 63.59 64.01 69.5

3. Modelling Forensic Recovery

• Results Using Live Data

Crime Ref SubDiv Offence Set Month Forensic Collected

Probability Predict Class

NN/24163/03

NN BDwell December 1 0.82 1

NC/2835/05 NC BDwell May 1 0.68 1

NN/7587/02 NN BOther June 0 0.67 1

NN/16787/02

NN TOMV October 1 0.62 1

NN/12133/04

NN TOMV July 1 0.62 1

NC/1757/01 NC TOMV May 0 0.49 0

NC/14454/02

NC TOMV November 0 0.37 0

NN/31877/03

NN BOther March 0 0.20 0

NW/14184/05

NW TFMV February 0 0.16 0

ND/6657/00 ND TFMV January 0 0.16 0

3. Modelling Forensic Recovery

• Gwent Police Trial– 11,800 Volume Crime Records– 10 Fold Cross Validation• Q-Prop Accuracy 81.79%• Naïve Bayes Accuracy 88.84%

3. Modelling Forensic Recovery

• How Good are the Models?

3. Modelling Forensic Recovery

• Every Northamptonshire CSI– 50 Random Crimes– Assess whether a Forensic Sample would be

collected

3. Modelling Forensic Recovery

• Every Northamptonshire CSI– 50 Random Crimes– Assess whether a Forensic Sample would be

collected• 41% Accuracy

• Which Offenders are Causing most Harm• Including Cross Border Offenders – Force Priorities– Harm Matrix

4. Priority & Prolific Offenders

• Which Offenders are Causing most Harm – Current Process:• Offender is “Nominated”• Scored Against Matrix• Placed on List

– Infrequently Reviewed• Insufficient Time• 20 Minutes to 2 Hours to Complete Scoring

4. Priority & Prolific Offenders

• Automated Process:

4. Priority & Prolific Offenders

4. Priority & Prolific OffendersOffender L2Offender

Offend Priority Nab Live Priority Nab Community Safety

Control Of Offenders Reduce Crime

Total Prism Score BEAT AREA CRIME

Eric Smith 20 10 6 0 0 62 98 Z2

Paul Jones 20 0 6 0 0 71 97 Z2

Mary Hands 20 10 6 4 0 54 94 Z1

John Fresh 20 0 0 20 20 30 90 Z2

Ali Khan 20 10 6 0 0 54 90 Z1

Ming Hu 20 10 6 4 0 50 90 Z1

Graham Zhu 20 10 6 0 0 50 86 Z1

Fred Brown 20 0 0 0 0 60 80 Z1

Sally Johns 20 12 6 0 0 40 78 Z1

David Green 20 0 0 16 10 30 76 Z2

Alison Blue 20 10 6 0 0 40 76 Z1

Tom Black 20 0 0 4 0 50 74 Z2

Vinny Smith 20 0 0 24 0 30 74 Z1

Saad Wang 20 12 6 0 0 36 74 Z1

Mendip Kaur 20 12 0 0 0 40 72 Z1

Brian Ling 20 0 0 0 0 52 72 Z1

Billy Smith 20 10 0 0 20 22 72 Z1

Ho Tu 20 10 6 0 0 36 72 Z1

Paul Wells 20 0 0 0 30 20 70 Z2

4. Priority & Prolific OffendersOffender

A1

A2

A3

A4

A5

A6

A7

A8

A9

B1

B2

B3

B4

B5

B6

B7

B8

C1

C2

C3

C4

TotalCrimes

NumBCUs

OffenderLatest BCU

Eric Smith 0 0 0 0 0 3 0 0 0 5 1 3 5 0 0 0 0 0 0 0 0 17 5 X1

Paul Jones 0 0 1 0 0 0 4 0 0 0 0 1 0 0 0 0 0 0 0 0 1 7 4 V3

Mary Hands 0 0 0 0 0 0 0 0 1 4 0 0 0 1 0 0 0 1 0 0 0 7 4 V2

John Fresh 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 4 4 V2

Ali Khan 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 1 0 1 0 0 4 4 V2

Ming Hu 0 4 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 5 0 0 10 3 U2

Jill King 0 0 0 0 0 0 0 4 0 0 0 0 1 0 0 0 3 0 0 0 0 8 3 W1

Fred Brown 0 0 0 2 0 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 7 3 Z1

Sally Johns 0 0 2 4 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 7 3 Z1

Lin Ho Pu 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 4 0 0 0 6 3 V2

Brian Ling 3 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 1 0 0 0 0 6 3 T1

Billy Smith 0 0 0 0 0 0 3 0 0 0 1 0 0 0 0 0 1 0 0 0 0 5 3 U2

Ho Tu 0 2 0 1 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 5 3 M3

Paul Wells 0 0 0 0 0 0 0 0 3 0 1 0 0 0 0 0 1 0 0 0 0 5 3 T2

• Motivation:– Sample Offender Test• Same Offender• 4 BSU’s Where Offender Not Known• 20 Minutes to 2 Hours• Scores From Low 100’s to High 400’s

– Place / Not Place on List

• Scores Not Related To Time Taken

4. Priority & Prolific Offenders

• Benefits:– Every Offender is Scored– Objective Scoring• Defendable• Repeatable

– Process Can Be Frequently Run• Offenders’ Scores Updated• Current

4. Priority & Prolific Offenders

• Offender Networks:– The Data WILL Contain Networks– Which Networks Cause the Most Harm• Prioritisation• Scoring

– Degrees of Freedom• Dependant Upon Priorities

4. Priority & Prolific Offenders

• Offender Networks:

4. Priority & Prolific Offenders

Questions?

Using Data Mining Techniques to Improve Efficiency in Police

Intelligence

Dr Rick Adderleywww.a-esolutions.com