The Lane’s Gifts vs. Google The Lane’s Gifts vs. Google Report part 2Report part 2

Andrew HolomanAndrew Holoman

Google’s Approach to Detecting Invalid ClicksGoogle’s Approach to Detecting Invalid ClicksGoogle’s Click Quality Team’s Mission Statement states:Google’s Click Quality Team’s Mission Statement states:Protect Google’s advertising network and provide excellent Protect Google’s advertising network and provide excellent

customer service to clients by: customer service to clients by: – Vigilantly monitoring invalid clicks/impressions and Vigilantly monitoring invalid clicks/impressions and

removing its sourceremoving its source– Reviewing all client requests and responding in a timely Reviewing all client requests and responding in a timely

mannermanner– Developing and improving systems that remove invalid Developing and improving systems that remove invalid

clicks/impressions and properly credit clients for invalid clicks/impressions and properly credit clients for invalid traffictraffic

– Educating clients and employees on invalid Educating clients and employees on invalid clicks/impressionsclicks/impressions

The Click Quality Team practices this by raising the The Click Quality Team practices this by raising the levels of click fraud detection methods where it becomes levels of click fraud detection methods where it becomes hard and unrewarding to commit click fraud against hard and unrewarding to commit click fraud against GoogleGoogle

Strategic ObjectivesStrategic Objectives

Google tries to achieve these objectives in Google tries to achieve these objectives in two ways:two ways:

– PreventionPrevention Discouraging invalid clicking activities on its Discouraging invalid clicking activities on its

Network by making life of unethical users more Network by making life of unethical users more difficult and less rewardingdifficult and less rewarding

– DetectionDetection Detecting and removing invalid clicks and the Detecting and removing invalid clicks and the

perpetratorsperpetrators

Preventive MeasuresPreventive Measures

Google tries to build mechanisms for preventing Google tries to build mechanisms for preventing invalid clicking that reduce inappropriate activities invalid clicking that reduce inappropriate activities and some of these are: and some of these are:

– Making it hard to create duplicate accounts and open Making it hard to create duplicate accounts and open new accounts after the old ones are terminatednew accounts after the old ones are terminated

– Making it hard to register using false identitiesMaking it hard to register using false identities

– Development of certain mechanisms that automatically Development of certain mechanisms that automatically reduce fraudulent activitiesreduce fraudulent activities

Characterization of Invalid Click DetectionCharacterization of Invalid Click Detection

Online filtering vs. Offline monitoring and analysisOnline filtering vs. Offline monitoring and analysis– Are there some time constraints on how fast the invalid click Are there some time constraints on how fast the invalid click

detection should be done?detection should be done? Automated vs. Manual detectionAutomated vs. Manual detection

– Were invalid clicks detected by a special-purpose software or Were invalid clicks detected by a special-purpose software or by human expert?by human expert?

Proactive vs. Reactive detectionProactive vs. Reactive detection– Has the detection of invalid clicks occurred before or after the Has the detection of invalid clicks occurred before or after the

advertisers complaint ?advertisers complaint ? Where were invalid clicks made?Where were invalid clicks made? Were invalid clicks associated with the AdSense or Were invalid clicks associated with the AdSense or

AdWords programs ?AdWords programs ?

Process of Detecting and Removing Invalid Process of Detecting and Removing Invalid ClicksClicks

Pre-filtering Pre-filtering - removal of the most obvious invalid - removal of the most obvious invalid clicks, such as “testing” and “meaningless” clicks clicks, such as “testing” and “meaningless” clicks before they are even seen by the filtersbefore they are even seen by the filters

Online FilteringOnline Filtering – several online filters monitor – several online filters monitor various logs for satisfying these conditions and various logs for satisfying these conditions and detect the clicks in these logs satisfying these detect the clicks in these logs satisfying these conditions; such clicks are marked as “invalid” conditions; such clicks are marked as “invalid” and are subsequently removedand are subsequently removed

Post-filteringPost-filtering – offline detection and removal of – offline detection and removal of invalid clicks that managed to pass the online invalid clicks that managed to pass the online filtering stagefiltering stage

Post-filtering Sub-stagesPost-filtering Sub-stages

What are the two sub-stages of post-filtering?What are the two sub-stages of post-filtering?

– Automated monitoringAutomated monitoring

– Manual reviewsManual reviews

Performance of Online FiltersPerformance of Online Filters

A true click is either valid or invalidA true click is either valid or invalid Google filters can label a click as valid or Google filters can label a click as valid or

invalidinvalid The actual click vs. click labeling can raise The actual click vs. click labeling can raise

confusion within the filtersconfusion within the filters– In finding the accurate number of valid and In finding the accurate number of valid and

invalid clicks: invalid clicks: TP+TN+FP+FN=NTP+TN+FP+FN=N– The accuracy rate is equal to (The accuracy rate is equal to (TP+TN)/NTP+TN)/N– The error rate is equal to (The error rate is equal to (FP+FN)/NFP+FN)/N

Indirect EvidenceIndirect Evidence

Two main pieces of evidence for Google’s Two main pieces of evidence for Google’s filters:filters:– Newly introduce and revised filters detect only Newly introduce and revised filters detect only

few additional invalid clicksfew additional invalid clicks– The offline invalid click detection methods The offline invalid click detection methods

detect relatively few invalid clicks in comparison detect relatively few invalid clicks in comparison to the filtersto the filters

Performance of FiltersPerformance of Filters

Why do Google's simple filters work well Why do Google's simple filters work well most of the time?most of the time?– The Combination of FiltersThe Combination of Filters– Extra complexity of some of the filtersExtra complexity of some of the filters– Simplicity of most of the attacksSimplicity of most of the attacks– The Long Tail of invalid clicksThe Long Tail of invalid clicks

Frequency

RankLong TailLeft Part

Conflict of InterestConflict of Interest

Two cases:Two cases:

– When one particular IP address was disabled When one particular IP address was disabled due to inappropriate clicking activitiesdue to inappropriate clicking activities

– The change in the double-click policy that was The change in the double-click policy that was considered in the Winter of 2005 and considered in the Winter of 2005 and implemented in March 2005implemented in March 2005

Automated Offline Detection Automated Offline Detection MethodsMethods

AlertsAlerts

Auto-termination system for publishersAuto-termination system for publishers

AlertsAlerts

They are used for detecting more complex They are used for detecting more complex and more subtle patterns of invalid clicking and more subtle patterns of invalid clicking activities that may or may not be validactivities that may or may not be valid– In what stage do alerts identify invalid clicks?In what stage do alerts identify invalid clicks?

– What are the two types of alerts?What are the two types of alerts?

– What are the two ways that alerts are issued?What are the two ways that alerts are issued?

Auto-termination System for AdSense Auto-termination System for AdSense PublishersPublishers

It’s an automated offline system for It’s an automated offline system for detecting the AdSense publishers who detecting the AdSense publishers who performing inappropriate behavior by performing inappropriate behavior by violating the Terms and Conditions of the violating the Terms and Conditions of the AdSense programAdSense program– What are these conditions? What are these conditions?

Manual Offline Detection Manual Offline Detection MethodsMethods

Investigation of AdvertisersInvestigation of Advertisers

Investigation of PublishersInvestigation of Publishers

Investigation of AdvertisersInvestigation of Advertisers

Advertiser complaintsAdvertiser complaints: an advertiser notices : an advertiser notices unusual clicking activities and requests Google unusual clicking activities and requests Google to investigate those activities for the presence of to investigate those activities for the presence of invalid clicksinvalid clicks

AlertsAlerts: alerts detect unusual patterns of : alerts detect unusual patterns of behavior of advertisers and trigger manual behavior of advertisers and trigger manual investigations of these patternsinvestigations of these patterns

Customer Service RepresentativesCustomer Service Representatives: they may : they may request to investigate an advertiser based on request to investigate an advertiser based on the advertiser’s request or based on their own the advertiser’s request or based on their own initiativesinitiatives

Investigations of the PublishersInvestigations of the Publishers

Publishers complaintPublishers complaint Advertiser’s complaintAdvertiser’s complaint Auto-termination systemAuto-termination system ClassifierClassifier Detection of duplicate publishersDetection of duplicate publishers Second-review publishersSecond-review publishers Customer service representativesCustomer service representatives Requests from the Click Quality TeamRequests from the Click Quality Team

Results of InvestigationsResults of Investigations

The invalid clicks are presentThe invalid clicks are present

No invalid clicks are presentNo invalid clicks are present

It is unclear if invalid clicks are presentIt is unclear if invalid clicks are present