voluntary action against cybercrime

Voluntary action against cybercrimeMeasuring and increasing its impact

Tyler Moore Lyle School of Engineering, Southern Methodist University, Dallas, TX, USA

Michel van Eeten Faculty of Technology, Policy and Management , Delft University of Technology, NL

Talk describes joint work with Johannes M. Bauer (Michigan State), Hadi Asghari (TU Delft), Richard Clayton (Cambridge), Shirin Tabatabaie (TU Delft), and Marie Vasek (SMU)

Contact: [email protected] Action Plan Meeting, Montreal22 October 2013

Motivation and context

• Wicked content and actors pervade cyberspace

1. Websites (distribute malware, host phishing,…)

2. End-user machines (botnets,…)• Most cleanup carried out by private actors

voluntarily• Incentives of Internet intermediaries to cooperate

largely determines effectiveness of response• Victim• Requesting party (often the victim, security cos.)• Party receiving notice (e.g., ISPs, hosting providers)

Agenda

• Empirical investigation of efforts to combat online wickedness• Notice and take-down regimes for cleaning

websites• End-user machine infections and ISPs’ response

• Mechanisms to improve cleanup• Reputation metrics to encourage ISP action• Notifications to remove malware from webservers

• Future opportunities and experiments to improve notification-driven voluntary action

Phishing websites

Comparing takedown speed by hosting method (phishing)Hosting method Lifetime hrs

(mean)Lifetime hrs (median)

Free webhosting

brand-owner aware 4 0

brand-owner NOT aware 115 29

overall 48 0

Compromised webservers

brand-owner aware 4 0

brand-owner NOT aware 104 10

overall 49 0

Botnet-hosted 70 33

Fake-escrow websites

Mule-recruitment websites

Comparing takedown speeds by scamScam type Lifetime

hrs (mean)Lifetime hrs (median)

Phishing

Free webhosting 4 0

Compromised webservers

4 0

Botnet-hosted 70 33

Fraudulent websites

Fake-escrow agents 222 25

Mule-recruitment websites

308 188

Takeaways from comparing website takedown efforts• Incentive on the party requesting content removal

matters most• Banks are highly motivated to remove phishing websites• Banks overcome many international jurisdictions and no

clear legal framework to remove phishing pages• Banks' incentives remain imperfect: they only remove

websites directly impersonating their brand, while overlooking mule-recruitment websites

• Lack of data sharing substantially hampers cleanup speed

• Technology chosen by attacker has small impact• Full details: http://lyle.smu.edu/~tylerm/weis08takedown.pdf

Agenda

• Empirical investigation of efforts to combat online wickedness• Notice and take-down regimes for cleaning websites• End-user machine infections and ISPs’ response



Research questions on end-user infections1. To what extent are legitimate ISPs critical

control points for infected machines? 2. To what extent do they perform differently

relative to each other, in terms of the number of infected machines in their networks?

3. How do countries perform compared to each other?

4. Which intermediary incentives work for and against security?

Methodology

• Using different longitudinal data sources of infected machines, each with several hundred million IP addresses• Spam trap data• Dshield IDS data• Conficker sinkhole data

• For each IP address, look up country and ASN• Map ASNs to ISPs (and non-ISPs) in 40 countries (~200

ISPs cover ~90% market share in wider OECD)• Connect data on infected machines with economic data

(e.g., # subscribers of ISP)• Compensate for known measurement issues

Research questions on end-user infections1. To what extent are legitimate ISPs critical

control points for infected machines? 2. To what extent do they perform differently

relative to each other, in terms of the number of infected machines in their networks?



<19>

Percentage of all infected machines worldwide located in top infected ISP networks (2009)

<20>

Percentage of all infected machines worldwide located in top infected ISP networks (2010)

Number and location of infected machines over time (2010, spam data)

22 April 2023 22

Findings (1) – ISPs are control points• Data confirms that ISPs are key intermediaries• Over 80% of infected machines in wider OECD

were located within networks of ISPs• Concentrated pattern: just 50 ISPs control

~50% of all infected machines worldwide• In sum: leading, legitimate ISPs have the bulk of

infected machines in their networks, not ‘rogue’ providers

Research questions

1. To what extent are legitimate ISPs critical control points for infected machines?

2. To what extent do they perform differently relative to each other, in terms of the number of infected machines in their networks?



<24>

Infected machines vs subscribers per ISP (spam)

22 April 2023 25

Findings (2) – ISPs differ significantly • ISPs of similar size vary by as much as two

orders of magnitude in number of infected machines

• Even ISPs of similar size in the same country can differ by one order of magnitude or more

• These differences are quite stable over time and across different data sources

Stability of most infected ISPs over time

30 ISPs are in the top 50 in all four years

Overlap of the 50 ISPs with the highest number of infected machines(2008-2011, spam data)

22 April 2023 26

Stability of most infected ISPs over time

24 ISPs are in the top 50 in all four years

Overlap of the 50 ISPs with the highest number of infected machines per subscriber(2008-2011, spam data)

22 April 2023 27

Most infected ISPs across all datasets

26 ISPs are in the top 50 most infected networks in all three data sources

Overlap of the top 50 ISPs with the highest number of infected machines across datasets (2010, absolute metrics)22 April 2023 28

Research questions





<30>

NL

Infection rates of ISPs per country (spam data)

Research questions





22 April 2023 32

What explains the huge variation in infection rates?• Even good ISPs tackle only a fraction of the bots in

their network• Evidence from recent study of the Dutch market

suggests ISPs contact less than 10% of the customers that are infected at any point in time – this is after Dutch ISPs signed the Anti-Botnet Treaty

• This discrepancy is partially because ISPs do not widely collect data on infected machines in their networks

• This situation is similar or worse in many other countries

<33>

contacting / quarantining ~ 1000 customers (~6%)

contacting / quarantining ~ 900 customers (~5%)

Impact of telco regulation on security• Engagement of ISPs by telecom regulators and

law enforcement improves security• For example, countries where regulators

participate in London Action Plan (LAP) have lower infection ratesExplanatory variable Pooled model Panel model

Coefficient Standard error Coefficient Standard error No. of subscribers -0.00456*** 0.00112 -0.00198** 0.00081 Cable ISP -0.00272** 0.00136 0.00050 0.00114 Cybercrime Conv. -0.00055 0.00204 -0.00083 0.00165 LAP membership -0.00735*** 0.00168 -0.00807*** 0.00142 Piracy rate 0.00041*** 0.00007 0.00030*** 0.00005 Education level -0.05886*** 0.01931 -0.06749*** 0.01967 Constant 0.10528*** 0.02181 0.09869*** 0.02133 N 826 824 Adjusted R2 0.28 n.a. Joint significance F = 50.22*** Wald = 239.49***

Notes: Statistical significance at 1% (***) and 5% (**); n.a.: not available.

Impact of competition on security

Agenda





Reputation metrics as incentives

• Market for security is hampered by information asymmetry between intermediaries and customers

• We often can’t tell which intermediaries are performing better than their peers/competitors

• This weakens the incentives to invest in security• Reliable reputation metrics might change this• Example: poor security ranking of Germany as a

country led to Botfrei

Reputation metrics as incentives

• NL government commissioned TU Delft to develop reputation metrics on botnet infections for the Dutch market, in collaboration with the ISPs

• NL government also asked us to not make the results public, but share them only with the group of ISPs working in the anti-botnet treaty

• Did the metrics have an impact?• Looking at the worst performer in mid 2010

<39>

Infection rates at main Dutch providers, before and after reputation metrics

More information on TU Delft work

“Economics of Malware” (OECD, 2008)http://goo.gl/6HS4d

“Role of ISPs in Botnet Mitigation” (OECD, 2010)http://goo.gl/4UZQF

“ISPs and Botnet Mitigation: A Fact-Finding Study on the Dutch Market (Dutch government, 2011) http://goo.gl/etFZj

Agenda

• Empirical investigation of efforts to combat online wickedness• Notice and take-down regimes for cleaning websites• End-user machine infections and ISPs’ response

• Mechanisms to improve cleanup• Reputation metrics to encourage ISP action• Notifications to remove malware from

webservers• Future opportunities and experiments to improve

notification-driven voluntary action

Voluntary cleanup of webservers distributing malware• Cleanup of hacked websites distributing malware

is coordinated and carried out by volunteers• Security companies• Search engines• Non-profit organizations• Web hosts and site owners

• Malware cleanup process1. Detect a website distributing malware2. Notify the website owner and hosting provider of

infection if compromised, or hosting provider and registrar if purely malicious

3. Search engines might block results until malware is removed

42

Do malware notices work?

• “SBW Best Practices For Badware Reporting”• We designed an experiment to assess the effectiveness of

malware notices in remediating malware• Investigated malware URLs submitted to StopBadware’s

Community Feed 10—12/2011• Randomly assigned URLs to 3 groups

• Control: no report• Minimal report: URL, IP, short description of malware,

date/time detected• Full Report: detailed description of malware (specific bad

code, special information needed to deliver malware)• Follow up 1, 2, 4, 8, 16 days after initial report day

43

Tyler Moore

I recommend turning the first bullet into a slide explaining the best practices, followed by another slide giving an example notice (maybe what is currently in slides 8&9). Then have the content in the rest of this slide explaining our experiment putting these notices to the test.

Example minimal notice

44

Example detailed noticeEverything in the minimal notice plus detailed evidence of infection

45

Tyler Moore

Leave this example notices here even if you also have them earlier on, since you'll want to refer to them after slide 7.

Results for cleanup after 16 days

Experimental Group

% Clean (all)

% Clean (maliciously registered)

% Clean (compromised servers)

Control 45% 46% 45%

Minimal 49% 53% 47%

Full 62% 58% 63%

Tracking cleanup over time

47

Tyler Moore

Probably want a slide to the effect of "Why isn't there a t-test?" Then explain survival analysis, and how we compare the distributions using a different statistical test suitable for survival analysis (get language from the paper).

Takeaways from malware notification experiment• Reporting works

• 40% cleaned up 1 day after receiving full report, vs. 18% w/o notice

• Fuller reports better than concise reports• But only the first report matters• Concise reports a waste of time

• Experimental design could serve as a template for evaluating other notification regimes

• Full details: http://lyle.smu.edu/~tylerm/cset12.pdf

Agenda





How can we further improve cleanup of infected end-user machines? • What we’ve learned so far

• ISPs are crucial intermediary with huge variation in infection rates

• Incentive on requesting party is key• Incident data is a prerequisite for cleanup• Most intermediaries don’t have strong incentive to look

hard for more comprehensive incident data

• Many collaborative data-sharing efforts and notification experiments• Pull vs. push mechanisms for notification• Countries (e.g., US, NL, AU, DE) trying different approaches

Research questions

1. What form of notification is most effective in getting intermediaries to act against abuse?

2. What complementary incentives make key intermediaries more likely to act voluntarily on notification?

• We (SMU and TU Delft) are starting a 3-year US-Dutch funded research project to answer these questions

Research approach

1. Construct taxonomy of incident types, intermediaries, incentives, and notification approaches

2. Perform observational studies that examine the impact notification has on reducing cybercrime levels, starting by quantifying the extent and type of notifications already taking place

3. Run experiments with infrastructure operators that vary the notification approach and cooperation level

• We need your help!

Concluding thoughts

• Sharing incident data is key to cleaning up malware-infected PCs and servers

• Because ISPs control 80% of the problem, they must be part of the solution

• Fortunately, there is great scope for improvement (even in the same market, ISPs of same size performances differ by orders of magnitude)

• We don’t know which interventions work best, so we need evidence-based policies and practices that align with or improve incentives

• For more: http://lyle.smu.edu/~tylerm/

voluntary action against cybercrime

Documents