yankee herd intelligence will reshape anti malware

December 2007

by Andrew Jaquith, Enabling Technologies Enterprise, Compliance, Security and Risk Management Program Manager, [email protected], 617-598-7351

© Copyright 1997-2007. Yankee Group Research, Inc. All rights reserved.

Yankee Group published this content for the sole use of Yankee Group subscribers. It may not be duplicated, reproduced or retransmitted in whole or in part without the express permission of Yankee Group Prudential Tower, 800 Boylston St. 27th Floor, Boston, MA 02199. Phone: (617) 598-7200. Fax: (617) 598-7400. E-mail: [email protected]. All rights reserved. All opinions and estimates herein constitute our judgment as of this date and are subject to change without notice.

Herd Intelligence Will Reshape the Anti-Malware Landscape

The Bottom Line: Financially motivated malware is forcing anti-malware vendors to dramatically change strategies. By 2010, vendors will largely abandon the signature-based technologies that have been the mainstay of the anti-virus industry for 20 years.

Key Concepts: Herd intelligence, whitelisting, detection and response, anti-virus, anti-malware, metrics

Who Should Read: Enterprise CSOs, security vendor CEOs

Practice Leader: Zeus Kerravala, Senior Vice President--Enterprise Research, [email protected], 617-880-7235

Executive Summary

The security of PCs is no longer a tractable problem that can be solved by better engineering, more thorough code reviews, user education or bigger budgets. Instead, it has become an economic problem, with criminals doing anything and everything to ensure that their revenue streams continue to flow.

A flood of financially motivated malware is forcing PC security vendors to dramatically change strategies because their products can’t keep up. By 2010, vendors will place far less importance on the signature-based technologies that have been the mainstay of the anti-virus industry for twenty years. In this report, Yankee Group describes how anti-malware vendors will:

• Strengthen services and tools to detect infections

• Make “herd intelligence” central to their long-term survival strategies

• Create common services for whitelisting known-good programs

• Benchmark customer infections

Vendor action will not be enough to put a dent in the malware problem. The US government (and other national governments) need to:

• Create economic incentives for consumer accountability

• Require parties registering domains to provide proof of identity

• Introduce a safe harbor provision for sharing infection data

• Require carriers to provide “clean pipes” to customers

• Increase cross-border efforts to investigate and prosecute malware authors

2 © Copyright 1997-2007. Yankee Group Research, Inc. All rights reserved.

Table of Contents

I. Introduction ··························································································································································· 2

II. Malware Creators Change Attack Strategies········································································································ 3 Web Becomes Preferred Distribution System for Malware ......................................................................................................... 4 Attack Strategies Favor Overwhelming Force................................................................................................................................. 5 Low-and-Slow Malware Replace Massive Attacks.......................................................................................................................... 6 The Malware Supply Chain Speeds Munitions Creation.............................................................................................................. 7

III. Shifting Attack Landscape Forces Anti-Malware Companies to Adjust······························································· 8 Windows Is Now Under Permanent Siege ....................................................................................................................................... 8 Anti-Malware Products Are Not Keeping Pace .............................................................................................................................. 9 Anti-Virus Companies Must Change with the Times .................................................................................................................... 10

IV. Herd Intelligence Inverts the Laboratory-Based Model ······················································································· 13 Decision-Making Moves to the Cloud.............................................................................................................................................. 14 Metrics and Benchmarking Transform Analysis.............................................................................................................................. 15

V. Conclusions and Recommendations······················································································································ 15 Recommendations for Security Vendors ......................................................................................................................................... 16 Recommendations for US Congress and Regulators .................................................................................................................... 17

VI. Further Reading······················································································································································ 17

I. Introduction As 2007 draws to a close, malware targeting consumers and enterprises has become more deadly and insidious. Financially motivated criminal gangs, seeking to sustain franchises built around identity theft, spam and fraud, have become adept at evading anti-malware vendors’ defenses. The increasingly volatile threat landscape is imperiling customers, straining security vendor resources and disrupting the anti-virus industry.

Desperate times call for desperate measures. For mainstream security software vendors, the time for action has arrived. In the January 2007 Yankee Group Note Anti-Virus is Dead; Long Live Anti-Malware, Yankee Group described a promising emerging technology called herd intelligence, a software-as-a-service approach that allows enterprises to detect previously unknown malware by leveraging collective knowledge gained by customer endpoints.

Yankee Group expects herd intelligence and other cloud-based technologies to rapidly become mainstream. In this report, we describe herd intelligence in detail, and recommend steps vanguard security software vendors should take to embed herd intelligence into their products and services.

December 2007

© Copyright 1997-2007. Yankee Group Research, Inc. All rights reserved. 3

II. Malware Creators Change Attack Strategies Anti-malware software is one of the largest computer security markets. Other than perimeter firewalls, no technology enjoys anti-virus software’s depth of penetration. Nearly every enterprise (99%) has deployed anti-virus software on every desktop.

For twenty years, anti-virus companies have operated using a familiar top-down model (see Exhibit 1).

Exhibit 1. Today’s Top-Down Anti-Malware Protection Model Source: Yankee Group, 2007

PC

PC

PC

DAT

DAT

DAT

PC

PC

PC

DAT

DAT

DAT

PC

PC

PC

DAT

DAT

DAT

Top-Down Model

Malware LabsMalwaresample

received

Creates signature/rule

Distributes update to nodes

Update service

Malware collectors—typically passive honeypots, human volunteers and simulators that mimic user browsing activities—discover and report potentially new strains of malware to vendor labs, who in turn analyze the samples and decide whether they are good or bad. When a sample is determined to be bona fide malware, the vendor generates a signature that enables anti-virus to detect the sample on an infected system. This signature is propagated to deployed anti-virus agents using an update service such as Symantec’s LiveUpdate or McAfee’s ePolicy Orchestrator (EPO). Not every vendor strictly adheres to this model, but the basic approach of analyzing in the lab, then pushing signatures to endpoints has remained remarkably constant because it has served customers well. The laboratory model assumes that four conditions hold:

• All harmful malware can be detected by sensors, simulators, honeypot networks or humans.

• Laboratories can quickly process and decide whether a sample is benign or malign.

• Malware signatures or rules can be efficiently distributed to endpoints.

• Endpoints can efficiently use and process the collective set of signatures (DAT files).


Anti-malware vendors have sought to increase their leverage in each of these areas through increased automation and larger scale detection efforts. To detect a higher percentage of malware in the wild, vendors have deployed large sensor networks that obtain and forward suspicious samples to their labs for analysis. Symantec’s sensor network, for example, consists of more than 40,000 sensors in 180 countries.

Additional technologies attempt to optimize performance in processing malware, distributing signatures and reducing the bloat in signature files. Vendors deploy automated virtualization and simulation engines to classify malware. Labs distribute signature files every hour or even more frequently, rather than once a week as was common just five years ago. Client-side technologies, such as genotyping, heuristic signatures and generic rules, attempt to lighten the load on endpoints.

Although these techniques have helped, collectively these measures cannot cope with the shift in strategy employed by today’s malware makers. Four trends have tilted the playing field toward the malware creators:

• The emergence of the as a malware super-distribution platform

• Attack strategies based on overwhelming force

• Low-and-slow, small-batch malware instead of massive attacks

• The malware supply chain

We describe these trends next.

Web Becomes Preferred Distribution System for Malware The emergence of the malware supply chain spotlights the importance of distribution. Malware creators have found an ideal, exceptionally efficient platform in the internet. More efficient than e-mail and less heavily filtered, the internet offers enterprising criminal minds significant opportunities for mischief. Key characteristics of the internet malware distribution platform include:

• JavaScript-based malware installers: The newest botnet frameworks, available for sale to criminal entrepreneurs, are thoroughly professional. Web sites that host tools such as Neosploit, create and serve up designer malware for web surfers based on the victim’s specific Windows OS and browser version, patch level and installed software. Payloads include key loggers, adware, spyware, password stealers and other delights.

• Web-based botnet control interfaces: As documented by Panda Security, underground software suppliers are making it easier for franchises to recruit and control bots (compromised Windows machines) using tools such as the Zunker control panel. Bearing an eerie resemblance to systems management software, Zunker’s sophisticated web user interface is slicker than most professional ISP control panels and shows a real-time display of compromised Windows machines. Botnet operators can see what programs are running on each bot, and remotely control victims’ instant messaging, e-mail traffic, security settings and more. Zunker can also induce its member bots to generate spam, initiate distributed denial of service attacks or download additional malware as needed. Other popular kits include WebAttacker, MPack and IcePack.

December 2007


• Fast-cycling malware web sites: Criminals have learned to exploit a loophole in the domain name system (DNS) registration process. Because some registrars allow registration with minimal vetting, opportunists abuse registration grace periods. This has allowed criminals to set up malware distribution sites, often masquerading as legitimate businesses, for a few days before vanishing.

• Abuse of carrier “dirty pipes”: Despite the fact that network operators control the ingress and egress points for their customers traffic, most pass on whatever network traffic they receive—good or bad. The lack of default “clean pipes” on the part of carriers gives criminals a free hand. Telefonica’s Juan Miguel Velasco López-Urda says that many carriers, particularly in the United States, play it safe by insisting they are no more than common carriers. López-Urda said, “There is no sense of urgency about this problem at all.” Worse, carriers such as AT&T send mixed messages by announcing deals with the Motion Pictures Association of America (MPAA) to detect file sharing piracy, while allowing malware authors to abuse their address space unfettered. Apparently, only movies and songs are dangerous.

The Trojan horse Gozi, discovered and documented by Don Jackson of SecureWorks, shows all of these vectors in action. Designed by a Russian criminal gang, Gozi infected more than 5,000 home and commercial PCs in a single attack. SecureWorks estimated that the black-market value of the data Gozi gathered from the initial infection of 10,000 accounts exceeded $2 million. As reported by Scott Berinato in CIO, Gozi was distributed widely across the internet via HTML IFRAMEs embedded as advertisements in third-party web sites. It infected victim computers silently via a vulnerability in Microsoft Windows ActiveX technology. Thirty leading anti-virus products did not detect Gozi for more than a month.

Today botnets are franchise operations like McDonald’s or Starbucks, but they are likely more profitable. According to Symantec’s September 2007 Internet Security Threat Report, 5 million bots were active in the first half of 2007. Botnet hunters are throwing up their hands at the futility of trying to take down botnet command and control networks. Gadi Evron, a botnet hunter who works for Beyond Security, recently stated in an email that “they are an oiled machine. We don't hurt them any more... Now it doesn't even hold back the tide.”

Attack Strategies Favor Overwhelming Force During their long history, anti-virus labs have become accustomed to seeing linear growth in malware samples year-over-year. But attackers are no longer playing by the rules. Attackers have employed a strategy of overwhelming anti-virus labs with unique samples, using clever packers that generate binaries that won’t be recognized as being similar to other malware. In 2005, a person (or group of persons) known as Holy Father offered a payload-packing service that, for a fee, was guaranteed to bypass all anti-virus products. Newer tools such as MPack have burnished Holy Father’s techniques to a fine gloss. Today, sellers openly advertise their effectiveness at bypassing signature-based products.

These and other techniques have resulted in a massive increase in the malware variants in circulation. Instead of linear 20% to 30% growth a year, the number of samples has increased geometrically (see Exhibit 2). Between 2002 and 2006, the number of malware variants increased ten times. In 2007, the pace quickened again. Panda Security reports that the total number of samples doubled in the fifteen-month period ending April 2007. They indicate that the total number of malware samples they have received now exceeds three million. Pedro Bustamante, a senior researcher with Panda Security, states that there is no practical upper limit to the number of variants malware authors can churn out. He expects that the total number will exceed 50 million by the end of 2008.


Low-and-Slow Malware Replace Massive Attacks As shown in Exhibit 2, the vertical climb in the total number of malware variants, when plotted on a graph, approaches asymptotic. Conversely, the number of endpoint nodes exposed to any particular malware sample is becoming very small. In contrast to past years, where malware authors crafted Big Bang worms such as Slammer to infect the largest number of hosts, today’s malware variants are so narrowly targeted that they might as well be invisible. The Storm Trojan, a nastier and more successful version of Gozi, exemplifies this trend. According Nicholas Albright of Digital Intelligence and Strategic Operations Group (DISOG), Storm morphs itself every 30 to 60 seconds, which “means that you are unlikely to infect yourself with the same piece of code twice.”

Exhibit 2. Low-and-Slow Malware Replaces Mass-Mailed Worms Source: Yankee Group, 2007

2002

1Million

1:1 Malware

Epidemics

Prevalence (Relative scale)

Percentage of Malware Variants

2004 2006 2008 2010

January 2003: Slammer

infects 75,000hosts in 15

minutes

10X increasein variants

2002-2006

May 2007: 100%of 125 most recent

threats reportedby SYMC and MFE

as "low risk"(low prevalence)

December 2007


Anti-virus vendors’ own lab data shows that malware variants are no longer targeting large populations. In May 2007, every one of the 125 most recent treats detected by Symantec and McAfee were reported as low risk, which generally means “we haven’t seen many of these.” This is yet another way that attackers are gaming the system. Low circulation means low perceived risk, thus low priority in generating signatures.

Low-and-slow malware also means that anti-virus products no longer catch the same things. In a May 15, 2007 blog posting, McAfee researcher Alyssa Myers notes:

“In February 2007 there were 761 viruses reported by two or more vendors and 1211 reported by only one vendor. How does this compare to just a few years ago, before the rise of the bot? In February 2004, there were 269 reported by two or more vendors, and 423 reported by one vendor. These numbers are not exactly cumulative; they’re only what are currently circulating. If something has not been reported for 12 months, it falls off the list.”

The day of the mass virus outbreak is over. Malware authors have replaced massive attacks with low-and-slow malware, manufactured in small batches and designed to infect silently.

The Malware Supply Chain Speeds Munitions Creation Malware creators have become ever more efficient at creating, assembling and distributing malicious software. Industry watchers have long noted that automated tools like MPack serve as force multipliers for attackers. But these are not simply efficiency aids for, “lazy criminals,” as Exploit Prevention Labs’ Roger Thompson puts it. Instead, they signify that the world of malware has stratified into its own supply chain, starting with raw materials and moving all the way through to finished goods distribution (see Exhibit 3).

Exhibit 3. The Malware Supply Chain Source: Yankee Group, 2007

Stage

Raw materials

SubcomponentAssembly

Finished Goods

Distribution

Actor

Vulnerability researcher

Vulnerability researcher, proof-of-concept web site

Script assembler

Botnet operators, organized crime

Product

Published vulnerability

Public posting of POC

Scripted exploit

Mass exploits

Constraints

Time to reverse engineer technical skill

Vendor pressure

Time to write scripts

Time to add to botnet payloads

Vulnerablilities AttacksRawmaterials

Subcomponentassembly

FinishedGoods Distribution


Today, the malware industry is as professional and specialized as any manufacturing operation. It operates with a speed and efficiency that would make Michael Dell gasp with envy. Security researchers who seek to make the world a better place unearth the malware supply chain’s raw materials, software vulnerabilities and theoretical breaks. Virus “information” sites supply information too. The Ukrainian site VX Heavens maintains a library of 66,000 live malware code samples, 98% of which target Windows.

Proof-of-concept code is further assembled into scripted payloads for attack tools such MPack, which according to Symantec, is openly marketed by Russian gangs. Botnet operators are run like franchises and distribute payloads via rogue web sites, peer-to-peer programs or adware/spyware software. After successful infections, carders harvest personal information such as credit card information from victim computers, where the information is bought, packaged and resold on underground bulletin boards.

Although none of these activities are new, the supply chain view portrays the malware world in a way that can be easily analyzed and understood. The manufacturing analogy is intentional. It is a manufacturing business like any other whose workers draw salaries. Unlike real supply chains, the raw materials are often free.

Like any machine well-oiled by money, it will continue to operate as long as it remains commercially profitable. As the Ancheta case of 2006 showed, the business is very profitable. Using a packaged exploit sourced on underground malware chatrooms, Jeanson James Ancheta, a 20-year-old botnet operator with no special technical skills, generated $60,000 for himself in four months by taking more than 400,000 Windows machines. Nearly 80% of his earnings came from kickbacks adware companies paid him for installing adware on victim machines.

III. Shifting Attack Landscape Forces Anti-Malware Companies to Adjust

Windows Is Now Under Permanent Siege Essentially all malware produced by criminal authors targets Microsoft Windows. Its ubiquity on the desktop, plus the lingering legacy of security design decisions that rendered earlier versions extremely vulnerable to attack make it an easy target.

Microsoft employs some of the world’s most talented security designers and engineers. Security receives the highest levels of executive attention within the company. We estimate that the company spends upwards of $150 million annually on security response activities, security re-engineering, design, penetration testing, privacy, education and outreach, and third-party security review activities. No software company does more to promote security engineering excellence than Microsoft. These efforts, initiated nearly five years ago, have markedly decreased the number and kind of threat vectors available to attackers. To Microsoft’s credit, the most serious threat vectors have been closed: Windows XP and more recently, Windows Vista. The Microsoft Security Intelligence Report (January-June 2007) states that Vista PCs have seen a 60% reduction in malware compared to XP.

Unfortunately for Microsoft, the company’s best efforts do not matter much. The production rate of the malware supply chain is accelerating, resulting in a surge of unique variants of viruses, worms and rootkits. Fusing the means of malware production to the most efficient software distribution system ever devised—the internet—means that more users are at risk than ever before. The increased professionalism of the malware supply chain also guarantees that opposition security analysts, working on salary for criminal enterprises in the Eastern bloc, will be able to exploit design weaknesses (ActiveX, networking features, browser helper objects) and implementation flaws (buffer overflows) for as long as Windows exists in its current form.

December 2007


Evidence of attacker intransigence abounds. In 2006, internet stock trader E-Trade marked down $31 million in losses, due to covering the costs of fraudulent trades made by customers whose identities had been stolen. E-Trade’s 10K explicitly named malware, such as rootkits and keyloggers, as the root cause for these problems. TD Ameritrade wrote down $4 million in losses in 2006 and some fraction—perhaps more than half—of $39 million in 2007. These companies were not alone. Behind closed doors, every financial institution Yankee Group speaks with cites identity theft, and the malware that enables it, as a chronic and increasingly serious problem.

Recent academic research shows that compromised PCs are a now commodity that can be rented, sold and traded. In the Carnegie-Mellon University/Berkeley/UC San Diego paper “An Inquiry into the Nature and Causes of the Wealth of Internet Miscreants,” the most common items for sale in an analyzed underground information market were bank logins, PayPal account credentials, e-mail proxies, hacked hosts and root passwords. The authors estimated that the stolen wealth represented by this particular market during a seven-month period exceeded $93 million. The cost to buy a compromised host ranged between $5 and $15.

Earlier this year, in the April 2007 Note, Yankee Group's Anywhere Threats Research Protects Web-Based Businesses, we predicted that malware will evolve to be both web-based and platform agnostic. In particular, Yankee Group expects that platform-agnostic malware will affect exclusively web-based applications (see the October 2007 Yankee Group Report The Web 2.0 Security Train Wreck). However, operating systems are not going away, and users will continue to keep significant tangible and valuable assets on their computers. The presence of cached web pages, stored credit card numbers and passwords, e-mail addresses and other plunderable booty ensures that attacks will continue to yield results. We expect that the threat landscape facing computer operating systems, Windows in particular, will get much worse before it gets better.

The security of Windows is no longer a tractable problem that can be solved by better engineering, more thorough code reviews, user education or bigger budgets. Instead, it has become an economic problem, one in which criminals will do anything and everything to ensure that their revenue streams continue to flow. They will employ tactics such as polymorphic code, vulnerability exploits, infiltration, deception and good old-fashioned trickery to make sure that corporate and consumer users continue to be infected.

Anti-Malware Products Are Not Keeping Pace During the past 20 years, security vendors have built strong and enduring franchises protecting their customers from malware. No other security product enjoys these pervasive penetration rates. The Yankee Group 2005 Security Leaders and Laggards Survey revealed that nearly 100% of corporate customers have deployed anti-virus products and the broader security suites they are bundled with.

Anti-malware products are decreasingly effective. Public comparative studies, such as Andreas Clementi’s AV-Comparatives.org, suggest that anti-virus products are poor at detecting previously unseen malware, with detection rates ranging from the low single digits to about 70%. These tests are imperfect because they don’t test all of an AV product’s runtime and behavior blocking protections. However, benchmarking studies Yankee Group has seen from other vendors largely confirm Clementi’s findings.


Anti-virus vendors protest that retrospective comparatives like Clementi’s are unfair because they don’t show how accurate detection scores might be if virus signatures were up-to-date. But the window between malware creation and downstream signature distribution is exactly the weakness malware authors are targeting: Companies privately acknowledge the difficulty in keeping up with malware authors. In closed-door sessions with Yankee Group, vendors have told us that:

• “The bad guys are essentially mounting a denial of service (DoS) against our labs by flooding us with malware samples.” —Vendor #1

• “The only way we’ve been able to keep up is by throwing bodies at the problem, and it just about killed us.” —Vendor #2

• “Last year, we saw more malware samples than we had in the previous ten years.” —Vendor #3

The strategy of overwhelming labs with new samples ensures that the signatures can’t possibly stay up-to-date. This strategy has been effective in ensuring a continuing stream of new infections. Anti-malware vendor Panda Security reports that 23% of consumer clients it scanned with its NanoScan on-demand scanner were infected — even when running completely up-to-date signature files, including its own. The news was slightly better for corporate clients, which had half the infection rate. Nonetheless, Panda’s figures show that for networks of more than 100 clients, the chance of having at least one infection jumps to 70%. This suggests the presence of malware in medium and large enterprises, regardless of how well-run its AV defenses are run, is a near-certainty.

Panda’s report, and similar ones we’ve been briefed on from vendors such as Sana Security and Prevx, raises difficult questions for incumbent anti-malware companies. They are forcing the clubby anti-virus industry to acknowledge that their products are becoming less and less effective.

Anti-Virus Companies Must Change with the Times The anti-malware industry needs to change, and not just because of the need to get ahead of the criminal gangs that pump out huge volumes of malware. Most of the leading security vendors are large enough that they have become attractive targets for class-action lawsuits. A product called Total Protection, for example, ought to detect more than 24% of the unknown threats it encounters in tests like Andreas Clementi’s.

Botnet hunter Gadi Evron puts matters succinctly: “Things are in fact FUBAR (fouled up beyond all recognition). We need new ideas and new solutions [because] honestly, although we want to feel we make a difference by taking care of this or that malware, or this [or] that command and control network we are powerless, and have not made a real difference in the past 6 years… while things got worse.”

Making a dent in the chronic and relentless scourge of malware will require vendors to make two kinds of changes: structural and technical.

December 2007


Structural Industry Changes

The size of the malware problem is bigger than any one company can solve on its own. To date, cooperative efforts between security vendors include regular industry conventions, ad hoc malware sample sharing, mailing lists, informal communications and loose attempts at assigning common names to malware families. But the industry can, and should, do more. It needs:

• Common whitelisting services for known binaries. Operating system and software vendors such as Microsoft, Adobe and Oracle should create a network service for registering and verifying the status of binary files such as DLLs, executables and drivers. Anti-malware products could query the database—in effect, a global whitelist—to determine whether particular binaries are from a particular vendor. Models for this already exist. For example, the Sun Microsystems’ Solaris Fingerprint Database provides a common facility for administrators to verify whether Solaris binaries are from Sun. The service has existed for more than seven years. It’s a model worth imitating.

• Common metrics. The anti-virus industry is the only 20-year-old market about which no information about field successes and failures exist. Outside of the laboratory, it is literally impossible to determine how often customers are infected. Vendors either don’t know, or they won’t talk. Instead, the industry should take a leaf from the Center for Disease Control and Prevention’s (CDC) playbook, and start pooling information on customer infections. Common language and common measurement techniques will result in better analysis and drive new product innovations. Perhaps most important, it will also give vendors genuinely interesting data to talk about, rather than stale laboratory test results. Pointless laboratory tests such as the WildList and Virus Bulletin’s VB100 are at least three orders of magnitude off.

• Common sense. Vendors must shift out of the mindset that all malware is preventable. When prevention fails, detection and recovery matter more. Anti-virus vendors, stuck in silver-bullet marketing mindsets, need to de-emphasize the front phase of security (prevention) and start focusing on the back two phases (detection and recovery).

Technology Improvements

Beyond structural industry remedies, anti-malware companies need to introduce new technologies for blocking, detecting and processing malware. The need for technical innovation is great. We expect that the number of malware samples will continue to grow geometrically for the foreseeable future. Realistically, anti-virus companies who wish to survive must figure out how to cope with volumes that will be at least two orders of magnitude more than today. Laboratory staffs will need to be 100 times larger, unless radical action is taken.

Technical innovations that will help anti-malware vendors cope better include (see Exhibit 4):

• Sandboxed runtime analysis: Most anti-virus vendors have added sandboxes—pseudo-virtual PCs—to detect malware. Vendors, such as Kaspersky Lab and Norman, branded them as part of their marketing strategies. The sandbox intercepts inbound binaries, executes them in a simulated Windows environment and looks for tell-tale signs of bad behavior. Sandboxed runtime analysis is well-suited to processing high volumes of samples in a laboratory setting or gateway server, although it can work on client machines too.

• Network-based infection detectors: When prevention of malware fails, detection of its post-infection behavior becomes paramount. Vendors such as TrendMicro and Simplicita (Sandvine) have introduced specialized botnet detection packages that look for anomalous network traffic patterns that botnet nodes typically exhibit, such as phoning home to known-bad IP address ranges, abusing DNS or joining an external IRC channels. Network specialists such as Arbor Networks and managed service providers such as Symantec are providing services that do this today.


• Herd intelligence: Vendors such as Panda Security, Prevx, ESET and Sana Security have introduced various methods for turning their deployed endpoints into malware collectors. When an unknown binary attempts to execute, the client-side agent sends detailed telemetry information to a remote centralized server and asks whether it is good, bad or unknown. The server makes a disposition decision (run, block, warn) based on the collective history accumulated by the herd. By pooling information about all executing programs across its installed base, the herd makes smarter decisions and can confer immunity faster to new variants.

Each of these three methods will help analyze ever-larger sets of malware. However, none are without problems. Sandboxing can be defeated by anti-VM detection code, such as the sample anti-Norman kit distributed by 7A69 Malware Labs. Likewise, network detection works only when malware communicates in well-known patterns—and these shift regularly.

Herd intelligence, for its part, generally requires a network connection to the “mother ship” in order to check the status of unknown binaries; this may not always be possible (or desirable).

As Yankee Group described in its January 2007 Note, Anti-Malware is Dead; Long Live Anti-Malware, herd intelligence offers the best chance for success. The reason why: scale. By turning every endpoint into a malware collector, the herd network effectively turns into a giant honeynet that can see more than existing monitoring networks. For example, were herd intelligence features deployed in scale across Symantec’s entire deployed base, the number of sensors would dwarf its own 40,000-node Global Threat Network by three orders of magnitude.

We describe the promise of herd intelligence next.

Exhibit 4. Technical Improvements Coming to Anti-Malware Software Source: Yankee Group, 2007

Technique

Sandboxing. Intercepts inbound binaries, executes them in a virtual PC environment and looks for tell-tale signs of bad behavior.

Herd intelligence.Turns every endpoint into a malware collector

Infection detectors. Response technology that alerts administrators when unknown malware has evaded preventative defenses.

Advantages

• Processes highvolumes of samples in a laboratory setting

• Can also work on client machines

• Internet-scale detection of unknown malware

• Very fast immunization against new malware variants

• Provides fallback net for cases when anti-malware software fails

• Detects wider array of conditions

• Vendor independent

Disadvantages

• Defeated by anti-VM detection code in some malware

• Excessive memory consumption on client PCs

• Requires endpoints to “phone home” to the cloud periodically

• Higher false positives than signature-based methods

• Bitter pill for anti-malware vendors to swallow

• Potential false positives

• Investigations can be time-intensive

Examples

ISS Proventia G200 applianceKaspersky Anti-VirusNorman Anti-Virus

ESET ThreatSensePanda Security Malware Radar and NanoScanPrevx 2.0/CSISana Security Primary ResponseNorton Anti-Virus “Community Watch”Microsoft Windows Defender SpyNetRelated whitelisting vendors: Lumension (SecureWave), Bit9

Trend Micro InterCloud Security ServicePanda Security Malware Radar and NanoScanSandvine (Simplicita) ZBXSupport Intelligence REACTMandiant First Response

December 2007


IV. Herd Intelligence Inverts the Laboratory-Based Model Herd intelligence’s primary advantage over other malware detection methods is scale. Scale enables the herd to counter malware authors’ strategy of spraying huge volumes of unique malware samples with, in essence, an internet-size sensor network.

Herd intelligence works inverts the anti-virus industry’s laboratory-based model for protecting clients against malware. In today’s laboratory-centered model (see Exhibit 5, left side), the vendor receives a new sample from other vendors, its sensor network, honeynets or individual volunteers. The sample may or may not be provided with information about the context the code was executing in or where it was downloaded from. The vendor’s laboratory analyzes the binary using automated methods if possible. If not, it is queued for manual analysis. Because of the huge number of samples in the analysis queue at any one time, the vendor prioritizes its work based on the number of times each sample had been detected previously.

If the laboratory determines that a particular sample is malicious, the vendor gives it a signature that uniquely identifies it. Once generated, the vendor’s update servers distribute the updated signature file to all subscribing clients during their next update cycle, typically within a few hours.

Exhibit 5. Herd Intelligence Reinvents Anti-Malware Analysis Source: Yankee Group, 2007

Bottom-Up Model

Reportsprogramand itscontext

PC

PC

PC

DAT

DAT

DAT

PC

PC

PC

DAT

DAT

DAT

PC

PC

PC

DAT

DAT

DAT

Malware Labs

Informs other nodes

Cloud

Exceptions

Unknownprogram

seen

PC

PC

PC

DAT

DAT

DAT

PC

PC

PC

DAT

DAT

DAT

PC

PC

PC

DAT

DAT

DAT

Top-Down Model

Malware LabsMalwaresample

received

Creates signature/rule

Distributes update to nodes

Update service

Today's Laboratory Model Herd Intelligence Model


Problems with the top-down laboratory model are manifold. The model assumes that all malware can be caught, samples can be quickly processed and signature updates can be rapidly distributed. But none of these assumptions are as firm as they used to be. Unique samples slip past sensor networks, rely on small batch sizes to fly under the radar and have short lifetimes. The net result is that the lag between outbreak and immunity is growing, not shrinking. Vendors are increasingly chasing yesterday's threats rather than today’s.

Herd intelligence, by contrast, immunizes endpoints faster by leveraging intelligence in the cloud (see Exhibit 5, right side). It is a bottom-up model. In the herd model, an endpoint wishing to execute an unknown binary uses behavioral heuristics to determine whether the program is good or bad. If it cannot divine its intent, it asks the cloud network for a determination about the program. Unknown binaries are submitted to the cloud, along with detailed telemetry information about the program’s execution context. These data include details such as the location the code was downloaded from, the DLLs and APIs it requires, what process spawned it, and additional run-time details.

Decision-Making Moves to the Cloud When making its decisions about an unknown program, the cloud-based model possesses advantages over the laboratory model, notably real-time detection, more detailed contextual information, centralized whitelists and blacklists, and internet-scale sensor reach. These advantages make it easier to make determinations. When a determination cannot be made, the sample is queued in the laboratory as a last resort.

The malware determination process, replicated across millions of endpoints, produces an emergent collective defense system. It confers immunity to the entire herd mere minutes after new samples are detected. Near-zero determination latency, in essence, neutralizes the low-and-slow strategy.

Before herd intelligence goes mainstream, certain technical and business-related challenges need to be worked out. These include:

• Avoidance of false positives: When nearly all samples are processed automatically, reason dictates that some good programs will be erroneously classified. McAfee’s Joe Telafici cautions, “All behavioral techniques to date require more vigilance on the part of the administrator and the researcher to prevent false positives.”

• Network requirements: Although client-side caching can alleviate the need for excessive connections to the cloud, the herd intelligence model derives its power from its ability to leverage collective knowledge over the network. Not all enterprise customers will allow their endpoint PCs to send detailed telemetric information outside the firewall. Anti-malware vendors will need to educate customers and provide more palatable alternatives, such as caching proxies that companies can deploy themselves.

• Data glut: Telemetric data provided by herd endpoints will be substantial. Anti-malware vendors will need to spend significant millions of dollars of capital to create scalable infrastructures to collect, process and store data furnished by endpoints.

• Vendor software updates issues: Cloud-based whitelists need to be rapidly informed of patches and updates to legitimate, good software. Operating system vendors (particularly Microsoft), ISVs and security vendors will need to keep each other informed about software updates and patches.

These concerns are not trivial. Moreover, not every anti-virus vendor views herd intelligence as a panacea. McAfee’s Telafici notes that the offensive and defensive strategies are constantly moving: “I view behavioral/herd approaches as providing at best another 3 to 5 years of breathing room.”

December 2007


Metrics and Benchmarking Transform Analysis Herd intelligence will do much to tilt the playing field back away from the attackers. But even an omniscient, massively distributed, lightning-quick herd intelligence network won’t catch all malware. That’s why infection-detection technologies, layered on top of preventative technologies such as anti-virus, will become increasingly important parts of enterprise security defenses.

Today, mainstream anti-malware companies rarely speak candidly about malware infestation rates for their enterprise customers. The bad news has historically been delivered by smaller, firms with something to prove. Support Intelligence outed Fortune 1000 companies infected by malware, among them Best Buy, American Electric Power, Dow Jones, 3M and AIG. Likewise, Mandiant’s Kevin Mandia tells Yankee Group that in 40 malware infection incidents his firm investigated recently, only 25% of these were caught by anti-virus, the rest, by humans who noticed something strange on their PCs.

Instead of pretending their products are invulnerable, anti-malware companies need to come clean and start talking about the data they’ve gathered about actual infections. Symantec has come close in some of its public statements. In a Dark Reading article, MSS VP Grant Geyer said that as many as 30 of its 81 managed services customers experiences at least one botnet-related incident per day. It’s a good start, but not enough. Anti-malware vendors should start pooling and comparing data, about:

• Customer infection rates, in detail, for particular infection vectors or malware families, such as the WMF exploit or Storm bot

• In-the-field detection rates for particular product technologies, namely signature-based, heuristic and behavior blockers

• Estimated customer workload for reacting to and eliminating infections

• Malware laboratory efficiency, such as staff sizes, processing queue lengths and percentages of malware classified automatically

Metrics for these and other categories will transform the market. These will enable disparate enterprises to benchmark their prevention and detection activities, determine whether they are better off or worse than similar enterprises in their peer group and compare vendor performance. As with any benchmarking activity, vendors must provide the right kinds of assurances and incentives to get enterprises to share telemetry and infection data. Fortunately, these issues are readily addressed (for recommendations, see the August 2006 Yankee Group Report Security Benchmarking Increases Customer Traction).

Yankee Group believes that anti-malware vendors will benefit from more transparency about their laboratory operations and product effectiveness. Customers will appreciate the honesty, and use benchmarks to justify spending decisions. Moreover, realistic positioning about the limitations of preventative products will also lay the groundwork for future purchases of enhanced detection technologies.

V. Conclusions and Recommendations Today’s anti-malware defenses are no longer fully shielding customers from the stepped-up attacks of malware creators. Attackers motivated by the purest and most addictive incentive—money—are attempting to overwhelm anti-malware companies with vaster numbers of novel malware variants. The entrenchment of the malware supply chain means that client operating systems, particularly Windows, are now under permanent siege.


In the new threat environment, top-down, lab-based models have become the weakest link in the chain. Yankee Group expects that anti-malware companies will be forced to develop new approaches to combat malware. These will include common, industry-wide services for whitelisting legitimate programs and infection metrics. Individual vendors will increase their ability to cope by implementing herd intelligence networks, virtualization and simulation techniques.

The malware siege means that vendor claims of bulletproof prevention are increasingly held aloft by nothing more than hot air. As the sky-high expectations of anti-malware vendors plummet back to earth, preventative technologies will necessarily be de-emphasized. Tools that help consumers and enterprises detect and react to malware infections will become more commonplace. With luck and a little bit of regulatory help, enterprises might even begin sharing operational data about their successes and failures.

Recommendations for Security Vendors Security vendors should move to radically expand their herd intelligence and infection detection capabilities. Yankee Group recommends that anti-malware vendors:

• Make herd intelligence central to their long-term survival strategies. To date, Panda Security is the only large security vendor that has embraced the herd intelligence message. Herd features have not yet been fully integrated into their primary endpoint client. However, the related Malware Radar and NanoScan products, which do have herd intelligence features, have allowed Panda to expand the number of malware samples they collect to 15,000 a day—ten times what Symantec collects. Anti-virus companies that are not taking steps today to plan for malware volumes 100 times their current load, are not thinking hard enough about the problem.

• Strengthen services and tools to detect bot infections. With the exception of Trend Micro, which already offers an enterprise botnet detection tool, the largest security vendors (Symantec, McAfee, CA), have treated corporate botnet infections as laboratory curiosities that deserve mention in quarterly threat reports rather than potential market opportunities. Meanwhile, Trend Micro and specialists such as Support Intelligence and Sandvine (Simplicita) have had the detection market to themselves.

• Create a common whitelisting service. The profusion of malware binaries suggests that it may soon be easier to whitelist programs than to blacklist them. Microsoft, security vendors and leading ISVs should create a central “fingerprint” database that allows legitimate parties to register current and updated versions of executables, binaries and DLLs. Sponsorship by Microsoft is essential, but control and funding should be jointly determined.

• Symantec should buy Sana Security. In March 2007, Symantec announced an OEM deal to offer Sana’s Primary Response SafeConnect behavior blocking software and herd intelligence software to consumers as Norton AntiBot. Symantec’s Rowan Trollope justified the deal by saying, “We realized that the botnet phenomenon had reached pandemic proportions.” The Sana deal comes on top of a combined $108 million spent in 2004 to buy Platform Logic and Whole Security, which gave them the precursors for a herd intelligence capability, notable the SONAR feature. It’s time for them to reach deep into their wallets once more, and buy Sana, whose technology will help them finish the job.

• Microsoft should beef up SpyNet. Launched to great fanfare at the RSA Conference in February 2006, SpyNet provides herd-like features but hasn’t been positioned by Microsoft as such. It has completely escaped mention in the most recent Security Intelligence Report. Microsoft should relaunch SpyNet and exploit it to its fullest potential as an internet-scale malware collection service.

December 2007


Recommendations for US Congress and Regulators

• Create economic incentives for consumer accountability. Consumers don’t have any incentive to protect their PCs from compromise, because most banks and credit card issuers promise to reimburse losses caused by identity theft. Compare this to the EU, for example, where stricter limits on bank liability have caused card issuers to roll out technologies such as EMV (chip and pin) cards. Realigned incentives would cause financial institutions to compete on the basis of having best security protections, rather than the best money-back guarantees to absorb losses due to users’ own neglect for their own security.

• Require parties registering domains to provide proof of identity. Today, any miscreant with a stolen credit card can register a domain and start polluting the internet. Legislation that mandates domain registrars to obtain proof of identity at time of registration would enable law enforcement to track down miscreants more easily. At the same time, registrants wishing not to disclose their identities to the public at large should be able to obtain the DNS equivalent of an unlisted number.

• Introduce a safe-harbor provision for sharing infection data. Security vendors and enterprises claim that their lawyers are preventing them from pooling data about malware infections. It’s time to deprive them of that excuse. A safe-harbor provision would shield vendors and enterprises from lawsuits when they share infection and telemetry data and facilitate more analysis of the malware problem.

• Require carriers to provide “clean pipes” to customers. Network operators shouldn’t charge extra for keeping obviously dangerous traffic off of the internet. The model should be inverted. Basic service should route, and lightly filter, only a small subset of internet port traffic, such as HTTP, FTP, SSH, SMTP, DNS and a few other protocols. Polluters should have to pay for the privilege of dirtying pipes with other traffic.

• Increase cross-border efforts to investigate and prosecute malware authors. Malware authors exploit the distributed, international nature of the internet as a way of evading prosecution. The United States should work with the G10, China, Russia and the United Nations to harmonize cyber-crime laws, coordinate investigations, share evidence and speed extraditions. Recent cross-border collaborations, such as the FBI’s Bot Roast II exercise, suggest the art of the possible.

VI. Further Reading

Yankee Group Link Research

Anti-Virus is Dead; Long Live Anti-Malware, Note, January 2007

Yankee Group’s Anywhere Threats Research Protects Web-Based Businesses, Note, April 2007

Cautious Optimism Prevails as Operators Look to Deep Packet Inspection for Enhanced Service Control, Report, May 2007

Security Benchmarking Increases Customer Traction, Report, April 2006

The Web 2.0 Security Train Wreck, Report, October 2007

yankee herd intelligence will reshape anti malware

Technology

malware vendors

motivated malware

malware landscapethe

technologies enterprise

herd intelligence

enterprise csos

signaturebased technologies

zeus kerrava