1

Security in Distributed Systems

EECS 591 - Distributed Systems University of MichiganThursday April 10th, 2003

Copyright 2002

2

Hey, your not Farnam …

Michael BaileyDirector of Engineering

Arbor Networksmike@arbor.net

2

3

Agenda

Security in Distributed SystemsExamples of current threats

DDoSWorms

Examples of current mechanismsFirewallsIDSVPNs

4

Readings and Bibliography William R. Cheswick, Steven M. Bellovin, and Aviel D. Rubin, “Firewalls and Internet Security: Repelling the Wily Hacker”, Addison-Wesley, Boston, MA, 2003, ISBN 0-201-63466-XAndrew S. Tanenbaum, and Maarten van Steen, “Distributed Systems Principles and Paradigms”, Prentice Hall, Upper Saddle River, NJ, 2002, ISBN 0-13-088893-1Bruce Schneier, “Secrets & Lies: Digital Security in a Networked World”, John Wiley & Sons, New York, 2000, ISBN 0-471-25311-1Props out to Paul Francis and Avi Rubin for several pages on content

3

Security in Distributed Systems

Copyright 2002

6

Security

“There is no such thing as absolute security” - Cheswick Security is all about managing risk.How much effort are you willing to go through to protect what from whom?

4

7

How do we think about security?

Goals +Adversaries +

Threats + Economics= Policies

These are separate from the mechanismsused to enforce the policy or the implementation of these mechanisms

8

Goals

ConfidentialityPrivacyAnonymity

IntegrityNon-repudiation

DependabilityAvailabilityReliabilitySafetyMaintainability

.. and loyal and trustworthy and brave and …

5

9

Adversaries

Lone CriminalsMalicious InsidersIndustrial EspionageOrganized crimeTerroristsPoliceNational Intelligence agencies

I am a L33t H4x0r D00d!

10

Threats

InterceptionInterruptionModificationFabrication

6

11

PoliciesThe Network Security Policy identifies the threats against which protection is required, and defines the required level of protection.

Least PrivilegeDefense In DepthChoke PointWeakest LinkFail Safe Stance etc.

Example :Strategy 1 : Everything is forbidden unless explicitly permitted.Strategy 2 : Everything is permitted unless explicitly forbidden.(11)

http://www.darmstadt.gmd.de/ice-tel/

12

Policy Questions

What resources are we trying to protect ? Which people do we need to protect the resources from ? How likely are the threats ? How important is the resource ? What measures can be implemented to protect the resource ? How cost effectively and in what time frame can these be implemented ? Who authorizes users ?

7

13

Security Mechanisms

EncryptionAuthenticationAuthorizationAuditing

14

Mechanisms and Implementation

Schneier encourages us to think of security needs as a system

ComplexBug-riddenEmergentInteractive

“A chain is only as strong as its weakest link” – CheswickMay not have to go through a specific mechanism, can go around it.

8

Denial of Service Attacks In Detail

Copyright 2002

16

Introduction

What is a Denial of Service attack?An attempt to consume finite resources, exploit weaknesses in software design or implementation, or exploit lack of infrastructure capacityEffects the availability and utility of computing and network resourcesCan be distributed for even more significant effect

9

17

These threats are hard and getting harder

The number of open and exploitable security vulnerabilities continues to rise.High bandwidth connectivity for individuals is now commonplace.Automated attack tools and techniques are openly available and require no technical sophistication.Security not yet understood as an operational cost of doing business.Very difficult to deploy effective preventive controls.

18

DoS History

Locally-induced crashexploit operating system or server software bug

Local resource consumptionfork() bomb, fill disks, deep directory nesting

Deny service to individual hostsforce crash or outage of critical services

Remotely-induced crash“magic” packets – ping of death, teardrop

Remote resource consumptionsyslog, SYN, fragment flood, UDP storm

10

19

DoS History (cont.)

Deny service to an entire networktarget vulnerable links or critical network infrastructure / information

Remotely-induced network outageattacks against routers, DNS serversredirected routes – forged routing information

Remote network congestionforged directed broadcasts – smurf, fraggleremote control of compromised hosts (“zombies”) for coordinated flooding - DDoS

20

DoS Present

Distributed attacksRemote control zombie armiesIP reflection/refraction

Obfuscated network audit trailForged/”spoofed” IP source addressesPulsing (on/off) attacksDecoys

Obfuscated attack signatureMimicking legitimate traffic (e.g. TCP ACK flood)Mask with legitimate trafficSignature based IDS evasion techniques (e.g. fragroute: chaffing, delays, duplicates, ordering).

11

21

DoS Futures

Network-based flood attacksvulnerable software is being patched

Subnet spoofingingress / egress filtering becoming more popular

Infrastructure attackstargeting upstream routers and links

Hit-and-runpulsing / short-lived floods

Internet-scalewidely-distributed, large-scale zombie “armies”

22

DoS Futures

Obfuscation of network audit trailredirection features of certain application protocols – recursive DNS queries, gnutella, etc.

Mutation of attack signaturesaddress, protocol, port randomizationzombie “robo-surfing”

Routing infrastructure attacksBGP route hijacking

Automated conscription of zombie armiesrecent Internet worms and virusesMicrosoft Outlook, IE, IIS, SMB

12

23

Timeline of a DDoS attack

A large set of machines are compromised

Attacker identifies exploitable hosts with scanners, or other techniquesAttacker accesses the system with automated remote exploits, sniffers, password cracking, worms, trojansAttacker installs attack tools

Attacker remotely instructs compromised machines to attack target

24

Example: Smurf AttackReflector Network

SRC DST3.3.3.100 2.2.2.255

1.1.1.100

SRC DST2.2.2.* 3.3.3.100

ICMP Echo Request

3.3.3.100

2.2.2.*

ICMP Echo Replies

Attacker Target

13

25

Example: TCP SYN Flood

SYN 141:141

SYN 182:182

ACK 142

ACK 183ClientServer

CLOSED CLOSED

SYN_SENT

ESTABLISHEDESTABLISHED

SYN_RCVD

Normal sequence for TCP connection establishment (3-way handshake)

26

Example: TCP SYN Flood (cont.)

ServerAttackerSYN 141:141

SYN 182:182ACK 142

SYN 241:241SYN 341:341SYN 441:441SYN 541:541SYN 641:641SYN 741:741

SYN 282:282ACK 242

SYN 382:382ACK 342

SYN_RCVD

SYN_RCVD

SYN_RCVDSYN_RCVD

SYN_RCVDSYN_RCVD

SYN_RCVD

SYN_RCVD

Listen Queue

SYN_RCVD

14

27

Preventive and Corrective Controls

Ingress / Egress filtering ( anti-spoofing )Rate limitingStatefull defenses ( e.g. tcp intercept )Patch vulnerable hosts and servicesProvisioning and capacity planningPacket filtering on provider side of WAN links

28

DoS Remediation

DetectionDetermine attack methodology and what resources are affected

TracebackDetermine the source and transit path

FilteringDetermine what traffic to block, and where best to block it

15

29

Mitigation Strategies

Unicast Reverse Path Forwarding (uRPF)Strict vs. loose uPRFPrevention of address spoofingShunning with uPRF and BGP on all border routers

CAR Rate limit attack traffic: ICMP, UDP, TCP SYNBe aware of unintended consequences!QoS Policy Propagation with BGP (special community)

ACLFilter traffic targeted at a destinationOff-ramping for filtering and forensics

30

More on DoS

Check out David Dittrich’s Sitehttp://staff.washington.edu/dittrich/misc/ddos.html

Read Steve Gibson’s http://grc.com/dos/drdos.htm

16

Worms

Copyright 2002

32

Worms and Viruses

Self propagating exploits are called worms.Virus are exploits that attach themselves to other programs.Tend to be quick movingTend to be massive in effectTend to be hard to clean up

17

33

Virus Damage Scenarios

BlackmailDenial of service as long as virus runsPermanently damage hardwareTarget a competitor's computer

do harmespionage

Intra-corporate dirty trickssabotage another corporate officer's files

34

How Viruses Work

Virus written in assembly languageInserted into another program

use tool called a “dropper”Virus dormant until program executed

then infects other programseventually executes its “payload”

18

35

How Viruses Spread

Virus placed where likely to be copiedWhen copied

infects programs on hard drive, floppymay try to spread over LAN

Attach to innocent looking emailwhen it runs, use mailing list to replicate

36

Antivirus and Anti-Antivirus Techniques

Integrity checkersBehavioral checkersVirus avoidance

good OSinstall only shrink-wrapped softwareuse antivirus softwaredo not click on attachments to emailfrequent backups

Recovery from virus attackhalt computer, reboot from safe disk, run antivirus

19

37

The Sapphire Worm

The Sapphire WormAt approximately 12:30 am EST on January 25, the Sapphire worm infected more than 120,000 computers, overwhelming many corporate and service provider networks.

38

The threat is HUGE

“This worm required roughly 10 minutes to spread worldwide. In the early stages the worm was doubling in size every 8.5 seconds. At its peak, achieved approximately 3 minutes after it was released, Sapphire scanned the net at over 55 million IP addresses per second. It infected at least 75,000 victims and probably considerably more.”

- Moore, Paxson, et. El. For details see: http://www.caida.org/analysis/security/sapphire/

20

39

Massive Effect

In not only bandwidth, but also routing infrastructure

40

Its NOT going away

Every new security hole is now a wormThe doomsday threshold is much smaller than anyone thought

All you need is a vulnerability that has target population of 70k hosts You’ll have near total penetration in less then ten minutes

There are lots of these that meet the threshold every year

Next timeits going to be an important service that’s hard to filterand the payload will not be benign

21

41

Other Recent Worms

sadmind/IIS Solaris rpc.sadmind (2 years old)Microsoft IIS Unicode directory traversal (7 months old)

CodeRed Microsoft IIS .ida buffer overflow (1 month old)

CodeRedII Microsoft IIS .ida buffer overflow (1 month old)

Nimda Microsoft Outlook, IE, IIS, file sharing, CodeRedII backdoor

42

Internet Worms and Viruses

Rise of Internet worms and viruses such as CodeRed and NimdaDevastating impact on enterprise networks with enormous clean up costDDoS payload; compromised hosts potentially serving as zombies

Nimda Instantaneous Firepower

Ethernet

DS3

T1/Cable

DialupISDN

DSL

26%

29%

33%

CodeRed Infected Demographics

Korea

.com

.eduGermany

ItalyBrazil

SpainNetherlands

China France

.net

11%

16%

49%

6%

6%

22

43

Internet Worm Monitoring

Nimda:5 billion infection attempts per day across the Internet Easier to contain, due to its "island-hopping" strategy

CodeRed :At least 40 billion hits each month - and growing Won't go away …the new Internet locust?

44

Summary

The Good NewsCodeRedII (and its variants) are dead

The Bad NewsCodeRed and Nimda are here to stayWidespread scanning for open servers11 Israeli hosts scanning 200-1000 hosts daily in DecemberUsing Active-X-based scanner, based on CSHttpClient User-AgentNew worms will be even betterExpect major DDoS attacks in the near future

23

Firewalls

Copyright 2002

46

Site with no firewall

ISP Router

Site Router

Site Network

Link (T1 etc.)

24

47

Site with firewall

ISP Router

Site Router

Site Network

Firewall

48

Site with firewall

ISP Router

Site Router

(Nothing is this simple!)Firewall

25

49

DMZ (“De-Militarized Zone”)

ISP Router

Firewall/NAT

DMZ:Network outside of Site security perimeter used to deploy firewall(s) and publicly available services (Web, FTP, DNS, etc.)

50

Various DMZ deployments are possible

ISP Router

Site Router

Site Router

Firewall/NATFirewall/NATFirewall/NAT

26

51

History: Firewalls were rogue components

Firewall/DMZ architecture never part of the “official” Internet Architecture

Purely a commercial creationDistrusted by IAB (Internet Architecture Board)

“Crunchy on the outside, soft on the inside”

“All security should be end-to-end”, etc…

52

Firewalls not just protection from outside attackers

Bandwidth controlBlock high bandwidth applicationsPointcast, Napster

Employee network usage controlBlock games, pornography, non-business uses

PrivacyDon’t let outside see what you have, how big you are, etc.Similar to making corporate phone directory proprietary

27

53

Firewall functions

Dropping packetsAccording to 5-tuple and direction of packet (incoming or outgoing)

Recall: 5-tuple = src/dst address, src/dst port, protocol

According to “conversation”Multiple related flows, like FTP, SIP

According to higher-layer info (i.e. URL)Steering packets/messages

To other filters, like spam filter, virus checker, HTTP filter, etc.

Logging flows and statistics

54

Simple firewall policy configuration

dropanyany-outsideany-insideallowFTPany-outsideany-inside

dropanyany-insideany-outside

allowHTTPany-outsideany-insidedropSMTPany-outsideany-insideallowSMTPdmz-mailany-inside

ActionAppDestSource

28

55

Conversations

FTP consists of two flows, control flow and data flowFirewall must be smart enough to read control flow, identify subsequent data flowTrue for SIP as well

56

Stateful and stateless firewalls

Original firewalls were statelessMaintain static filter list, but no per flow stateFor TCP, only look at SYN

Means that non-SYN TCP packets are allowed even if should be blocked

No concept of conversationModern firewalls are typically stateful

Maintains dynamic list of all allowed flowsBetter capability, harder to scale

29

57

Problem for app developer

Obviously, your application may be blocked by the firewallTwo basic strategies:

1. Hide the application inside HTTP2. Make it easy for the firewall

administrator to allow your applicationWhich strategy you use depends on why the app is being blocked

58

Intentional versus unintentional blocking

Unintentional blocking:Blocking is a side effect of a broader policy

i.e., all UDP blocked, even though in principle the admin has no problem with your application

Intentional blocking:The admin knows of your application, and really does want to block it

i.e. Napster

30

59

Strategy for intentional blocking

Long term, this is a hard battle to winCan try to hide everything in what looks like normal HTTP, but the administrator can fight this in various ways:

Block on specific URLsBlock on specific IP addressesDisallow the application on the client computers

Better to solve the network admin’s concerns

Allow a caching proxy in the DMZAlthough this didn’t work for Pointcast….

60

Strategy for unintentional blocking

“Hide” the application in HTTPBut also allow the application to run “natively” if you get performance benefits

Make firewall configuration for allowing the application as simple as possiblei.e. one or a small number of specific portsGet the port blessed by IANA

Internet Assigned Numbers Authority

31

IDS

Copyright 2002

62

Intrusion detection

“Building burglar alarms for the net”Idea: make systems sensitive to threatening actions, and make them capable of alerting authorities when they notice anomaliesNecessarily post-hocBroad types

Statistical analyzers (anomaly based)Rules-based systems, Attack-signature detectors (misuse)Others

32

63

Know Your Attacker

Most attackers run scripts to probe for vulnerabilities, then return later to exploit them Probes tend to come in waves as new holes are discoveredProbes look very different than typical network useActual attack may come long after probe

64

Paradigms in Intrusion Detection

Misuse Detection Intrusion Detection Systems (MD)

define “what is abnormal” using attack signaturestraffic that matches an attack signature as attack traffic

Anomaly Detection Intrusion Detection Systems (AD)

define “what is normal” using profilestraffic that does not match the profile as abnormal

33

65

The world’s simplest ID system

v=listen(frequently-exploited-unused-port);while(1) {

s=accept(v, who, howbig);notify_the_authorities(s, who, howbig);close(s);

}This won’t catch stealth scannersDoesn’t have a global viewCan’t detect attacks on systems in useSurprisingly effective at catching scans nonetheless

66

Statistical analysis

Constantly capture packets, watch logs, note typical flows

I.E. “95% of traffic flows from inside the firewall to outside web services”Set off alarm bells when traffic not matching typical flows is seenCan be a first alert against configuration problems

Gains a global picture of the system

34

67

Rule-based systems

Monitor logs and network for behavior violating or matching static rulesRequire some knowledge of attack behaviorsLess prone to false alarmsOften combined with anomaly detectors

68

Others: nfr

Truly a post-hoc systemIdea: a “flight data recorder” for the network to aid in post-hoc recovery and retaliationActually morphing into a rules-based system built around a fast packet capture engine

Powerful filter programming languageFree!

35

69

Using an IDS

Plan your incident response process well before you install the systemKnow what you’re looking forMake the system comprehensiveDon’t overreact to alarmsIf using a rules-based system, keep up with vulnerability reports

VPNs

Copyright 2002

36

71

VPN Taxonomy

VPN

End-to-endNetwork

Provider-based Customer-basedProvider-based Customer-based

L3L2

ATMFrame RelayLAN

72

What is a VPN?

Making a shared network look like a private networkWhy do this?

Private networks have all kinds of advantages

(we’ll get to that)But building a private network is expensive

(cheaper to have shared resources rather than dedicated)

37

73

History of VPNs

Originally a telephone network conceptSeparated offices could have a phone system that looked like one internal phone system

Benefits?Fewer digits to dialCould have different tariffs

Company didn’t have to pay for individual long distance calls

Came with own blocking probabilities, etc.Service guarantees better (or worse) than public phone service

74

Original data VPNs

Lots of different network technologies in those daysDecnet, Appletalk, SNA, XNS, IPX, …None of these were meant to scale to global proportionsVirtually always used in corporate settings

Providers offer virtual circuits between customer sites

Frame Relay or ATMA lot cheaper than dedicated leased lines

Customer runs whatever network technology over these These still exist (but being replaced by IP VPNs)

38

75

Advantages of original data VPNs

Repeat: a lot cheaper than dedicated leased lines

Corporate users had no other choiceThis was the whole business behind frame-relay and ATM services

Fine-grained bandwidth tariffsBandwidth guarantees

Service Level Agreements (SLA)“Multi-protocol”

76

How has the world changed?

Everything is IP nowSome old stuff still around, but most data networks are just IP

So, why do we still care about VPNs???

39

77

IP VPN benefits

IP not really global (private addresses)

VPN makes separated IP sites look like one private IP network

SecurityBandwidth guarantees across ISP

QoS, SLAsSimplified network operation

ISP can do the routing for you

78

End-to-end VPNs

Solves problem of how to connect remote hosts to a firewalled network

Security and private addresses benefits onlyNot simplicity or QoS benefits

40

79

End-to-end VPNs

Solves problem of how to connect remote hosts to a firewalled network

Site (private network)Internet

RemoteHost

RemoteHost

FW/VPN

SiteHost

SiteHost

IPsecTunnels

80

Provider-based end-to-end VPNs

Used for instance when enterprise pays for employee access, wants it to go through enterprise network

I know Cisco did thisBut never used that much

Business model didn’t take offUsed even less now

In part because VPN client comes with windows OS???

The tunneling technology commonly used for roaming dialup though

41

81

Reiterate network VPN benefits

Makes separated IP sites look like one private IP networkSecurityQoS guaranteesSimplified network operation

82

Site

Customer-based Network VPNs

CE

Site

SiteSite

CE

CECE

Internet

Customer buys own equipment, configures IPsec tunnels over the global internet, manages addressing and routing. ISP plays no role.

42

83

Customer-based Network VPNs

Great for enterprises that have the resources and skills to do it

Large companiesMore control, better security model

Doesn’t require trust in ISP ability and intentionsCan use different ISPs at different sites

But not all enterprises have this skill

84

Site

Provider-based Network VPNs

PE

Site

Site

Site

PE

PEPE

ISP

Provider manages all the complexity of the VPN. Customer simply connects to the provider equipment.

CE

CE CE

CE

43

85

Model for customer

Attach to ISP router (PE) as though it was one of your routersRun routing algorithm with it

OSPF, RIP, BGPPE will advertise prefixes from other sites of same customer

86

Various PPVPN issues

Tunnel type?IPsec (more secure, more expensive)GRE etc.

How to discover which customer is at which PE?

Don’t want PEs without given customer to participate in routing for that customer

How to distinguish overlapping private address spaces