identifying and mitigating network...

Identifying and Mitigating Network Threats BRKSEC-2014

John Stuppi, CCIE #11154

Senior Network Security Engineer

Cisco Security Intelligence Operations (SIO)

© 2013 Cisco and/or its affiliates. All rights reserved. BRKSEC-2014 Cisco Public

Housekeeping

We value your feedback—do not forget to complete your online session evaluations after each session and complete the Overall Conference Evaluation which will be available online starting Thursday Visit the World of Solutions Please remember this is a non-smoking venue! Please switch off your mobile phones Please make use of the recycling bins provided Please remember to wear your badge at all times including the party!


Agenda State of Network Security

Threat Models for IP Networks

Incident Response

Device Hardening Overview – Three Planes of the Network

Attack Identification – NetFlow – Packet Capture – DNS Analysis

Case Study

Keeping Up To Date


Abstract This session provides strategies that will enable participants to make use

of their existing network infrastructure to identify and defend against complex threats.

You will learn Best Common Practices (BCPs) that will help in the detection and mitigation of existing, current, and emerging threats that can adversely impact your infrastructure.

I hope to impart knowledge on you of the various sources of events, messages, and data-types used during incident handling.

I will NOT try to sell you Cisco products!

I do NOT know everything…far from it!


Related Sessions

LTRSEC-2014 - Basic Network Threat Defense, Countermeasures, and Controls (Joseph Karpenko - Sr. Security Engineer and Randall Ivener - Manager, SIO) - In this four hour instructor-led lab, attendees will learn about inherent security features and techniques on Cisco IOS software including Management Plane hardening, Infrastructure Access-lists, and Data Plane hardening.

LTRSEC-2015 - Advanced Network Threat Defense, Countermeasures, and Controls (Joseph Karpenko - Sr. Security Engineer and Randall Ivener – Manager, SIO) - In this four hour instructor-led lab, attendees will perform the role of a Security Incident Response Investigator and must correctly detect, classify, and prevent threats targeting a network by configuring and deploying advanced threat defenses and countermeasures, such as Control Plane Policing, IOS NetFlow, and Remotely-Triggered Black Hole (RTBH) Routing on network infrastructure devices.

LTRSEC-2016 - Firewall Threat Defense, Countermeasures, and Controls (Andrae Middleton- Security Engineer and Panos Kampanakis - Consulting Engineer) - This instructor-led lab will provide administrators and engineers of Cisco Firewalls and Cisco next-generation Firewalls the knowledge and understanding to protect their networks against threats and attacks, leveraging industry standard and Cisco Best Practices.




Incident Response



Case Study

Keeping Up To Date

State of Network Security


Network Security = Cyber Security

9

Cyber (computer) security: The field covers all the processes and mechanisms by which computer-based equipment, information and services are protected from unintended or unauthorized access, change or destruction. Computer security also includes protection from unplanned events and natural disasters. Source: Wikipedia

http://en.wikipedia.org/wiki/Computer_security

http://en.wikipedia.org/wiki/Computer_security


2013 Cisco Annual Security Report

10

“…recent campaigns against a number of high-profile companies — including U.S. financial institutions — serve as a reminder that any cyber security threat has the potential to create significant disruption, and even irreparable damage, if an organization is not prepared for it.” “Cybercrime is no longer an annoyance or another cost of doing business. We are approaching a tipping point where the economic losses generated by cybercrime are threatening to overwhelm the economic benefits created by information technology. Clearly, we need new thinking and approaches to reducing the damage that cybercrime inflicts on the well- being of the world.” John Stewart, Cisco Senior Vice President and Chief Security Officer


Verizon 2013 Data Breach Investigations Report

11

“The bottom line is that unfortunately, no organization is immune to a data breach in this day and age…” “We have the tools today to combat cybercrime, but it’s really all about selecting the right ones and using them in the right way.” “In other words, understand your adversary – know their motives and methods, and prepare your defenses accordingly and always keep your guard up…” Source: Verizon 2013 Data Breach Investigations Report


Infrastructure Security Is An Attractive Target

12

The number of DDoS attacks in Q1 2013 increased by 21.75 percent over the same period of last year.

Attacks targeting the infrastructure layer represented more than a third of all attacks observed during the first three months of 2013. "What defined this quarter (Q1 2013) was an increase in the targeting of Internet Service Provider (ISP) and carrier router infrastructures…" Source: Prolexic


Network Security is a System Firewall + AV ≠ Network Security

– Just ask the Financial Community! Network security is not something you can just buy

– Technology will assist – Policy, operations, and design are more important Network security system

– A holistic collection of network-connected devices – Strategies, Technologies, Solutions – Best Common Practices (BCPs) that work in complementary ways to provide

security to information assets




Incident Response



Case Study

Keeping Up To Date



Knowledge of threats provides a firmer understanding of vulnerabilities and risks associated with your network Without a thorough understanding of threats, you cannot take the necessary

steps to implement an effective security solution Vulnerability scoring can help determine risk (CVSS)

– www.first.org/cvss – Anyone using CVSS?

http://www.first.org/cvss


Threats Against IP Networks

Many factors threaten network infrastructures – Natural disasters – Unintentional, man-made attacks based on human error – Malicious attacks Clear distinction between human error and malicious attacks is intent Protection against malicious and unintentional attacks must both be considered

– An outage is an outage Recent Examples

– GoDaddy (BGP issue) (Sept 2012) Corrupted Router Tables

– CloudFlare (BGP FlowSpec policy) outage (March 2013) Large Packet Size Attribute

Doh!


Threat and Attack Models Description

Resource Exhaustion Attacks

DoS attack makes target unavailable for its intended service

Attempted by direct, transit, or reflection-based attack

Spoofing Attacks Uses packets that masquerade with false data (such as source IP address) to exploit a trust relationship

Transport Protocol Attacks

Prevents upper-layer communication between hosts or hijacks established session

Exploits previous authentication measures

Enables eavesdropping or false data injection

Routing Protocol Attacks

Prevents or disrupts routing protocol peering or redirects traffic flows

Attempts to inject false information, alter existing information, or remove valid information


Threat and Attack Models (continued)

Description

Attacks Against IP control-plane Services

Attacks against DHCP, DNS, and NTP

Affects network availability and operations

Unauthorized Access Attacks Attempts to gain unauthorized access to restricted systems and networks

Software Vulnerabilities

Software defect that may compromise confidentiality, integrity, or availability of the device and data plane traffic

Malicious Network Reconnaissance

Gathering info about a target device, network, or organization

Enables attacker to id specific security weaknesses that may be exploited in a future attack.


Collateral Damage

Attacks may have additional consequences – Cloaked malicious behavior A DoS attack against one remote network may adversely affect other networks

resulting in collateral damage and a wider impact Collateral damage must also be considered when evaluating risk and impact of

potential attacks And speaking of collateral damage…


Remember this? Did you live through it?

21

More on this later…

http://www.cisco.com/web/about/security/intelligence/ERP-financial-DDoS.html




Incident Response



Case Study

Keeping Up To Date

Incident Response

Preparation • Prep the network • Create tools • Test tools • Prep procedures • Train team • Practice

Identification • How do you know about

the attack? • What tools can

you use? • What’s your process for

communication?

Classification • What kind of attack

is it?

Traceback • Where is the attack

coming from? • Where and how is it

affecting the network?

Reaction • What options do you have

to remedy? • Which option is the best

under the circumstances?

Postmortem • What was done? • Can anything else be

done to prevent it? • How can it be less

painful in the future?

Six Phases Of Incident Response

Preparation •Prep the network •Create tools •Test tools •Prep procedures •Train team •Practice




communication?


is it?












Preparation

Includes technical and non-technical components Encompasses Best Common Practices (BCPs) The hardest and most important phase Failure is likely without adequate preparation Do not implement foundational best practices and processes during an attack

Develop and Deploy a Solid Security Foundation


Preparation (cont.)

Know the enemy – Understand what drives the miscreants – Understand their techniques Create the security team and policies and procedures Prepare the tools

– Network telemetry – Reaction tools – Understand performance characteristics Harden devices

– Cisco Guide to Harden Cisco IOS Devices – http://www.cisco.com/en/US/tech/tk648/tk361/technologies_tech_note09186a0080120f

48.shtml PRACTICE, PRACTICE, PRACTICE!

http://www.cisco.com/en/US/tech/tk648/tk361/technologies_tech_note09186a0080120f48.shtml



Identification • How do you know

about the attack? • What tools can

you use? • What’s your process

for communication?


is it?












How to Know When You or Your Customer is Under Attack

Identification

Preemptive awareness of malicious or nefarious behaviors and other incidents What tools are available? What can you do today on a tight budget…or NO Budget? Indicators Of “Compromise” (IOC)


Identification – Sources of Detection

User/Customer call “The Internet is down”!! Unexplained changes in network

baseline SNMP: line/CPU overload, drops Bandwidth Cisco IOS NetFlow ACLs & Firewall Rules Logs – System, Application, DNS Sinkholes Packet capture

IDS/IPS Alarms Anomaly detection

DNS Analysis AV/Anti-Malware/Anti-Spyware Port scans Vulnerability Scans Patch Status Configuration/Change

Management Alerts Data Loss Prevention





communication?

Classification •What kind of attack is it?












Classification

Classification—understand the details and scope of the attack Identification is not sufficient Once an attack is identified, details matter Guides subsequent actions Identification and classification are often simultaneous


Classification (continued)

Qualify and quantify the attack without jeopardizing services availability – What type of attack has been identified? – What’s the effect of the attack on the victim(s)? – What next steps are required (if any)? – Don’t contribute to the attack (e.g. crash a router!) At a minimum

– Source and destination address – Protocol information – Port information





communication?


is it?

Traceback • Where is the attack coming from?

• Where and how is it affecting the network?









Traceback

Traceback—what are the sources of the attack? – How to trace to network ingress points – Your Internet connection is not the only vector – Understand your topology Traceback to network perimeter…then work with your ISP.

– NetFlow – Backscatter – Packet accounting


Traceback (cont.)

Retain attack data – Use to correlate inter-domain traceback – Required for prosecution – May help to deter future attacks – Clarify billing and other disputes – Postmortem analysis





communication?


is it?




Reaction • What options do you have to remedy?

• Which option is the best under the circumstances?






Reaction

Should you mitigate the attack? – Where? – How? No reaction is a valid form of reaction in certain circumstances Reaction often entails more than haphazardly applying

an ACL to a router





communication?


is it?












Postmortem—the Forgotten Step

Analyze what worked, what didn’t, and what can be improved Learn how to protect against repeat occurrences Understand if the DoS attack was the real threat or a smoke screen for

something else that just happened – Remember “Cloaked malicious behavior”? Learn how to make it faster, easier, and less painful in the future Gather and understand metrics

– Resources, headcount, additional information




Incident Response



Case Study

Keeping Up To Date

Device Hardening


Device Hardening

The Cisco Guide to Harden Cisco IOS Devices – http://www.cisco.com/en/US/tech/tk648/tk361/technologies_tech_note09186a0080120f

48.shtml Collection of BCPs and recommendations that secure an IOS device Structured around the three planes of a network

– Management Plane – Control Plane – Data Plane Contains an introductory overview on Secure Operations

– Not a network plane – Very important to the overall security and health of a network




Secure Operations

The operating procedures and methodologies in use on a network contribute greatly to the security of the network Key recommendations

– Monitor Cisco Security Advisories – Leverage AAA – Centralize Log Collection and Monitoring – Use Secure Protocols – Gain Traffic Visibility with NetFlow – Implement Configuration Management


Monitor Vendor Security Advisories

In order for a network to be secure, the network devices within the network must use patched software Security bugs in Cisco products are disclosed using Security Advisories,

Notices, and Responses – http://www.cisco.com/go/psirt By monitoring these documents, an administrator is more able to learn about

security vulnerabilities that may affect their network – Available as RSS feed: http://tools.cisco.com/security/center/rss.x?i=44 – Email Subscription: [email protected] Please subscribe!!

http://www.cisco.com/go/psirt

http://tools.cisco.com/security/center/rss.x?i=44

mailto:[email protected]

Three Planes of the Network


Three Planes of the Network - Summary

Data plane – Packets going through the router Control plane

– Routing protocols gluing the network together Management plane

– Tools and protocols used to manage the device


Route Processor CPU

Punt

ed P

acke

ts

Rec

eive

/Hos

t Pat

h

CEF Forwarding

Path Ingress Packets Forwarded Packets

Data Plane All Packets

Forwarded Through the Platform

Data Plane

Data Plane

Multiple Paths for Punted Packets

Receive/Host Path

Transit/cef-exception Path

Tran

sit/E

xcep

tion

Path


Data Plane

The Data Plane of the network is made up of user and application data transiting your network infrastructure – The very reason that the network exists in the first place Examples of Data Plane traffic includes:

– User web browsing – Email – Streaming video – IP Telephony – Facebook & Twitter Data Plane survivability is secondary to that of the Management and Controls

Plane – Without the Management and Control Planes the Data Plane will cease to function


Limiting CPU Impact of Data Plane Traffic

There are many types of Data Plane traffic that can elevate the CPU load of an IOS device, for example: – Packets logged through Access Control Lists – Packets containing IP options – Packets requiring fragmentation – Non-IP traffic To reduce the possibilities of CPU load induced outages, a network should:

– Filter packets containing IP options where they are not needed – Minimize CPU intensive features such as ACL logging and IP fragmentation – Limit the generation of ICMP Unreachable and Time-exceeded messages


Route Processor CPU

Punt

ed P

acke

ts

Rec

eive

/Hos

t Pat

h

CEF Forwarding


Control Plane


Receive/Host Path


Tran

sit/E

xcep

tion

Path

Control Plane ARP, BGP, OSPF,

NTP…and Other “Glue” Protocols

Control Plane


Control Plane

The Control Plane of a network consists of the protocols that enable the network elements to function cooperatively Examples of Control Plane protocols include:

– Interior Gateway Protocols such as EIGRP or OSPF – Border Gateway Protocol (BGP) – Network Time Protocol (NTP) – Label Distribution Protocols (LDP) Used with Multiprotocol Label Switching (MPLS)

The resiliency of the Control Plane is vital to the success of the Management and Data Planes


Control Plane Security The control plane is the logical group that contains all routing, signaling,

link-state, and other control protocols used to create and maintain the state of the network and interfaces

Critical that control plane resources and protocols are protected –Keep the network up and running at all times –Prevent traffic redirection that could result in a DoS condition or eavesdropping –Survivability of control plane ensures that management and data planes are maintained and operational

The control plane also enables other protection mechanisms to help mitigate the risk of security attacks


Route Processor CPU

Punt

ed P

acke

ts

Rec

eive

/Hos

t Pat

h

CEF Forwarding


Management Plane


Receive/Host Path


Tran

sit/E

xcep

tion

Path

Mgmt Plane SSH, TFTP, SNMP,

FTP…and Other Mgmt Protocols

Mgmt Plane


Management Plane

The Management Plane of the network is made up of the protocols that support the operational needs of the network – SSH – SNMP The Management Plane includes:

– Interactive access to network devices – Statistics collection using SNMP – Software image deployment It is imperative that the Management Plane remain operational during a

security event – Without the Management Plane, it might be impossible to return the network to a

functional and secure state


What Is a Punt?

Receive adjacency (more on the next slide) Transit packets that need additional processing Specific router configuration: ACL logging, Cisco IOS FW, etc. IP Options set Require fragmentation ICMP Unreachables due to routing, MTU, or filtering Expired TTL (ICMP Time Exceeded) – good example = ? Destinations lacking a next-hop adjacency

(ARP—CEF Glean punt) Malformed fields (ICMP parameter error)


Receive Adjacencies

CEF entries for traffic destined for router

– Real interfaces and loopbacks – Broadcast and multicast address space Packets with a next hop of receive are sent to the device’s RP CPU for

processing Receive Adjacency traffic usually includes routing protocols, management

services, and multicast control


Receive Adjacencies Example

58

router#show ip cef Prefix Next Hop Interface 0.0.0.0/0 10.89.210.1 GigabitEthernet0/0 0.0.0.0/8 drop 10.82.0.1/32 10.89.210.1 GigabitEthernet0/0 10.89.210.0/23 attached GigabitEthernet0/0 10.89.210.66/32 receive GigabitEthernet0/0 127.0.0.0/8 drop 192.168.128.2/32 192.168.208.2 GigabitEthernet0/1 192.168.128.3/32 192.168.208.3 GigabitEthernet0/1 192.168.128.20/32 192.168.208.20 GigabitEthernet0/1 192.168.128.21/32 192.168.208.3 GigabitEthernet0/1 192.168.128.26/32 receive Loopback0


Receive Adjacencies Example (continued)

59

router#show ip interface brief Interface IP-Address OK? Method Status Protocol GigabitEthernet0/0 10.89.210.66 YES NVRAM up up GigabitEthernet0/1 192.168.208.26 YES NVRAM up up GigabitEthernet0/1.208 unassigned YES unset up up In2/0 unassigned YES NVRAM up up Loopback0 192.168.128.26 YES NVRAM up up Tunnel1 unassigned YES NVRAM up up




Incident Response



Case Study

Keeping Up To Date

Attack Identification


Network Behavioral analysis (NBA) Networks and network enabled devices constantly create traffic.

However, this traffic follows certain patterns according to the applications and user behavior

• Analyzing these patterns allows us to see what is NOT normal

• The key is to collect traffic information (NetFlow) and calculate various statistics

• These are then compared against a baseline and abnormalities are then analyzed in more detail

Cisco IOS NetFlow


Network Telemetry: NetFlow

NetFlow is telemetry pushed from routers/switches Each device can be a sensor Negligible performance impact on

routers Not just a Cisco thing Like a phone bill

– Packet capture is like a wire tap


NetFlow Collection

NetFlow data can be collected and relayed to multiple tools Cyber Threat

Defense – See: BRKSEC-2661


What Is a Traditional IP Flow?

1. Inspect a packet’s seven key fields and identify the values

2. If the set of key field values is unique, create a flow record or cache entry

3. When the flow terminates, export the flow to the collector

NetFlow Export

Packets Reporting

NetFlow Key Fields

1

2 3


Provides syslog-like information without having to buy a firewall

One NetFlow packet has information about multiple flows

router (config-if)# ip flow ingress router (config)# ip flow-export destination 172.17.246.225 9996

Export Packets • Approximately 1500

bytes • Typically contain 20–

50 flow records • Sent more frequently

if traffic increases on NetFlow-enabled interfaces

NetFlow - Internal Threat Information Resource

NetFlow Cache

… Flow Record

Flow Record

Header • Sequence number • Record count • Version number


router#show ip cache flow IP packet size distribution (10482236 total packets): 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480 .000 .287 .195 .003 .000 .513 .000 .000 .000 .000 .000 .000 .000 .000 .000 512 544 576 1024 1536 2048 2560 3072 3584 4096 4608 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 --- output removed for brevity --- Active flows timeout in 30 minutes Inactive flows timeout in 15 seconds --- output removed for brevity --- last clearing of statistics never Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec) -------- Flows /Sec /Flow /Pkt /Sec /Flow /Flow UDP-NTP 24723 0.0 1 96 0.0 0.0 15.4 UDP-other 541966 0.1 15 136 2.0 15.0 15.3 ICMP 64 0.0 1 56 0.0 8.0 15.0 IGMP 3 0.0 1 32 0.0 0.0 15.3 IP-other 58863 0.0 31 84 0.4 273.1 13.8 Total: 625619 0.1 16 127 2.4 38.7 15.2 SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP Pkts Gi0/0 192.168.134.3 Null 224.0.0.2 11 07C1 07C1 412 Gi0/0 192.168.134.25 Local 192.168.134.21 11 0065 0065 20 Gi0/0 192.168.134.144 Null 224.0.0.5 59 0000 0000 28 Gi0/0 192.168.134.25 Local 192.168.134.21 11 0065 07AF 1

NetFlow: On-device Cache Output

Traffic classification

Flow Summary

Detail


NetFlow Extensibility and Flexibility Requirements

Traditional NetFlow with the v5, v7, or v8 NetFlow export – New requirements: build something flexible and extensible Phase 1: NetFlow version 9

– Advantages: extensibility Integrate new technologies/data types quicker

(MPLS, IPv6, BGP next hop, etc.) Integrate new aggregations quicker

– Note: for now, the template definitions are fixed Phase 2: Flexible NetFlow

– Advantages: cache and export content flexibility User selection of flow keys User definition of the records

Exporting Process

Metering Process


Data FlowSet Template FlowSet Option Template FlowSet FlowSet ID #1

Data FlowSet FlowSet ID #2

Template ID

(Specific Field Types

and Lengths)

(Version, Number of Packets,

Sequence Number,

Source ID)

Flows from Interface A

Flows from Interface B

To Support Technologies such as MPLS or Multicast, this Export Format Can Be Leveraged to Easily Insert New Fields

Option Data FlowSet

FlowSet ID

Option Data

Record (Field

Values)

Option Data

Record (Field

Values)

Template Record

Template ID #2

(Specific Field Types and Lengths)

Template Record

Template ID #1

(Specific Field Types and Lengths)

Data Record (Field Values)

Data Record (Field Values)

Data Record

(Field Values)

Matching ID numbers is the way to associate template to the data records Same header format as prior NetFlow versions Each data record represents one flow

NetFlow Version 9 Export Packet


Introduction to Flexible NetFlow (FNF)

Fixed export formats (NetFlow version 1, 5, 7, 8) are not flexible and adaptable. Each new version contains new export fields; incompatible with previous version. Flexible NetFlow completely separates the collection and export process Allows customization of NetFlow collection Offers new export protocols (UDP, SCTP)

Flexible NetFlow availability started in Release 12.4(9)T

FNF Technology White Paper - http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6555/ps6601/ps6965/prod_white_paper0900aecd804be1cc.html

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6555/ps6601/ps6965/prod_white_paper0900aecd804be1cc.html

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6555/ps6601/ps6965/prod_white_paper0900aecd804be1cc.html


FNF Components

The flow monitor is a flow cache with flow records – Applied to an interface – Flow monitors can be ingress or egress – Packet sampling possible per flow monitor

Flow monitor components – Flow record—defines what is captured by NetFlow – Flow records have two formats:

– Pre-defined or user-defined schemes – Include key and non-key fields

Flow exporter—where NetFlow will be exported – Multiple flow exporters per Flow Monitor


FNF - Multiple Monitors with Unique Key Fields

Key Fields Packet 1

Source IP 3.3.3.3

Destination IP 2.2.2.2

Source port 23

Destination port 22078

Layer 3 Protocol TCP - 6

TOS Byte 0

Input Interface Ethernet 0

Source IP Dest. IP Dest. I/F Protocol TOS … Pkts

3.3.3.3 2.2.2.2 E1 6 0 … 11000

1.1.1.1 2.2.2.2 E1 6 0 … 11000

Traffic Analysis Cache

Flow monitor

1

Traffic

Key Fields Packet 2

Source IP 3.3.3.3

Dest IP 2.2.2.2

Input Interface Ethernet 0

Packet Section 1010101

Source IP Dest. IP Dest. I/F Input I/F Sec … Pkts

3.3.3.3 2.2.2.2 E1 E1 101 … 11000

Security Analysis Cache

Non Key Fields

Packets

Bytes

Time Stamps

Next-Hop Address

Non Key Fields

Packets

Time Stamps

Flow monitor

2


FNF Configuration Example

Configure the Interface int s3/0 ip flow monitor my-monitor input

4

flow monitor my-monitor exporter my-exporter record my-app-traffic

Configure the Flow Monitor 3

flow exporter my-exporter destination 10.1.1.1

Configure the Exporter 2

flow record my-app-traffic match transport tcp source-port match transport tcp destination-port match ipv4 source address match ipv4 destination address collect counter bytes collect counter packets

Configure the Flow Record

1


NetFlow Deployment Considerations

Enable Netflow on all router interfaces – NetFlow should typically be enabled on all router

interfaces where possible, it is useful for on-box troubleshooting via CLI as well as for export to analysis systems

How to sample Netflow records – 1:1 NetFlow is useful for troubleshooting, forensics, traffic

analysis, and behavioral/relational anomaly-detection – Sampled NetFlow is useful for traffic analysis and

behavioral/relational anomaly-detection.


Reducing NetFlow Performance Impact

Flexible NetFlow (collect only what is really required) Aging timers Sampled NetFlow Leverage distributed architectures (VIP, linecards) Flow masks (only Cisco Catalyst 6500/7600) Filters (router or collector) Data compression (collector) Increase collection bucket sizes (collector) Place collector and router on the same LAN segment/

dedicated interface

NetFlow Traceback Techniques


Traceback with NetFlow

router1#sh ip cache flow | include <destination>

Se1 <source> Et0 <destination> 11 0013 0007 159

…. (lots more flows to the same destination)

router1#sh ip cef se1

Prefix Next Hop Interface

0.0.0.0/0 10.10.10.2 Serial1

10.10.10.0/30 attached Serial1

The Flows Come from Serial 1

Victim

Find the Upstream Router on Serial 1

Continue on This Router


Useful NetFlow CLI Tricks

Router>show ip cache flow | include <ip address> – Determine flows pertaining to a specific victim or attacker Router>show ip cache flow | include _1 $

– Determine single packet flows (potential scanning flows) Router>show ip cache flow | include K|M $

– Determine really large flows (in 1,000s or 1,000,000s of packets)

Router>show ip cache flow | include [protocol|port] – Determine flows with specific protocols/ports


Router# show ip cache flow … SrcIf SrcIPaddress SrcP SrcAS DstIf DstIPaddress DstP DstAS Pr Pkts B/Pk 29 192.1.6.69 77 aaa 49 194.20.2.2 1308 bbb 6 1 40 29 192.1.6.222 1243 aaa 49 194.20.2.2 1774 bbb 6 1 40 29 192.1.6.108 1076 aaa 49 194.20.2.2 1869 bbb 6 1 40 29 192.1.6.159 903 aaa 49 194.20.2.2 1050 bbb 6 1 40 29 192.1.6.54 730 aaa 49 194.20.2.2 2018 bbb 6 1 40 29 192.1.6.136 559 aaa 49 194.20.2.2 1821 bbb 6 1 40 29 192.1.6.216 383 aaa 49 194.20.2.2 1516 bbb 6 1 40 29 192.1.6.111 45 aaa 49 194.20.2.2 1894 bbb 6 1 40 29 192.1.6.29 1209 aaa 49 194.20.2.2 1600 bbb 6 1 40

What Does a DoS Attack Look Like?

Typical DoS attacks have the same (or similar) entries:

– Input interface, destination IP, one packet per flow, constant bytes per packet (B/Pk)

Remember our NetFlow CLI Tricks? Export to a security-oriented collector: Lancope, Arbor

80


0 1 2 3

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

|Version| IHL |Type of Service| Total Length |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| Identification |Flags| Fragment Offset |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| Time to Live | Protocol | Header Checksum |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| Source Address |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| Destination Address |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| Options | Padding |

Useful Fields for Security Monitoring Attacks or Worms that Use Consistent Packet Size

Several Flows with the Same Fragment Offset: Same Packet Sent Over and Over

Very Large Packets or Attacks that Might Have the Same Generated Identification

Flow Issued From the Same Origin




Incident Response



Case Study

Keeping Up To Date

Using the Network to Capture Attack Packets


Packet Capture

Sometimes there is no substitute for looking at the packets on the wire Facilitate packet capture from Cisco switches

– SPAN – RSPAN – ERSPAN – VACL Capture Router IP Traffic Export (RITE) and Embedded Packet Capture (EPC) allows

packet capture from routers Can utilize open source tools such as tcpdump, snoop, Wireshark

(http://www.wireshark.org) Use macroanalytical telemetry (SNMP, NetFlow, RMON) to guide your use of

microanalytical telemetry (packet capture)

http://www.wireshark.org/


Embedded Packet Capture (EPC)

Capture packets flowing from, to, or through a given device Captures can be filtered using ACLs Captured packets can be viewed on router or exported for analysis in sniffer

programs such as Wireshark CEF and process switched flows are supported IPv4 and IPv6 are supported Available since Release 12.4(20)T

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6555/ps9913/datasheet_c78-502727.html http://www.cisco.com/en/US/docs/ios-xml/ios/epc/configuration/15-2mt/nm-packet-capture.html

85

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6555/ps9913/datasheet_c78-502727.html




http://www.cisco.com/en/US/docs/ios-xml/ios/epc/configuration/15-2mt/nm-packet-capture.html










Embedded Packet Capture (EPC) - Configuring EPC Define a Capture Point

Define a Capture Buffer

86

Router#monitor capture point ip cef CAPTURE-POINT FastEthernet0/0 both

Switching Path (Either cef or

process-switched)

Capture Point Name

Interface on Which to Capture (or All)

Traffic Flow Direction (In, Out, or Both)

Protocol (Either ip for IPv4 or ipv6 for IPv6)

Router#monitor capture buffer CAPTURE-BUF size 1024 max-size 1518 circular

Capture Buffer Name

Maximum In-Memory Size in Kilobytes (KB)

Maximum Packet Slice Size

Buffer Type (linear Locks When Full, circular Wraps)


Embedded Packet Capture (EPC) - Using EPC

Optionally limit the packets captured – By time – By number of packets captured – Drop every Nth packet – Limit packet rate Associate the capture buffer to the capture point

Start the capture

87

Router#monitor capture point associate CAPTURE-POINT CAPTURE-BUF

Router#monitor capture point start CAPTURE-POINT


Embedded Packet Capture (EPC) - Using EPC

88

If the buffer is linear, the capture will run until the buffer is full or until the limiting criterion is met

The capture can also be manually stopped

Or the buffer can be exported to a network server (or local file) in

Router#monitor capture point stop CAPTURE-POINT

Router#show monitor capture buffer CAPTURE-BUF dump 21:24:44.951 UTC Jun 18 2012 : IPv4 LES CEF : Gi0/0 None 022253E0: 002699F6 DDD00018 74B5A41B 080045C0 .&.v]P..t5$...E@ 022253F0: 00528449 0000FE06 1594C0A8 D001C0A8 .R.I..~...@(P.@( 02225400: D0B50017 B6E1D9F7 0E0CECF6 59A75018 P5..6aYw..lvY'P. 02225410: 10050576 00000D0A 0D0A5573 65722041 ...v......User A 02225420: 63636573 73205665 72696669 63617469 ccess Verificati 02225430: 6F6E0D0A 0D0A5573 65726E61 6D653A20 on....Username: 02225440: 77 w

Router#monitor capture buffer CAPTURE-BUF export tftp://192.168.1.100/pod1.pcap


Embedded Packet Capture (EPC) – Wireshark Export

89


Embedded Packet Capture (EPC) Considerations

90

Make sure there is enough free memory to hold the maximum configured capture buffer size

In 12.4T, capture buffers could only be exported to network servers – Local export available in 15.0

Jumbo frame support (packets over 1024 bytes) added in 15.0

Access-list filtering does not match on packets generated by the device itself

MPLS frames are not currently captured

Multicast frames are only captured on ingress


Firewall Packet Capture

Real time option

Capture wizard in ASDM

Capture packets on an interface that match an ACL, or match line

Key steps – Create an ACL that will match interesting traffic – Define the capture and bind it to an access-list and interface – View the capture on the firewall or copy it off in .pcap format

capture <capture-name> [access-list <acl-name>] [buffer <buf-size>] [ethernet-type <type>] [interface <if-name>] [packet-length <bytes>] [circular-buffer] [type raw-data|asp-drop|isakmp|webvpn user <username>] [match <prot> {host <sip> | <sip> <mask> | any} [eq | lt |gt <port>] {host <dip> | <dip> <mask> | any} [eq | lt | gt <port>]] [real-time [dump] [detail] [trace]] [trace [detail] [trace-count <1-1000>]]


Firewall Packet Capture

Traffic can be captured before and after it passes through the firewall; one capture on the inside interface, one capture on the outside interface

Capture buffer saved in RAM (default size 512KB)

Default—stop capturing when buffer is full

Default packet length is 1518 bytes

Copy captures off by means of TFTP or HTTPS

Outside Inside

Capture In Capture Out


Packets Are Captured in Packet Flow

Packets are captured at the first and last points they can be in the flow

Ingress packets are captured before any packet processing has been done on them

Egress packets are captured after all processing (excluding L2 source MAC rewrite)

Ingress Packets Captured

Egress Packets Captured


Firewall Capture Command: Example Problem: User on the inside with an IP of 10.1.3.2 is having a problem

accessing www.cisco.com (198.133.219.25); the user is getting PATed to 192.168.2.2

Outside Inside

Capture In Capture Out

Internet

www.cisco.com

198.133.219.25 10.1.3.2 10.1.3.2 192.168.2.2

Step 1: Create ACL for both inside and outside interface

Step 2: Create captures on both inside and outside interface

Step 3: Have inside user access www.cisco.com

Step 4: Copy the captures off to a TFTP server

Step 5: Analyze captures with sniffer program

http://www.cisco.com/




Firewall Capture Command: Example Step 1: Create ACL for both inside and outside interface

– !-- Outside Capture ACL access-list 100 permit tcp host 192.168.2.2 host 198.133.219.25 eq 80 access-list 100 permit tcp host 198.133.219.25 eq 80 host 192.168.2.2

– !-- Inside Capture ACL access-list 101 permit tcp host 10.1.3.2 host 198.133.219.25 eq 80 access-list 101 permit tcp host 198.133.219.25 eq 80 host 10.1.3.2

Step 2: Create captures on both inside and outside interface – capture out access-list 100 interface outside packet-length 1518 – capture in access-list 101 interface inside packet-length 1518


Firewall Capture Command: Example (continued)

Step 3: Have inside user access www.cisco.com

Step 4: Copy the captures off to a TFTP server !-- ASA ver 7.0+ / FWSM 3.0+ copy capture copy /pcap capture:out tftp://10.1.3.5/out.pcap copy /pcap capture:in tftp://10.1.3.5/in.pcap OR copy using https: https://<FW_IP>/capture/out/pcap



Firewall Packet Capture: Example (cont.)

Step 5: Analyze captures with sniffer program Outside CAPTURE

Inside CAPTURE Outbound SYN, No SYN+ACK


show asp drop

98

• show asp drop shows packets and flows dropped by accelerated security path (ASP)

• Can aid in troubleshooting and incident response • Many different drop codes • Absence of a code means firewall has not dropped packet/flow for that

reason ASA# show asp drop Frame drop: Invalid UDP Length (invalid-udp-length) 9 No valid adjacency (no-adjacency) 66 No route to host (no-route) 38 Flow is denied by configured rule (acl-drop) 21245 First TCP packet not SYN (tcp-not-syn) 30 TCP data exceeded MSS (tcp-mss-exceeded) 13 TCP failed 3 way handshake (tcp-3whs-failed) 10 TCP invalid ACK (tcp-invalid-ack) 293


Capturing Packets Dropped by the ASP

Capture all packets dropped by the ASP – ASA# capture drops type asp-drop all

Capture on a specific drop reason – ASA# capture drops type asp-drop invalid-tcp-hdr-length

ASA# capture drop type asp-drop ?

acl-drop Flow is denied by configured rule all All packet drop reasons bad-crypto Bad crypto return in packet bad-ipsec-natt Bad IPSEC NATT packet bad-ipsec-prot IPSEC not AH or ESP bad-ipsec-udp Bad IPSEC UDP packet bad-tcp-cksum Bad TCP checksum bad-tcp-flags Bad TCP flags




Incident Response



Case Study

Keeping Up To Date

DNS Analysis


Utilizing DNS Telemetry for Detection

The Domain Name System (DNS) is a ‘background’ service we often don’t think about, but is in actuality used many, many times each day

Many types of application use name-based lookups – Web browsers, email servers, web servers – Malware such as trojans and bots running on compromised hosts

We can and should examine DNS logs and statistics

Correlate DNS-related info with other forms of telemetry (NetFlow, packet capture, application logs, etc.)

http://www.darkreading.com/monitoring/got-malware-three-signs-revealed-in-dns/240154181




Example: dnstop query types

103

0 new queries, 38 total queries Wed Jun 1 17:35:51 2013

Query Type count %

---------- --------- ------

A? 9 23.7

NS? 1 2.6

SOA? 1 2.6

PTR? 15 39.5

MX? 10 26.3

TXT? 2 5.3

Source: http://dns.measurement-factory.com/tools/dnstop/

http://dns.measurement-factory.com/tools/dnstop/




Example: dnstop sources output

0 new queries, 38 total queries Wed Jun 1 17:35:51 2013 Sources count % ------------ ----- ------ 172.19.61.44 19 50.0 172.19.60.28 9 25.0 172.19.61.33 9 25.0


Example: dnstop destination output

0 new queries, 38 total queries Wed Jun 1 17:35:51 2013 Destinations count % ---------------- ------ ------ 172.19.226.120 29 77.7 10.158.254.13 5 15.0 172.19.220.131 4 9.3


RRDTool Graph of DNS Queries/Sec

Source: http://oss.oetiker.ch/rrdtool/

Fri Jan 31 14:02:05 2012

http://oss.oetiker.ch/rrdtool/


DNS Correlation: Detecting Botnets

Kumamoto University in Tokyo have published several very good papers on DNS correlation – see http://dua.cc.kumamoto-u.ac.jp/~musashi/

http://dua.cc.kumamoto-u.ac.jp/~musashi/




Incident Response



Case Study

Keeping Up To Date

Case Study: Financial DDoS Attacks (Fall 2012)


Cisco Security Intelligence Operations Timeline Se

pt. 2

4

Cyber Risk Report

Oct

. 1

IntelliShield Security Bulletin

Oct

. 1

Event Response Page

Oct

. 3

Cisco IPS Signature

Oct

. 3

Threat Defense Bulletin

Oct

. 3

Cisco Security Blog Post

Oct

. 4

Applied Mitigation Bulletin

Customer Meetings

www.cisco.com/security


IntelliShield: Security Activity Bulletin

111

http://tools.cisco.com/security/center/viewAlert.x?alertId=27076

http://tools.cisco.com/security/center/viewAlert.x?alertId=27076


Cisco Event Response Page http://www.cisco.com/web/about/security/intelligence/ERP-financial-DDoS.html

http://www.cisco.com/web/about/security/intelligence/ERP-financial-DDoS.html


DDoS Attack Pattern Traffic Profiles – ICMP & UDP

113

Protocol Port Payload Notes Cisco Mitigations

Internet Control Message Protocol (ICMP; protocol 01)

N/A Varies

Flood of ICMP messages sent to the targeted/victim address. ICMP Type 8/Code 0 has been observed, but other ICMP types and codes could be used.

IPS: 6902.0 - Net Flood ICMP Request

IPS: 6901.0 - Net Flood ICMP Reply

IPS: 6903.0 - Net Flood ICMP Any

IPS: 2152.0 - ICMP Flood Cisco IOS tACL

Cisco IOS NetFlow Cisco ASA/ASA-SM/FWSM

UDP 80

"/http1" (hexadecimal value \x2f\x68\x74\x74\x70\x31)

"A" (hexadecimal value \x41)

The Data field, or payload, of the UDP message contains all /http1. The Data field, or payload, of the UDP message contains all /As.

IPS: 6910/0 - Net Flood UDP IPS: 4002.0 - UDP Host

Flood Cisco IOS tACL

Cisco IOS NetFlow Cisco ASA/ASA-SM/FWSM


DDoS Attack Pattern Traffic Profiles - TCP

114

Protocol Port Payload Notes Cisco Mitigations

TCP 53 None Flood of TCP SYN segments sent to TCP port

53

IPS: 6009.0 - SYN Flood DOS IPS: 6920.0 - Net Flood TCP

Cisco IOS NetFlow

TCP 80 None Flood of TCP SYN segments sent to TCP port

80

IPS: 6009.0 - SYN Flood DOS IPS: 6920.0 - Net Flood TCP

Cisco IOS NetFlow

TCP 80 Varies

HTTP GET method requests using varying HTTP header values and URI requests. HTTP GET method requests are sent to root document web pages and nonroot document web pages.

IPS: 1493/0 - Distributed Denial of Service on Financial

Institutions IPS: 6009.0 - SYN Flood DOS

Cisco IOS NetFlow

TCP 80 Varies

HTTP POST method using varying HTTP header values and submitted data. HTTP POST method sent to web pages expecting data input (for example, pages that require user login, contain forms, or expect user-submitted data).

IPS: 6009.0 - SYN Flood DOS Cisco IOS NetFlow

Cisco ASA/ASA-SM/FWSM


Cisco Applied Mitigation Bulletin (AMB)

Identification Techniques • NetFlow • Syslogs – IOS/ASA • IPS Events Mitigation Techniques • Transit Access Control

Lists – IOS/ASA • Unicast Reverse Path

Forwarding – IOS/ASA • Application Layer Protocol

Inspection - ASA • Threat Detection - ASA

http://tools.cisco.com/security/center/viewThreatOutbreakAlert.x?alertId=27115

http://tools.cisco.com/security/center/viewThreatOutbreakAlert.x?alertId=27115


Technical Recap of Attack

116

• Volume-based attack methods

• Outages caused by operational errors and untested configurations

• Scaling challenges with stateful devices such as firewalls

• Enormous attacks on multiple customers simultaneously

• Preannouncement of the attacks and attack targets

• Wide-scale coordination of “attackers” using social media

• Distribution of tools to enable newly recruited attackers

• Broad availability of high-bandwidth connections

• Security event coordination among Financial Institutions

Same Old, Same Old What was different this time?


Lessons Learned Old is New Again, DoS attacks are back in vogue

– Attacks can disrupt business operations

Security teams need to be prepared to respond to incidents – Develop play books to assist in incident response – TEST, TEST, TEST!

Traditional infrastructure may not be a solution – Content switches, load balancers, IPS, and firewalls will become points of failure

Scrubbing services offered by service providers are not the same as on-premise solutions – Scrubbing needs to be done on the provider side as well as at the edge – Cloud and virtualized scrubbing

Coordinate with LEOs – Law enforcement may offer information about other similar attacks which can help


Lessons Learned (continued) Setup communication channels with your ISAC, your vendor, your service

provider, and other organizations in your industry – Information gleaned from these partners could reduce incident response time and

provide additional countermeasures

Baseline your network, users, and business applications – Do not allow internal devices to be used to attack external hosts or exfiltrate data

Conduct a postmortem and root cause analysis – Business as usual may hinder responses – Updated incident response play books based on lessons learned – Be open to new solutions such as cloud mitigation options




Incident Response



Case Study

Keeping Up To Date

Keeping Up to Date


Cisco Security Intelligence Operations Portal Inform, Protect, Respond

• IntelliShield Alerts, IPS sigs, Applied Mitigation Bulletins, PSIRT Advisories, Threat Outbreak Alerts, SendorBase Virus Watch, Event Responses

• Event-based, early-warning security intelligence

• Proven Cisco mitigation solutions help protect networks

• Real-time threat activity, correlation and tracking and trend analysis

• Security best practice guidance • Event-driven alerts and reporting • IntelliShield Cyber Risk Report Podcast

for global security trends • Cisco Security Blog


Join the Conversation! Visit the Cisco Security Blog

–blogs.cisco.com/security

Posts from across Cisco’s security community

Wide range of topics – Emerging threats – Innovative techniques – Security research

Tell us what you think!

http://blogs.cisco.com/security


Intelligence at a Glance Cisco IntelliShield Event Response

Microsoft Security Bulletin ID Cisco IntelliShield Alert ID CVE ID Cisco Mitigations CVSS Base Score Impact on Cisco Products Related Information

Summary information, threat analysis, and mitigation techniques that feature Cisco products


Security Intelligence Operations Services IntelliShield Alert

Tactical, operational and strategic intelligence Vendor neutral Life cycle reporting Vulnerability workflow

management system Comprehensive searchable

alert database

Threat and vulnerability intelligence alerting service Receive vital intelligence that is relevant and targeted to your Environment


Security Intelligence Operations Services Cisco Cyber Risk Report

Addresses applicable major risk management categories: vulnerability, physical, legal, trust, identity, human, and geopolitical…and others.

The CRRs are a result of collaborative efforts, information sharing, and collective security expertise of senior analysts from Cisco security services

A Strategic Intelligence Report that Highlights Current Security Activity and Mid-to Long-range Perspectives


Vulnerability Characteristics Mitigation Technique Overview Risk Management Device-Specific Mitigation and

Identification

Security Intelligence Operations Services Cisco Applied Mitigation Bulletin

Actionable Intelligence that can be used with your existing Cisco infrastructure


IPv6 Service Provider Portal/Page Best Practices Attack Identification and Mitigation

Techniques Annual and Threat Reports

Security Intelligence Operations Services Cisco Tactical Resources

Information derived from research and expertise


Cisco IOS Software Checker

http://tools.cisco.com/security/center/selectIOSVersion.x

Tool on Security Intelligence Operations Portal

Query Cisco IOS Software versions against published Security Advisories

Simplifies identification of affected software versions

Long requested feature by Cisco customers




Maximize your Cisco Live experience with your free Cisco Live 365 account. Download session PDFs, view sessions on-demand and participate in live activities throughout the year. Click the Enter Cisco Live 365 button in your Cisco Live portal to log in.

Complete Your Online Session Evaluation

Give us your feedback and you could win fabulous prizes. Winners announced daily. Receive 20 Cisco Daily Challenge

points for each session evaluation you complete. Complete your session evaluation

online now through either the mobile app or internet kiosk stations.

129

identifying and mitigating network...

Documents