Hands-On Microsoft Windows Server 2003

Networking

Chapter 9

IP Security

2

Objectives

• Describe IP security issues and how the IPSec protocol addresses them

• Choose the appropriate IPSec mode for a given situation

• Implement authentication for IPSec• Enable IPSec• Creat IPSec policies• Monitor and troubleshoot IPSec

3

IP Security (IPSec) Overview

• Techniques used to eavesdrop on IP-based communication– Packet sniffing

• Packet sniffer software used to view all packets traversing your network

– Data replay• Packets are captured and replayed at later time

– Data modification• Packets can be modified before being replayed

– Address spoofing• Hackers can falsify source IP address and gain

unauthorized access to resources

4

IPSec Overview (Continued)

• IPSec– Secures IP-based communication– Supported by Internet Engineering Task Force

(IETF)– Authenticates endpoints of any IP-based

conversation– Encryption can be used to hide contents of data

packets– Exists at Network layer of TCP/IP architecture

5

IPSec Overview (Continued)

– Not supported by pre-Windows 2000 operating systems

– Can significantly slow communication on a network

– Cannot be routed through NAT

– Adds complexity to a network

– Can be valuable addition to a network when data integrity or confidentiality are required

6

IPSec (Continued)

7

IPSec Modes

• Tunnel mode– IPSec communication between two networks

• Transport mode– IPSec communication between two hosts

• Authentication headers (AH) mode – Enforces authentication of the two IPSec clients – Includes a digital signature on each packet

• Encapsulating security payload (ESP) mode– Has all the features of AH mode plus encryption

of data in the packet

8

AH Mode

• Provides authentication of the two endpoints• Adds checksum to the packet• Does not provide data confidentiality• Payload of the packet is unencrypted• Used in situations where you are concerned

about packets being captured• Less processor-intensive than ESP mode

9

ESP Mode

• Provides authentication of the two endpoints– Guarantees that the two endpoints are known

• Adds a checksum to each packet– Guarantees that the packet was not modified in

transit

• Encrypts the data in the packet– Ensures that unintended recipients cannot read

the data in the packet

• Used in most implementations of IPSec because data encryption is desired

10

Transport Mode

• Used between two hosts• Both endpoints in the communication must

support IPSec– Limits implementation of IPSec

– Many devices, such as printers, rarely offer IPSec support

11

Transport Mode (Continued)

12

Transport Mode (Continued)

13

Tunnel Mode

• Used between two routers• Hosts communicating through the routers do not

need to support IPSec– Routers take the original IP packets and

encapsulate them

– Any IP device can take advantage of routers running IPSec in tunnel mode

14

Tunnel Mode (Continued)

15

Tunnel Mode (Continued)

16

IPSec Authentication

• IPSec communication– Both endpoints are authenticated

• Internet Key Exchange (IKE)– Process used by two IPSec hosts to negotiate

their security parameters• Security association (SA)

– Term used when security parameters have been agreed upon

• Authentication methods– Preshared key– Certificates– Kerberos

17

Preshared Key

• A combination of characters entered at each endpoint of the IPSec connection

• Authentication– Both endpoints know the same secret, and no

one else has been told– Advantage

• Simplicity• Authentication occurs as long as the preshared

key is typed in correctly on each device– Disadvantage

• Movement of the preshared key when configuring the two devices

18

Certificates

• May be presented for authentication• Useful when clients are from outside of your

organization• Disadvantage of using third-party certificates

– Cost: Each client needs to buy a certificate

– Clients may not be technically savvy enough to obtain certificates

19

Kerberos

• Authentication system used by Windows 2000/XP/2003 for access to network resources

• In Active Directory– Domain is equivalent to a Kerberos realm

• Advantage– Seamless integration with domain security

• Not a commonly supported authentication system for IPSec on non-Microsoft products

• Not appropriate for Windows computers that are not part of the Active Directory forest

20

Enabling IPSec• IPSec Policies

– Used to enable IPSec– Can be configured manually on each server or

distributed through Group Policy– Define the circumstances under which IP traffic is

• Tunneled using IPSec• Permitted without using IPSec• Blocked

– Also define• Type of authentication• Which network connections are affected• Whether IPSec is to be used in tunnel mode or

transport mode

21

Enabling IPSec (Continued)• IPSec policies installed by default

– Server (Request Security)– Client (Respond Only)– Secure Server (Require Security)

• Default policies– Configured to use Kerberos for authentication– Permits ICMP traffic– Respond to requests to use IPSec– Differ in the way they request security

• IPSec Policy– Must be in place to use IPSec

22

Assigning IPSec Policies

• Single server – Can be configured with many IPSec policies

• No Policy is used until assigned• Once policy has been assigned

– Does not take effect immediately– IPSec Policy Agent must be restarted

23

Creating an IPSec Policy

• IPSec policy– Composed of IPSec rules

• IPSec rule composed of– IP filter list

– IPSec filter action

– Authentication methods

– Tunnel endpoint

– Connection type

24

IPSec Policy Rules

25

Creating an IPSec Policy (Continued)

• IP filter lists and IPSec filter actions – Maintained in a central list by Windows Server

2003

– Once created, can be reused by other rules within a policy or other policies

26

Local Security Policy Snap-in

27

Creating an IPSec Policy (Continued)

• To create a new IPSec policy– Use the IP Security Policy Wizard

• IP Security Policy Wizard requests– Name, description– Whether to activate the default response rule – Authentication type

28

Activating the Default Response Rule

29

Authentication Options for the Default Response Rule

30

Creating Rules

• IPSec policy– Once created, must be edited to add rules

– Rules define how different types of IP traffic are handled

– Default Response rule exists by default

• Create IP Security Rule Wizard– Used when adding a rule

– Allows you to configure the most commonly used options

31

Tunnel Endpoint for a New Rule

32

Network Type for a New Rule

33

IP Filter Lists

34

The Filter Action Window• Three default actions

– Permit• Allows packets to pass through the IP filter

unmodified

– Request Security (Optional)• Attempts to create IPSec connections with all other

computers• Uses non-IPSec communication if an SA cannot be

established

– Require Security• Accepts non-IPSec packets• Responds only using IPSec packet

35

Filter Actions

36

IPSec Filter Lists

• If multiple applications running on a server– may be unnecessary for all IP traffic to be

encrypted• Creating IP filter list

– Give it a name and description (optional)– Add IP filters– Specify the traffic to which the list will apply

37

IPSec Filter Lists (Continued)

• IP Filter Wizard– Requests a description for the new IP filter

– Mirrored option applies the IP filter to • Opposite source• Selected destination ports specified in the IP filter

– Second window requests the source IP address in the filter

– Third asks for the destination IP address in the filter

38

Creating an IP Filter List

39

The Mirrored Option for a New IP Filter

40

Source IP Address for a New IP Filter

41

Filter Actions

• Define what is done to traffic that matches an IP filter list

• Default filter actions– Permit– Request Security (Optional)– Require Security

42

Filter Actions (Continued)

• IP Security Filter Action Wizard– Used to create filter actions

– First window requests a name and description

– Second window asks for an action behavior

– Third window asks whether to allow unencrypted communication with computers that do not support IPSec

– Allows you to add only one security method

43

Cryptography Algorithms• IPSec

– Offers both data integrity and encryption• Two algorithms used for AH and ESP data

integrity– Secure Hashing Algorithm (SHA1)

• Produces a 160-bit message digest• Federal Information Processing Standards (FIPS)

specifies this for use in U.S. federal government contracts

– Message Digest 5 (MD5)• Commonly used hashing algorithm for commercial

applications• Produces a 128-bit message digest• Less secure than SHA1, but faster

44

Cryptography Algorithms (Continued)

• Two algorithms used for ESP data encryption– Data Encryption Standard (DES)

• Common encryption algorithm that uses a 56-bit key

• First designated for U.S. federal government use in 1977

45

Cryptography Algorithms (Continued)

– Triple Data Encryption Standard (3DES)• Performs three rounds of encryption using

three different 56-bit keys giving an effective key length of 168-bits

• Windows 2000 computers must have installed the High Encryption Pack or have Service Pack 2 to use 3DES

46

Troubleshooting IPSec

• Common IPSec troubleshooting tools– Ping

– IPSec Security Monitor

– Event Viewer

– Resultant Set of Policy

– Netsh

– Oakley logs

– Network Monitor

47

Troubleshooting IPSec (Continued)

• Ping– Used to test network connectivity between two

hosts– Default IPSec policies

• Permit ICMP packets• Do not interfere with the operation of ping

– Does not test IPSec specifially– Can be used to confirm that two hosts can

communicate

48

Troubleshooting IPSec (Continued)

• IPSec Security Monitor– An MMC snap-in that allows you to view the

status of IPSec SAs

– Can be used to confirm that an SA was negotiated between two hosts

– Can be used to view the configuration of the IPSec policy that is applied

49

Troubleshooting IPSec (Continued)

• Event Viewer– To enable logging

• Set the key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\ Services\IPSec\EnableDiagnostics to a value of 7

• Resultant Set of Policy– Allows you to

• View which policies apply• Simulate the application of new policies to test

their results

50

Troubleshooting IPSec (Continued)

• Netsh– Allows you to configure network-related settings

– Configuration categories include• Bridging, DHCP, diagnostics• IP configuration, remote access, routing• WINS, and remote procedure calls

– Can be used to modify IPSec configuration

51

Troubleshooting IPSec (Continued)

• Oakley Logs– Track the establishment of SAs– Must be enabled with the command “netsh ipsec

dynamic set config ike logging 1”• Network Monitor

– Can be used to view packets that are traveling on the network

– Can identify IPSec traffic– Cannot view encrypted information inside an

IPSec packet– Not useful for troubleshooting application-level

problems if traffic is encrypted

52

Summary

• IPV4– Has no built in security mechanisms– Uses IPSec as an add-on protocol to make

communication• Secure from packet sniffing, data replay• Data modification, and address spoofing

53

Summary (Continued)• IPSec

– Operates at the Network layer

– Not supported by pre-Windows 2000 operating systems

– Cannot be used with NAT

– AH mode does not perform data encryption

– ESP mode has ability to perform data encryption and authentication

– Transport mode is used between two hosts

– Tunnel mode is used between two routers

54

Summary (Continued)• Windows Server 2003 implementation

– Can perform authentication using a preshared key, certificates, or Kerberos

• IPSec policies– Contain rules that control authentication, which

traffic is affected, what is done to the affected traffic

• Filter lists– Used in IPSec rules to define the packets affected

by a rule

• Filter actions– Define what is done to traffic that matches filter list

55

Summary (Continued)• SHA1 and MD5

– Algorithms used for data integrity• DES and 3DES

– Algorithms used for data encryption• Tools used to troubleshoot IPSec

– Ping– IPSec Security Monitor snap-in– Event Viewer– Resultant Set of Policy snap-in– Netsh– Oakley logs– Network Monitor