02 - workbook - teachers edition

Datacenter Solutions Teachers

Version 1.0 Last update: 06/12/2012 Use: Teachers & Students Author: Samuel CUELLA

Workbook

2012-‐2013

Linu

x Techno

logies

Datacenter SolutionsLabs

Test Page ii

Table of Contents1. OpenVPN ................................................................................................................................... 1

1.1. Common configuration ...................................................................................................... 11.2. Bridged VPN .................................................................................................................... 71.3. Routing VPN .................................................................................................................. 11

2. Squid ...................................................................................................................................... 162.1. Preliminary Setup ........................................................................................................... 162.2. Squid as a caching proxy .................................................................................................. 162.3. Squid as a surrogate ........................................................................................................ 19

3. Mail Services ............................................................................................................................ 213.1. Local only mail delivery ................................................................................................... 213.2. Mail domains ................................................................................................................. 233.3. Virtual domains .............................................................................................................. 48

4. iSCSI ....................................................................................................................................... 604.1. Simple iSCSI ................................................................................................................... 604.2. Multipath iSCSI ............................................................................................................... 63

5. DRBD: Distributed Replicated Block Device .................................................................................... 675.1. Two-nodes ..................................................................................................................... 675.2. Three-nodes .................................................................................................................. 74

6. High Availability ........................................................................................................................ 796.1. High available website ..................................................................................................... 796.2. High available iSCSI-based SAN .......................................................................................... 85

7. SELinux .................................................................................................................................... 947.1. Enabling SELinux ............................................................................................................. 947.2. Working with booleans .................................................................................................... 957.3. Using audit2allow ........................................................................................................... 977.4. Creating a policy ............................................................................................................. 99

8. Quality Of Service .................................................................................................................... 1058.1. Bandwidth fairness ........................................................................................................ 1058.2. Traffic Shaping .............................................................................................................. 107

9. IPSec ..................................................................................................................................... 1109.1. Host to host ................................................................................................................. 1109.2. Network to network ...................................................................................................... 120

10. RADIUS ................................................................................................................................ 12310.1. Setting up the RADIUS Server ........................................................................................ 12310.2. Setting up dd-wrt ........................................................................................................ 12610.3. Tests with the client ..................................................................................................... 127

11. Puppet ................................................................................................................................. 12911.1. Preparing the DNS Server ............................................................................................. 12911.2. Working with Puppetmaster .......................................................................................... 129


Test Page 1

1. OpenVPNIn this lab, you'll learn how to configure a VPN to allow clients to connect to a private network from the outside.The network topology will be the following:

For the lab testing purposes, you will have to work with the host-only and NAT vmware networks. Create three(clone) virtual machines and configure them to match (nearly) the topology:

vpn-client will have two network cards (one not shown on the schema). The first network card will be bound tothe host-only network whereas the second network card will be connected to the NAT'ed network. This secondnetwork card will be only used to install the required packages and deconnected/removed on purpose.

vpn-server also needs two network cards: The first one will be connected to the host-only network and the secondone will be connected to the NAT'ed network, which represent the remote private network you want to establisha VPN to.

network-host will figure a random network host in the private network. It only needs one network card, connectedto the NAT'ed network.

NoteThis host is not mandatory and will be used for ping purposes only. If you need to spare resources,you can omit this virtual machine and use the host operating system itself: It has an IP address in thevirtual NAT'ed network too. Find it using your host operating system tools.

The switch and router roles will be actually undertaken by the host operating system and VMware services. Youdon't have to create that part by yourself.

Configure VMware host-only network to be 172.17.2.0/24 and disable the DHCP service on that network too.

1.1. Common configuration

1.1.1. Server

1.1.1.1. Host setup

Q: Set the hostname to vpn-server and configure your host-only interface (first one, eth0 if you've followedthe setup) with the 172.17.2.2 address. Also make sure that your second NIC gets its address via DHCP.

A: The following answer is Debian-specific. Feel free to adapt it if you're using another distro.


Test Page 2

root@debian-master:~# echo "vpn-server" > /etc/hostnameroot@debian-master:~# /etc/init.d/hostname.shroot@debian-master:~# hostname vpn-server

Note

Your shell prompt will not be updated until you log off and in or start a new shell.

Edit /etc/network/interfaces as follows:

[...]allow-hotplug eth0iface eth0 inet static address 172.17.2.2 netmask 255.255.255.0

allow-hotplug eth1iface eth1 inet dhcp

root@vpn-server:/home/supinfo# ifdown eth0 eth1 && ifup eth0 eth1

Q: Install the openvpn package.

A:root@vpn-server:~# apt-get install openvpn

Note

This is Debian-specific. Feel free to adapt to your system if you're using another distro.

1.1.1.2. PKI Setup

In this section, you will use the easy-rsa tool to create a PKI with a certification authority and key/certificatesfor server and clients. OpenVPN ships easy-rsa in the /usr/share/doc/openvpn/examples/easy-rsa/2.0 directory. Create a local copy of this tool to work with in /root/easy-rsa:

root@vpn-server:~# cp -R /usr/share/doc/openvpn/examples/easy-rsa/2.0 easy-rsa

Q: Configure easy-rsa with the following values:

Item Value

Country US


Test Page 3

Item Value

Province UT

City SaltLakeCity

Organization Utopia

Email [email protected]

A: Edit the vars file as follows:

export KEY_COUNTRY="US"export KEY_PROVINCE="UT"export KEY_CITY="SaltLakeCity"export KEY_ORG="Utopia"export KEY_EMAIL="[email protected]"

Q: Build your CA with Utopia CA as the common name. Don't forget to initialize and the environment before.

A:root@vpn-server:~/easy-rsa# . vars NOTE: If you run ./clean-all, I will be doing a rm -rf on /root/easy-rsa/keysroot@vpn-server:~/easy-rsa# ./clean-allroot@vpn-server:~/easy-rsa# ./build-caGenerating a 1024 bit RSA private key.....++++++...............................++++++writing new private key to 'ca.key'-----You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [US]:State or Province Name (full name) [UT]:Locality Name (eg, city) [SaltLakeCity]:Organization Name (eg, company) [Utopia]:Organizational Unit Name (eg, section) []:Common Name (eg, your name or your server's hostname) [Utopia CA]: Name []:Email Address [[email protected]]:

Q: Generate the Diffie-Hellman parameter

A:root@vpn-server:~/easy-rsa# ./build-dh

Q: Create a certificate/key for your server, vpn-server.


Test Page 4

A:root@vpn-server:~/easy-rsa# ./build-key-server vpn-serverGenerating a 1024 bit RSA private key..........++++++.....++++++writing new private key to 'vpn-server.key'-----You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [US]:State or Province Name (full name) [UT]:Locality Name (eg, city) [SaltLakeCity]:Organization Name (eg, company) [Utopia]:Organizational Unit Name (eg, section) []:Common Name (eg, your name or your server's hostname) [vpn-server]:Name []:Email Address [[email protected]]:

Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:An optional company name []:Using configuration from /root/easy-rsa/openssl.cnfCheck that the request matches the signatureSignature okThe Subject's Distinguished Name is as followscountryName :PRINTABLE:'US'stateOrProvinceName :PRINTABLE:'UT'localityName :PRINTABLE:'SaltLakeCity'organizationName :PRINTABLE:'Utopia'commonName :PRINTABLE:'vpn-server'emailAddress :IA5STRING:'[email protected]'Certificate is to be certified until Mar 14 12:47:54 2022 GMT (3650 days)Sign the certificate? [y/n]:y

1 out of 1 certificate requests certified, commit? [y/n]yWrite out database with 1 new entriesData Base Updated

Q: Create a key/certificate for your client, vpn-client.


Test Page 5

A:root@vpn-server:~/easy-rsa# ./build-key vpn-clientGenerating a 1024 bit RSA private key......++++++......++++++writing new private key to 'vpn-client.key'-----You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [US]:State or Province Name (full name) [UT]:Locality Name (eg, city) [SaltLakeCity]:Organization Name (eg, company) [Utopia]:Organizational Unit Name (eg, section) []:Common Name (eg, your name or your server's hostname) [vpn-client]:Name []:Email Address [[email protected]]:

Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:An optional company name []:Using configuration from /root/easy-rsa/openssl.cnfCheck that the request matches the signatureSignature okThe Subject's Distinguished Name is as followscountryName :PRINTABLE:'US'stateOrProvinceName :PRINTABLE:'UT'localityName :PRINTABLE:'SaltLakeCity'organizationName :PRINTABLE:'Utopia'commonName :PRINTABLE:'vpn-client'emailAddress :IA5STRING:'[email protected]'Certificate is to be certified until Mar 14 12:51:47 2022 GMT (3650 days)Sign the certificate? [y/n]:y

1 out of 1 certificate requests certified, commit? [y/n]yWrite out database with 1 new entriesData Base Updated

Q: Copy the required the Diffie-Hellman parameter and all required keys and certificates to /etc/openvpn.

A:root@vpn-server:~/easy-rsa# cp keys/{dh1024.pem,ca.crt,vpn-server.crt,vpn-server.key} /etc/openvpn/

Q: Take a snapshot of your virtual machine. This is not a mandatory step, but it'll help you later.

1.1.2. Client

In this part, you are going to prepare the second virtual machine to act as a vpn client through the host-onlynetwork. However, you need to install the openvpn package and thus to access the internet. To do so, connectthe first network card to the host-only network and add a second one to the virtual machine and connect it to theNAT'ed network. The second card will be shutdown before connecting to the VPN.

1.1.2.1. Host setup


Test Page 6

Q: Set the hostname to vpn-client and configure your host-only interface (first one, eth0 if you've followed thesetup) with the 172.17.2.23 address. Also make sure that your second NIC gets its address via DHCP.

A: The following answer is Debian-specific. Feel free to adapt it if you're using another distro.

root@debian-master:~# echo "vpn-client" > /etc/hostnameroot@debian-master:~# /etc/init.d/hostname.shroot@debian-master:~# hostname vpn-client

Note

Your shell prompt will not be updated until you log off and in or start a new shell.

Edit /etc/network/interfaces as follows:

[...]allow-hotplug eth0iface eth0 inet static address 172.17.2.23 netmask 255.255.255.0

iface eth1 inet dhcp

root@vpn-client:/home/supinfo# ifdown eth0 eth1 && ifup eth0 eth1

Q: Install the openvpn package.

A:root@vpn-client:~# apt-get install openvpn

Note

This is Debian-specific. Feel free to adapt to your system if you're using another distro.

Q: Remove (or unplug) eth1 from your virtual machine.

1.1.2.2. Keys

Q: Use ssh to transfert all needed keys and files from the server to the client. Store these files in /etc/openvpn.

A:root@vpn-client:~# scp [email protected]:/root/easy-rsa/keys/vpn-client.key /etc/openvpn/root@vpn-client:~# scp [email protected]:/root/easy-rsa/keys/vpn-client.crt /etc/openvpn/root@vpn-client:~# scp [email protected]:/root/easy-rsa/keys/ca.crt /etc/openvpn/


Test Page 7

Q: Take a snapshot of your virtual machine. This is not a mandatory step, but it'll help you later.

1.2. Bridged VPN

1.2.1. Server configuration

In this section, you're going to create a bridged VPN. In such a setup, the VPN service will listen on the WANinterface (the 172.17.2.0/24 network in this lab) and bridge incoming traffic with the NAT'ed networtk. The NAT'ednetwork address depends on your environment. Use the ifconfig eth1 command on your server to get its address.In my test environment, the address is 192.168.82.155/24. I'll be using this address all along the following answers.Please adapt to your setup.

Outgoing traffic from the VPN server will go through a special TAP device, that you have to create. The bridgingwill also be done manually to bridge the tap device and the NIC connected to the NAT'ed network. This meansall traffic coming in and out from the VPN will be indistinguishable from the 'real' LAN traffic. Therefore, you caneither choose to let the existing LAN DHCP server distribute network parameters, or have OpenVPN managing itsown IP pool. In a real setup if you choose the later option, be sure to configure the LAN DHCP to exclude theseVPN-reserved IP adresses from its allocation pool.

1.2.1.1. Creating the bridge

Q: Use the openvpn binary to create a tap0 interface.

A:root@vpn-server:~# openvpn --mktun --dev tap0

Q: Create a new br0 bridge

A:root@vpn-server:~# brctl addbr br0

Q: Get current eth1 network settings and write them down.

A:root@vpn-server:~# ifconfig eth1eth1 Link encap:Ethernet HWaddr 00:0c:29:0f:94:c0 inet addr:192.168.82.155 Bcast:192.168.82.255 Mask:255.255.255.0

Q: Configure tap0 and eth1 with the 0.0.0.0 address and put them up and in promiscious mode. If you don'tknow how to put an interface into promiscious mode, search the ifconfig manpage for "prom".

A:root@vpn-server:~# ifconfig tap0 0.0.0.0 up promiscroot@vpn-server:~# ifconfig eth1 0.0.0.0 up promisc

Q: Add these two interfaces to the bridge

A:root@vpn-server:~# brctl addif br0 tap0root@vpn-server:~# brctl addif br0 eth1

Q: Configure the br0 bridge with the network settings previously set on eth1.

A:root@vpn-server:~# ifconfig br0 192.168.82.155 netmask 255.255.255.0


Test Page 8

1.2.1.2. Service configuration

Q: Use the server.conf.gz example config file that ships with OpenVPN to create /etc/openvpn/server.conf.

A:root@vpn-server:~/easy-rsa# zcat \>/usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz > /etc/openvpn/server.conf

Q: Configure OpenVPN to use the keys you've copied previously, and enforce the following requirements:

• Listen on the standard UDP port, only on the interface connected to the 172.17.2.0 network

• Act as a bridged VPN, distributing adresses from .10 to .20 in the NAT'ed network range.

• Push the NAT'ed network default gateway

• Run as nobody/nogroup

A: Edit /etc/openvpn/server.conf as follows:

local 172.17.2.2port 1194proto udp

dev tap0

ca ca.crtcert vpn-server.crtkey vpn-server.keydh dh1024.pem

server-bridge 192.168.82.2 255.255.255.0 192.168.82.10 192.168.82.20push "route 192.168.82.0 255.255.255.0"

Warning

Be sure to use "tap0" and not just "tap" as the tap device. What's the difference? "tap0" is thedevice you've actually created and bridged with eth1. "tap" is a random tap device, there is noguarantee that OpenVPN will use the tap0 you've created.

Q: Start the service.

A:root@vpn-server:/etc/openvpn# /etc/init.d/openvpn startStarting virtual private network daemon: server.

1.2.2. Client configuration


Q: Use the client.conf example file that ships with OpenVPN to create the /etc/openvpn/client.conf file.


Test Page 9

A:root@vpn-client:~# cp /usr/share/doc/openvpn/examples/sample-config-files/client.conf /etc/openvpn/

Q: Edit the service configuration to use the right keys and match your server's configuration. Also run theservice as nobody/nogroup.

A: Edit /etc/openvpn/openvpn.conf as follows:

client

dev tapproto udp

remote 172.17.2.2 1194

user nobodygroup nogroup

ca ca.crtcert vpn-client.crtkey vpn-client.key

Note

On the client side, there is no need to specify a given tapX device. Just use "tap" that will letOpenVPN create the tap device as needed.

Q: Start the service

A:root@vpn-client:~# /etc/init.d/openvpn startStarting virtual private network daemon: client.

1.2.3. Routing

Q: On the client, show your routing table. Try to ping the server (using its NAT'ed network address) and yourhost operating system if you know its address on the NAT'ed network (usually .1).

A:supinfo@vpn-client:~$ /sbin/route Kernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Iface192.168.82.0 * 255.255.255.0 U 0 0 0 tap0172.17.2.0 * 255.255.255.0 U 0 0 0 eth0supinfo@vpn-client:~$ ping -c 1 192.168.82.15564 bytes from 192.168.82.155: icmp_req=1 ttl=64 time=0.841 mssupinfo@vpn-client:~$ ping -c 1 192.168.82.164 bytes from 192.168.82.1: icmp_req=1 ttl=64 time=3.27 ms

Q: Set your DNS server to 8.8.8.8

A:root@vpn-client:~# echo "nameserver 8.8.8.8" > /etc/resolv.conf


Test Page 10

Q: Try to ping google. Does it works? Why?

A:supinfo@vpn-client:~$ ping google.frping: unknown host google.fr

It doesn't work, because the 8.8.8.8 DNS is unreachable. It can't be reached because there is no route toit nor default route in the routing table.

As you've configured the server to push its default route, you could expect that route to be applied on theclient. Moreover, this is the expected behavior. However, OpenVPN seems to have a bug in this part: Theroute is pushed and parsed on the client but not applied. A patch as been sent to the OpenVPN team tofix this.

Q: Fix the problem.

A:root@vpn-client:/home/supinfo# route add default gw 192.168.82.2root@vpn-client:/home/supinfo# ping -c1 google.frPING google.fr (74.125.230.248) 56(84) bytes of data.64 bytes from par08s10-in-f24.1e100.net (74.125.230.248): icmp_req=1 ttl=128 time=22.4 ms

1.2.4. Disabling automatic IP distribution

Q: On the server, edit your configuration to disable IP distribution, and restart the service.

A: Edit /etc/openvpn/server.conf as follows(change server-bridge directive):

#server-bridge 192.168.82.2 255.255.255.0 192.168.82.10 192.168.82.20server-bridge

And restart the service

root@vpn-server:~# /etc/init.d/openvpn restart

Q: Restart the service on the client as well.

A:root@vpn-client:~# /etc/init.d/openvpn restart

Q: On the client, show your routing table. What's missing? Why? Show tap0 details to confirm.

A:root@vpn-client:~# routeKernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Iface172.17.2.0 * 255.255.255.0 U 0 0 0 eth0root@vpn-client:~# ifconfig tap0tap0 Link encap:Ethernet HWaddr 8e:8a:0b:06:67:93 BROADCAST MULTICAST MTU:1500 Metric:1

There is no route to the NAT'ed network you're bridged to, because OpenVPN doesn't give addesses itselfanymore. The situation is exactly the same as if had just plugged a network cable in a switch.

Q: Query the DHCP to get an address on tap0.

A:root@vpn-client:~# dhclient tap0


Test Page 11

Q: Show your routing table and try to ping google.

A:supinfo@vpn-client:~$ /sbin/route Kernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Iface192.168.82.0 * 255.255.255.0 U 0 0 0 tap0172.17.2.0 * 255.255.255.0 U 0 0 0 eth0default 192.168.82.2 0.0.0.0 UG 0 0 0 tap0supinfo@vpn-client:~$ ping -c 1 google.fr64 bytes from fa-in-f94.1e100.net (173.194.70.94): icmp_req=1 ttl=128 time=37.3 ms

1.3. Routing VPN

In this section, you're going to experiment with the second - and most used - OpenVPN operation mode: routing.Before going any further, make sure to reset your configuration. To do so you can either (on both client and server):

• Restore from the snapshot you've taken right after the "Common" section

• Start from fresh virtual machines and redo the "Common" section steps

• Manually undo what you've done in the brigded setup (bridge, routes, configuration, ...)

1.3.1. Server configuration

1.3.1.1. Service setup

In this section, you're going to configure the OpenVPN service to:

• Create a Vitrual Private Network with a dedicated network address

• Accept connections

The purpose of this VPN is to accept connections from "foreign" clients coming from the 172.17.2.0/24 networkand give them access to the VMware NAT'ed network which is connected to your server on the second networkcard. The NAT'ed network is managed by VMware and has a different address on any machine. Use the ifconfigeth1 command to get the NAT'ed network address.

In the test environement where this lab is writtent, the VMware NAT'ed network is 192.168.82.0/24. The VPNalso need a network address of its own. Please adhere to the following convention: If NAT'ed network address isx.y.z.0/24, then the VPN network address will be x.y.z-1.0/24. Having a 192.168.82.0/24 NAT'ed network address,the VPN network address will be 192.168.81.0/24

Q: Use the server.conf.gz example config file that ships with OpenVPN to create /etc/openvpn/server.conf.

A:root@vpn-server:~/easy-rsa# zcat \>/usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz > /etc/openvpn/server.conf

Q: Configure OpenVPN to use the keys you've copied previously, and enforce the following requirements:

• Listen on the standard UDP port, only on the interface connected to the 172.17.2.0 network

• Act as a routing VPN, using a dedicated network address computed as previously defined.

• Push the route to the NAT'ed network to VPN clients

• Run as nobody/nogroup

A: Edit /etc/openvpn/server.conf as follows:


Test Page 12

local 172.17.2.2port 1194proto udp

dev tun

ca ca.crtcert vpn-server.crtkey vpn-server.keydh dh1024.pem

server 192.168.81.0 255.255.255.0push "route 192.168.82.0 255.255.255.0"


A:root@vpn-server:/etc/openvpn# /etc/init.d/openvpn startStarting virtual private network daemon: server.

1.3.2. Client configuration


Q: Use the client.conf example file that ships with OpenVPN to create the /etc/openvpn/client.conf file.

A:root@vpn-client:~# cp /usr/share/doc/openvpn/examples/sample-config-files/client.conf /etc/openvpn/

Q: Edit the service configuration to use the right keys and match your server's configuration. Also run theservice as nobody/nogroup.

A: Edit /etc/openvpn/openvpn.conf as follows:

clientdev tunproto udp

remote 172.17.2.2 1194

user nobodygroup nogroup

ca ca.crtcert vpn-client.crtkey vpn-client.key

Q: Start the service

A:root@vpn-client:~# /etc/init.d/openvpn startStarting virtual private network daemon: client.


Test Page 13

1.3.2.2. Tests

Q: Check if a new network interface has been created.

A:root@vpn-client:~# ifconfigtun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:192.168.81.6 P-t-P:192.168.81.5 Mask:255.255.255.255 UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:20 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:0 (0.0 B) TX bytes:2910 (2.8 KiB)

Q: Show the current routing table (no dns resolution).

A:root@vpn-client:~# route -nKernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Iface192.168.81.5 0.0.0.0 255.255.255.255 UH 0 0 0 tun0192.168.81.1 192.168.81.5 255.255.255.255 UGH 0 0 0 tun0192.168.82.0 192.168.81.5 255.255.255.0 UG 0 0 0 tun0172.17.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

Q: Try to ping the vpn server on its 3 ip addresses:

• Host-only network (172.17.2.2)

• VPN (first address on the subnet: in this example 192.168.81.1)

• NAT'ed address (read it on the server)

A:root@vpn-client:~# ping -c 1 172.17.2.2PING 172.17.2.2 (172.17.2.2) 56(84) bytes of data.64 bytes from 172.17.2.2: icmp_req=1 ttl=64 time=0.287 ms

--- 172.17.2.2 ping statistics ---1 packets transmitted, 1 received, 0% packet loss, time 0msrtt min/avg/max/mdev = 0.287/0.287/0.287/0.000 msroot@vpn-client:~# ping -c 1 192.168.81.1PING 192.168.81.1 (192.168.81.1) 56(84) bytes of data.64 bytes from 192.168.81.1: icmp_req=1 ttl=64 time=0.635 ms

--- 192.168.81.1 ping statistics ---1 packets transmitted, 1 received, 0% packet loss, time 0msrtt min/avg/max/mdev = 0.635/0.635/0.635/0.000 msroot@vpn-client:~# ping -c 1 192.168.82.155PING 192.168.82.155 (192.168.82.155) 56(84) bytes of data.64 bytes from 192.168.82.155: icmp_req=1 ttl=64 time=0.608 ms

--- 192.168.82.155 ping statistics ---1 packets transmitted, 1 received, 0% packet loss, time 0msrtt min/avg/max/mdev = 0.608/0.608/0.608/0.000 ms

Q: Boot the third virtual machine while making sure that the virtual network card is connected to the NAT'ednetwork. Get it's IP address and try to ping it from the vpn client. Does it works? Why?


Test Page 14

A:root@vpn-client:~# ping -c 1 192.168.82.148PING 192.168.82.148 (192.168.82.148) 56(84) bytes of data.

--- 192.168.82.148 ping statistics ---1 packets transmitted, 0 received, 100% packet loss, time 0ms

It doesn't work, because the VPN server hadn't been configured to route packets from a network to another.

1.3.3. Routing

Q: Configure the server to route packets between the VPN (172.17.2.0/24) and the VMware NAT'ed network.

A:root@vpn-server:~# echo "1" > /proc/sys/net/ipv4/ip_forward

Note

You can also use sysctl. Please also note that this setting will be restored to default (0) uponreboot. To avoid that and keep routing enabled, you can edit /etc/sysctl.conf.

Q: Try to ping the third virtual machine again from the vpn-client. Does it works? Why?

A:supinfo@vpn-client:~$ ping -c 1 192.168.82.148PING 192.168.82.148 (192.168.82.148) 56(84) bytes of data.

--- 192.168.82.148 ping statistics ---1 packets transmitted, 0 received, 100% packet loss, time 0ms

It doesn't work, because of the network topology: The vpn client as a route to the 192.168.82.148 thirdvirtual machine through the VPN tunnel, but the 192.168.82.148 third virtual machine as no route torespond back to the 192.168.81.6 vpn client. Therefore the third virtual machine will send the ping responseto its default gateway (your host operating system) which doesn't have a route to 192.168.81.0/24 either.To make the VPN network and the LAN communicate, the LAN default gateway must be aware of the VPNand have a route to that VPN. In this case, you could configure your host operating system to route thepackets correctly. However, there is a quick fix alternative: Manually adding a route from the third virtualmachine to the VPN.

Q: Implement a quick fix for the routing problem by adding a route from the third virtual machine to the VPN.

A:root@debian-master:~# route add -net 192.168.81.0 \>netmask 255.255.255.0 gw 192.168.82.155

Q: Try to ping the third virtual machine again from the vpn-client.

A:supinfo@vpn-client:~$ ping 192.168.82.148PING 192.168.82.148 (192.168.82.148) 56(84) bytes of data.64 bytes from 192.168.82.148: icmp_req=1 ttl=63 time=6.44 ms64 bytes from 192.168.82.148: icmp_req=2 ttl=63 time=1.26 ms


Test Page 15

1.3.4. Internet access through the VPN

Q: Configure the vpn client to use the Google DNS (8.8.8.8) as its main (and only) DNS server.

A:root@vpn-client:~# echo "nameserver 8.8.8.8" > /etc/resolv.conf

Q: Configure the VPN server to NAT outgoing VPN traffic.

A:root@vpn-server:~# iptables -t nat -A POSTROUTING \>-s 192.168.81.0/24 -o eth1 -j MASQUERADE

Note

As eth1 ip address is configured dynamically by a DHCP I'm using the MASQUERADE target.However a SNAT target is perfectly working & valid.

Note

As a side effet, all traffic coming from the VPN is now going to be NAT'ed regardless of itsdestination (to the internet or within the VMware virtual NAT'ed network). This means thatnow when a VPN client will try to communicate with another host located on the VMwareNAT'ed virtual network, it will do it through the VPN server that will NAT it's query. Thenetwork host will not know about the VPN client: From the network host perspective, it iscommunicating with the VPN server itself. Therefore, there is no need for additional routinginformation between the VPN and the virtual network.

Q: Configure the VPN client to use the tunnel as its default route.

A:root@vpn-client:~# route add default tun0

Q: Try to ping Google.

A:supinfo@vpn-client:~$ ping google.comPING google.com (74.125.230.228) 56(84) bytes of data.64 bytes from par08s10-in-f4.1e100.net (74.125.230.228): icmp_req=1 ttl=127 time=23.4 ms


Test Page 16

2. SquidIn this chapter, you're going to configure a squid server to act as a caching proxy and also as an accelerator. You'llneed a virtual machine to run squid. This virtual machine will be connected to the NAT'ed virtual network. Theclient will be your host operating system browser.

2.1. Preliminary Setup

Q: Set your machine hostname to squid-server.

A:root@debian-master:~# echo "squid-server" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh

Q: Install the following packages:

• squid

• squidclient

Note

This is Debian-specific. If you're using another distro, you may have to install more or lesspackages. Refer to your documentation.

A:root@squid-server:~# apt-get install squid squidclient

Q: Create a backup copy of /etc/squid/squid.conf and empty this file.

A:root@squid-server:~# cp /etc/squid/squid.conf .root@squid-server:~# cat /dev/null > /etc/squid/squid.conf

2.2. Squid as a caching proxy

2.2.1. Base configuration

Q: Configure squid to use the following settings:

• Listen on port 3128 on all interfaces

• Store a 4GB cache using ufs in /var/cache/squid. Organize it using 16 top-level directories and 256second level directories.

• Enable cache_object protocol for localhost (127.0.0.1/32) only.

• Enable ports 80 and 443


Test Page 17

• Only tunnel HTTPS traffic

• Only accept requests from the NAT'ed virtual network.

Proxy runs as proxy/proxy. Don't forget to set permissions on /var/cache/squid accordingly.

A: Create directories as needed:

root@squid-server:~# mkdir /var/cache/squidroot@squid-server:~# chown proxy:proxy /var/cache/squid/

Edit your configuration as follows:

http_port 3128

cache_dir ufs /var/cache/squid 4096 16 256

acl all src 0/0acl cacheProto proto cache_objectacl localhost src 127.0.0.1/32acl authorizedPorts port 80 443acl SSLPort port 443acl connect method CONNECTacl myNetwork src 192.168.82.0/24

http_access allow cacheProto localhosthttp_access deny cacheProtohttp_access deny !authorizedPortshttp_access deny connect !SSLPorthttp_access allow myNetwork

http_access deny all

Q: Test your configuration and (re)start the service

A:root@squid-server:~# squid -k parseroot@squid-server:~# /etc/init.d/squid restart

Q: Edit your host operating system browser settings to use your squid virtual machine as a proxy. You shouldbe able to browse the internet.

2.2.2. Restrictions

Q: Prevent users from browsing linuxfr.org and slashdot.com during work days (Monday to Friday) between9:00 and 18:00.

A: Edit your configuration file as follows (and restart the service):


Test Page 18

[...]acl myNetwork src 192.168.82.0/24

acl worktime time MTWHF 09:00-18:00acl timewaste dstdomain .linuxfr.org .slashdot.org

http_access allow cacheProto localhosthttp_access deny cacheProtohttp_access deny !authorizedPortshttp_access deny connect !SSLPorthttp_access deny !myNetwork

http_access allow timewaste !worktimehttp_access deny timewaste

http_access allow myNetworkhttp_access deny all

Q: You need to block more websites, like bash.org and xkcd.com. Configure squid to read domains to blockfrom /etc/squid/bad-domains (create that file and fill it with blocked domains).

A: Edit your configuration file to:

acl timewaste dstdomain "/etc/squid/bad-domains"

And fill /etc/squid/bad-domains with websites to block:

.linuxfr.org

.slashdot.org

.bash.org

.xkcd.com

Q: Add an exception ACL to allow these otherwise blocked websites on friday afternoons (14h-18h).

A: Edit your configuration as follows:

acl funkytime time F 14:00-18:00

http_access allow timewaste !worktimehttp_access allow timewaste funkytimehttp_access deny timewaste

2.2.3. Authentication

Q: Configure Squid to require users authentication through PAM. Users shouldn't have to enter their credentialsmore than once during their typical work day (8 hours).

A: Edit the configuration as follows(and restart the service):


Test Page 19

acl timewaste dstdomain "/etc/squid/bad-domains"

auth_param basic program /usr/lib/squid/pam_authauth_param basic realm "Squid Authentication"auth_param basic credentialsttl 8 hour

acl knownUsers proxy_auth REQUIRED

http_access deny !knownUsers

http_access allow cacheProto localhost

Q: What's the main issue with such a setup?

A: Security: System passwords are sent unencrypted through the HTTP communication.

Q: Configure Squid to use a NCSA-like password file instead of PAM. Create the /etc/squid/users.passwd usinghtpasswd(from the apache2 package) and fill it with some users in it for your tests.

A: Create the password file:

root@squid-server:~# htpasswd -c /etc/squid/users.passwd supinfoNew password: Re-type new password: Adding password for user supinforoot@squid-server:~# htpasswd /etc/squid/users.passwd rick

Configure and restart squid:

auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/users.passwd

2.3. Squid as a surrogate

In this section, you're going to make squid accelerate a single webserver. Clients will connect to squid thinkingthey are talking to the real webserver. In turn, Squid will either use its cache or query the real webserver to answerthe client's request.

Before going any further, be sure to stop any running instances of Squid and to empty the configuration you'vedone so far. Start with an empty /etc/squid/squid.conf file.

Remove the proxy configuration you've done before from the web browser you've been using for tests. On yourhost operating system, create a static resolution entry to make it resolve www.supinfo.com to your virtual machineIP address.

Q: Configure Squid to meet the following requirements:

• Listen on port 80

• Forward all requests to www.supinfo.com, regardless of supplied URI or Host header.

• Use ACL's to prevent squid from talking to other servers than www.supinfo.com.

A: Edit /etc/squid/squid.conf as follows(and restart the service):


Test Page 20

http_port 80 accel defaultsite=www.supinfo.com

cache_peer www.supinfo.com parent 80 0 no-query originserver name=accelPeer

acl all src 0/0acl accelerated dstdomain www.supinfo.comhttp_access allow acceleratedhttp_access deny all:q

cache_peer_access accelPeer allow acceleratedcache_peer_access accelPeer deny all

Q: Test your setup from your host operating system (Be sure to resolve www.supinfo.com to the virtualmachine).

A:samuel@chickamauga ~ $ ping www.supinfo.comPING www.supinfo.com (192.168.82.156) 56(84) bytes of data.64 bytes from www.supinfo.com (192.168.82.156): icmp_req=1 ttl=64 time=0.354 ms64 bytes from www.supinfo.com (192.168.82.156): icmp_req=2 ttl=64 time=0.283 mssamuel@chickamauga ~ $ curl http://www.supinfo.com/<script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script><script type="text/javascript">[...]


Test Page 21

3. Mail Services3.1. Local only mail delivery

In this section, you're going to configure postfix to deliver mail directly to the local users. You'll need a stand alonevirtual machine for this setup.

Before going any further, make sure postfix is installed on your system (postfix package on Debian-based systems)and that no other smtp daemon is running on port 25.

Q: Set the hostname to mangus. Ensure this setting will presist accross reboots.

A:root@debian-master:~# echo "mangus" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh

Q: Create the following users, make sure their hone directory is also created on the way. Give them randompasswords as needed.

• bob

• sophie

• sarah

• john

If your distro doesn't set bash as new users default shell, you might want to ensure this using -s switch(or enjoy sh fancy features).

A:root@magnus:~# useradd -m bob -s /bin/bashroot@magnus:~# useradd -m sophie -s /bin/bashroot@magnus:~# useradd -m sarah -s /bin/bashroot@magnus:~# useradd -m john -s /bin/bash

Q: Configure postifx to deliver local mail only. It should allow bob to mail sophie from their respective terminals.Postfix should not listen on the network, only on the loopback interface.

A: The /etc/postfix/main.cf should look like this:

smtpd_banner = $myhostname ESMTP $mail_namebiff = no

# appending .domain is the MUA's job.append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings#delay_warning_time = 4h

inet_interfaces = loopback-onlymynetworks_style = hostdefault_transport = error: Local delivery only!

alias_maps = hash:/etc/aliases

mydestination = magnus, localhost, localhost.localdomain, magnus.localdomain


Test Page 22

Q: As supinfo, try to send john a mail.

A:supinfo@magnus:~$ mail john@localhostSubject: Hi BuddyThis is a sample mail .Cc:

Q: As john, check your mailbox to see if something has arrived for you.

A:john@mangus:~$ mailMail version 8.1.2 01/15/2001. Type ? for help."/var/mail/john": 1 message 1 new>N 1 [email protected] Thu May 24 12:48 14/482 Hi Buddy& t 1Message 1:From [email protected] Thu May 24 12:48:38 2012X-Original-To: john@localhostTo: john@localhostSubject: Hi BuddyDate: Thu, 24 May 2012 12:48:37 +0200 (CEST)From: [email protected] (supinfo)

This is a sample mail

&

Or alternatively:

john@mangus:~$ cat /var/mail/john From [email protected] Thu May 24 12:48:38 2012Return-Path: <[email protected]>X-Original-To: john@localhostDelivered-To: john@localhostReceived: by magnus.localdomain (Postfix, from userid 1000) id B0C6C53041; Thu, 24 May 2012 12:54:49 +0200 (CEST)To: john@localhostSubject: Hi BuddyMessage-Id: <[email protected]>Date: Thu, 24 May 2012 12:48:38 +0200 (CEST)From: [email protected] (supinfo)

This is a sample mail

3.1.1. Aliases

Q: Add a new alias that resolves bobby to bob@localhost.

A: Add the following to /etc/aliases:

bobby: bob@localhost


Test Page 23

Q: Rebuild the alias table

A:root@magnus:~# newaliases

Q: Try to send a mail to bobby@localhost.

A:root@magnus:~# mail bobby@localhostSubject: Test.Cc: Null message body; hope that's ok

Q: Check your mails as bob

A:bob@mangus:~$ mailMail version 8.1.2 01/15/2001. Type ? for help."/var/mail/bob": 1 message 1 new>N 1 root@mangus Thu May 24 16:42 13/384 Test& t 1Message 1:From root@mangus Thu May 24 16:42:39 2012X-Original-To: bobby@localhostTo: bobby@localhostSubject: TestDate: Thu, 24 May 2012 16:42:39 +0200 (CEST)From: root@mangus (root)

3.2. Mail domainsIn this section, you'll setup a mail server that receive (and sends) mail for a given domain. You'll need two virtualmachines:

• Mail Server + DNS

• Workstation

You can keep the "magnus" host you've setup in the previous part as the first virtual machine.

3.2.1. Setting up DNS

Q: Install a DNS server on magnus and create a utopia.net zone with the following records:

• A record the resolves to magnus

• MX record that annonces magnus as the mail host of utopia.net

• Three smtp, imap and pop3 CNAME records resolving to magnus.

A: Add a zone declaration in Bind configuration:

zone "utopia.net" { type master; file "/etc/bind/db.utopia.net";};


Test Page 24

And fill the zone file:

@ IN SOA utopia.net. root.utopia.net. ( 20120524 ; Serial 604800 ; Refresh 86400 ; Retry 2419200 ; Expire 604800 ) ; Negative Cache TTL;@ IN NS ns@ IN MX 10 mail.utopia.net.

ns IN A 192.168.82.156mail IN A 192.168.82.156magnus IN A 192.168.82.156imap IN CNAME magnuspop3 IN CNAME magnussmtp IN CNAME magnus

Q: On the second virtual machine, set the first virtual machine as the DNS.

A:root@debian-master:~# echo "nameserver 192.168.82.156" > /etc/resolv.conf

3.2.2. Mail server configuration

Q: Create a "Maildir" directory in each of your users' home directory. This directory should belong to the user.Either create it as $USER or change ownership afterwards.

A:root@mangus:~# for u in bob sophie sarah john; do mkdir "/home/$u/Maildir"; \>chown $u:$u "/home/$u/Maildir"; done;

Q: Configure mangus to accept mail for utopia.net. Mail should be stored in the maildir format, in each user's$HOME/Maildir.

A: Edit /etc/postfix/main.cf as follows:


Test Page 25

mydomain = utopia.net

smtpd_banner = $myhostname ESMTP $mail_namebiff = no


# Uncomment the next line to generate "delayed mail" warnings#delay_warning_time = 4h

mydestination = magnus localhost $mydomain

inet_interfaces = allmynetworks_style = host

alias_maps = hash:/etc/aliaseshome_mailbox = Maildir/

Q: Enforce your modifications

A:root@magnus:~# postfix reload

Or alternatively:

root@magnus:~# /etc/init.d/postfix restart

Q: Install a maildir-aware mail client such as nail or heirloom-mailx on Debian-based systems.

A:root@mangus:~# apt-get install heirloom-mailx

Note

This is Debian-specific.

Q: As supinfo, point the mail environment variable to your maildir.

A:supinfo@mangus:~$ export MAIL=/home/supinfo/Maildir

3.2.3. Configuring a client

3.2.3.1. Network setup

Q: You don't want anyone to modify /etc/resolv.conf: Kill any running dhcp client.


Test Page 26

A:root@debian-master:~# pkill -9 dhc

Q: Configure the resolver to use the first virtual machine as the main (and single) DNS server.

A:root@debian-master:~# echo "nameserver 192.168.82.56" > /etc/resolv.conf

3.2.3.2. Configuring the local smtpd

When sending mail using the mail command, you need a local smtpd. Even if you don't want to receive anythingand only send outgoing messages, you'll need that daemon. The Debian default install provides a running exim4smtpd, but configured for local delivry only. You need to reconfigure it to be able to send mail to remote (not thelocal machine) domains.

If you're using a Debian-based system, use the following command to enable external mail delivery:

root@debian-master:~# dpkg-reconfigure exim4-config

Just select internet site and accept all default values.

The default configuration also prevents exim4 from sending mail to local addresses. Execute the followingcommands to remove this limitation:

root@debian-master:~# sed -i 36,38d /etc/exim4/conf.d/router/200_exim4-config_primaryroot@debian-master:~# sed -i 1040,1042d /etc/exim4/exim4.conf.templateroot@debian-master:~# /etc/init.d/exim4 restart

3.2.3.3. Sending mail to utopia.net

Q: Use the mail command to send a random mail to [email protected].

A:root@debian-master:~# mail [email protected]: Hi from WorkstationEverything is up and running.EOT

Q: Check your mails as supinfo (on the server)


Test Page 27

A:supinfo@mangus:~$ mailHeirloom mailx version 12.4 7/29/08. Type ? for help."/home/supinfo/Maildir": 1 message 1 new>N 1 supinfo Tue May 29 17:51 22/875 Hi from Workstation? t 1Message 1:From [email protected] Tue May 29 17:51:29 2012Return-Path: <[email protected]>X-Original-To: [email protected]: [email protected]: Tue, 29 May 2012 17:51:37 +0200To: [email protected]: Hi from WorkstationUser-Agent: Heirloom mailx 12.4 7/29/08Content-Type: text/plain; charset=us-asciiFrom: supinfo <[email protected]>Status: R

Everything is up and running

3.2.4. Remote access

3.2.4.1. IMAP

Q: Install courier-imap on the server.

A:root@mangus:~# apt-get install courier-imap

Note

This is Debian-specific

Q: If you're running Debian, also install the libgamin0 package to avoid error messages from the imap server.

A:root@mangus:~# apt-get install libgamin0

Q: Make sure courier-imap is running.

A:root@mangus:~# /etc/init.d/courier-imap restart

Q: Use telnet to test IMAP access from a virtual machine or directly from the host operating system: Justconnect to the IMAP server (on port 143) and type-in IMAP commands.


Test Page 28

A:samuel@chickamauga ~ $ telnet 192.168.82.156 143Trying 192.168.82.156...Connected to 192.168.82.156.Escape character is '^]'.* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2010 Double Precision, Inc. See COPYING for distribution information.01 LOGIN supinfo supinfo* OK [ALERT] Filesystem notification initialization error -- contact your mail administrator (check for configuration errors with the FAM/Gamin library)01 OK LOGIN Ok.02 LIST "" ** LIST (\Unmarked \HasNoChildren) "." "INBOX"02 OK LIST completed03 SELECT INBOX* FLAGS (\Draft \Answered \Flagged \Deleted \Seen \Recent)* OK [PERMANENTFLAGS (\* \Draft \Answered \Flagged \Deleted \Seen)] Limited* 1 EXISTS* 1 RECENT* OK [UIDVALIDITY 1338302965] Ok* OK [MYRIGHTS "acdilrsw"] ACL03 OK [READ-WRITE] Ok04 FETCH 1 ALL* 1 FETCH (FLAGS (\Seen \Recent) INTERNALDATE "29-May-2012 17:51:29 +0200" RFC822.SIZE 895 ENVELOPE ("Tue, 29 May 2012 17:51:37 +0200" "Hi from Workstation" (("supinfo" NIL "supinfo" "debian-master.localdomain")) (("supinfo" NIL "supinfo" "debian-master.localdomain")) (("supinfo" NIL "supinfo" "debian-master.localdomain")) ((NIL NIL "supinfo" "utopia.net")) NIL NIL NIL "<[email protected]>"))04 OK FETCH completed.

3.2.4.2. POP3

Q: Install courier-pop on the server.

A:root@mangus:~# apt-get install courier-pop

Q: Make sure the pop3 service is running.

A:root@mangus:~# /etc/init.d/courier-pop restart

Q: Use telnet (or any interactive multi-purpose clilent such as netcat) to test the pop3 service running on theserver.


Test Page 29

A:supinfo@mangus:~$ telnet localhost pop3Trying 127.0.0.1...Connected to localhost.Escape character is '^]'.+OK Hello there.USER supinfo+OK Password required.PASS supinfo+OK logged in.LIST+OK POP3 clients that break here, they violate STD53.1 895.RETR 1+OK 895 octets follow.Return-Path: <[email protected]>X-Original-To: [email protected]: [email protected]: from debian-master.localdomain (unknown [192.168.82.142]) by mangus.utopia.net (Postfix) with ESMTP id 313BF530A8 for <[email protected]>; Tue, 29 May 2012 17:51:29 +0200 (CEST)Received: from supinfo by debian-master.localdomain with local (Exim 4.72) (envelope-from <[email protected]>) id 1SZOht-0003PK-Kl for [email protected]; Tue, 29 May 2012 17:51:37 +0200Message-Id: <[email protected]>Date: Tue, 29 May 2012 17:51:37 +0200To: [email protected]: Hi from WorkstationUser-Agent: Heirloom mailx 12.4 7/29/08MIME-Version: 1.0Content-Type: text/plain; charset=us-asciiContent-Transfer-Encoding: 7bitFrom: supinfo <[email protected]>

Everything is up and running.^]

telnet> quitConnection closed.

3.2.4.3. SMTP Authentication

With the current mail server configuration, the mail server will accept mail for any user in the utopia.net domain.However, it will only relay mail from users on the local machine. It means that if you want to send a mail [email protected], you need to be "physically" logged on the server. The server will refuse to forward mail is theclient isn't connected from the same machine. You can try this out from your second virtual machine:


Test Page 30

supinfo@debian-master:~$ telnet 192.168.82.156 smtpTrying 192.168.82.156...Connected to 192.168.82.156.Escape character is '^]'.220 mangus.utopia.net ESMTP PostfixHELO workstation250 mangus.utopia.netMAIL FROM: [email protected] 2.1.0 OkRCPT TO: [email protected] 5.7.1 <[email protected]>: Relay access denied^]

telnet> quitConnection closed.

This is the expected and desired behavior.

If you want to allow "remote" (not logged on the server) users to send mail to other domains, you can:

• Accept to relay mail from a subset of network hosts

• Use user-based authentication

Of course the second solution is far more flexible than the first one which is not really user-aware as it only dealswith IP addresses. The first solution is implemented through the mynetworks directive.

In this section, you'll setup a user-based authentication using cyrus-sasl.

Q: Install cyrus-sasl packages. On a standard Debian install you only need to install sasl2-bin

A:root@mangus:/home/supinfo# apt-get install sasl2-bin

Q: Configure Cyrus sasl to use the saslauthd deamon as the password checking method for the smtpdapplication.

A:root@mangus:~# echo "pwcheck_method: saslauthd" > /etc/postfix/sasl/smtpd.conf

NoteThis is Debian-specific. Debian's postfix has been patched to have this configuration file under /etc/postfix. Vanilla versions of Cyrus SASL search for application config files under the /etc/sasl2/ directory.

Q: Now you need to set the backend that cyrus-sasl's saslauthd is going to check credentials against. There aremany popular options, such as pam or shadow. There is another interesting option you can use as a backend:rimap. This backend takes user-supplied credentials and try them against a given imap server. If the imapserver grants the connection, the backend does so. In other words, the SMTP authentication is tied to theIMAP authentication mechanism. It provides a reliable way to manage authentication in only one place.

Setup cyrus-sasl to use the rimap backend and to authenticate against the IMAP server you've configuredearlier. You'll find all necessary information in the saslauthd man page.


Test Page 31

A: On a Debian system, modify the /etc/default/saslauthd file as follows:

# Should saslauthd run automatically on startup? (default: no)START=yes[...]# Example: MECHANISMS="pam"MECHANISMS="rimap"

# Additional options for this mechanism. (default: none)# See the saslauthd man page for information about mech-specific options.MECH_OPTIONS="127.0.0.1"[...]

Q: Postfix and saslauthd will communicate through a unix-domain socket. This socket defaultly lives in the /var/run/saslauthd directory. However if postfix runs in a chrooted environement, it won't find thisdirectory. Make sure postfix is running and find out if it's running chrooted.

A: First, check if the smtp daemon is configured to run chrooted:

root@mangus:~# cat /etc/postfix/master.cf | grep ^smtpsmtp inet n - - - - smtpd

The fifth field controles whether the daemon will run chrooted or not. It shows a dash. It means to usethe default that is 'yes', therefore postfix runs chrooted in the queue_directory directory. Get thequeue_directory using postconf:

root@mangus:~# postconf | grep queue_directoryqueue_directory = /var/spool/postfix

The smtpd deamon runs chrooted in the /var/spool/postfix directory.

Q: If postfix runs chrooted, create a var/run/saslauthd directory within the chroot. Ensure that thesasl group is the group owner of this directory.

A:root@mangus:~# mkdir -p /var/spool/postfix/var/run/saslauthdroot@mangus:~# chgrp sasl /var/spool/postfix/var/run/saslauthd

Q: The postfix daemons, which runs as postfix needs to go through this directory to open the socket. Makethe postfix user a member of the sasl group.

A:root@mangus:~# gpasswd -a postfix sasl

Q: Configure postfix to:

• Enable SMTP authentication

• Support pre-RFC or not RFC-abiding clients

• Allow mail relay from authenticated clients

A: Add the following directives to /etc/postfix/main.cf:


Test Page 32

smtpd_sasl_auth_enable = yesbroken_sasl_auth_clients = yes smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination

Q: Restart postfix and saslauthd services.

A:root@mangus:~# /etc/init.d/postfix restartroot@mangus:/home/supinfo# /etc/init.d/saslauthd restart

Q: Try to telnet to your mail server (from either the workstation or the server). Say Hi with a EHLO. What doyou notice?

A:supinfo@mangus:~$ telnet localhost smtpTrying 127.0.0.1...Connected to localhost.Escape character is '^]'.220 mangus.utopia.net ESMTP PostfixEHLO workstation250-mangus.utopia.net[...]250-AUTH LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM250-AUTH=LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM

The server now announces that it supports authentication, and lists supported methods. It also announcesit in both syntaxes, RFC and pre-RFC (equal sign).

Q: Use telnet from the workstation to test the authentication process. You'll need to issue a AUTH command.This command takes a base64 encoded string holding the login and password in the following format:

\0login\0password

Use the base64 command to generate the encoded string.

If you're on a network with restrictive policies on SMTP your mail might not reach the destination server.However the "Authentication successful" message will demonstrate that the feature is working and the mailbeen sent.


Test Page 33

A:supinfo@debian-master:~$ printf "\0supinfo\0supinfo" | base64AHN1cGluZm8Ac3VwaW5mbw==supinfo@debian-master:~$ telnet 192.168.82.156 smtpTrying 192.168.82.156...Connected to 192.168.82.156.Escape character is '^]'.220 mangus.utopia.net ESMTP PostfixEHLO workstation250-mangus.utopia.net250-PIPELINING250-SIZE 10240000250-VRFY250-ETRN250-AUTH LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM250-AUTH=LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM250-ENHANCEDSTATUSCODES250-8BITMIME250 DSNAUTH PLAIN AHN1cGluZm8Ac3VwaW5mbw==235 2.7.0 Authentication successfulMAIL FROM: [email protected] 2.1.0 OkRCPT TO: [email protected] 2.1.5 OkDATA354 End data with <CR><LF>.<CR><LF>Subject: Test mailAuthenticated .250 2.0.0 Ok: queued as 22603530DA^]

telnet> quit

3.2.5. Security

In the previous section, you've setup SMTP authentication to only allow authenticated remote users to have theirmail relayed to other domains. This is mandatory if you don't want your server to be an open relay for spammers.However the authentication method sends clear-text passwords over the network: Base64 is an encoding system,not a cipher algorithm. In this section, you'll enable SSL/TLS support in postfix to provide secure SMTPS, IMAPS(respectively SMTP and IMAP over SSL) and POP3 over SSL communication channels.

3.2.5.1. Creating certificates

The first step is to create certificates. To create certificates, you need a certifiacation authority. in this section,you're going to create a new certificate authority and self-signed certificates. You can either use the CA.pl scriptthat comes with openssl or directly the openssl command as you did in the "OpenSSL: Keys and certificates"chapter of the Edge Computing course. However the solution of all questions will use the warpper script only.Stick with what you're used to use.

Q: Use the CA.pl script to create a new certification authority. If you don't know how to use CA.pl, issue a manCA.pl statement. If CA.pl is not in the PATH, use your distro's package management system to locate it.


Test Page 34

A:root@mangus:~# /usr/lib/ssl/misc/CA.pl -newcaCA certificate filename (or enter to create)

Making CA certificate ...Generating a 1024 bit RSA private key.......................................................++++++........++++++writing new private key to './demoCA/private/cakey.pem'Enter PEM pass phrase:Verifying - Enter PEM pass phrase:-----You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [AU]:USState or Province Name (full name) [Some-State]:UtahLocality Name (eg, city) []:ProvoOrganization Name (eg, company) [Internet Widgits Pty Ltd]:UtopiaOrganizational Unit Name (eg, section) []:Common Name (eg, YOUR name) []:ca.utopia.netEmail Address []:

Q: Create a certificate and key for smtp.utopia.net and sign it using the CA. Don't protect the key with apassword. Remember that the only really important field is the CN: It should match the server's hostname,here smtp.utopia.net.


Test Page 35

A:root@mangus:~# /usr/lib/ssl/misc/CA.pl -newreq-nodesGenerating a 1024 bit RSA private key....................++++++.++++++writing new private key to 'newkey.pem'-----You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [AU]:USState or Province Name (full name) [Some-State]:UtahLocality Name (eg, city) []:ProvoOrganization Name (eg, company) [Internet Widgits Pty Ltd]:UtopiaOrganizational Unit Name (eg, section) []:Common Name (eg, YOUR name) []:smtp.utopia.netEmail Address []:

Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:An optional company name []:Request is in newreq.pem, private key is in newkey.pemroot@mangus:~# /usr/lib/ssl/misc/CA.pl -signreqUsing configuration from /usr/lib/ssl/openssl.cnfEnter pass phrase for ./demoCA/private/cakey.pem:Check that the request matches the signatureSignature okCertificate Details: Serial Number: e1:d8:25:12:be:99:bc:ab Validity Not Before: May 31 15:09:17 2012 GMT Not After : May 31 15:09:17 2013 GMT Subject: countryName = US stateOrProvinceName = Utah localityName = Provo organizationName = Utopia commonName = smtp.utopia.net X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: B0:86:C3:6F:B5:28:C8:AF:E9:BD:2A:08:59:76:99:6B:B4:DB:67:1B X509v3 Authority Key Identifier: keyid:01:A9:62:BC:2B:5C:A9:8D:7C:80:9C:55:B9:1B:1C:B5:2E:92:27:16

Certificate is to be certified until May 31 15:09:17 2013 GMT (365 days)Sign the certificate? [y/n]:y

1 out of 1 certificate requests certified, commit? [y/n]yWrite out database with 1 new entriesData Base UpdatedSigned certificate is in newcert.pem

Q: Keep the newly created files newkey.pem and newcert.pem as smtpd-key.pem and smtpd-cert.pem in your home directory (user's or root's, prefeably the latter). You'll use these files later.


Test Page 36

A:root@mangus:~# mv newkey.pem smtpd-key.pemroot@mangus:~# mv newcert.pem smtpd-cert.pem

Q: Go through the same steps to create a certificate (and key) for imap.utopia.net. Rename the resulting filesto imapd-cert.pem and imapd-key.pem.


Test Page 37

A:root@mangus:~# /usr/lib/ssl/misc/CA.pl -newreq-nodesGenerating a 1024 bit RSA private key.......................++++++...................................++++++writing new private key to 'newkey.pem'-----You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [AU]:USState or Province Name (full name) [Some-State]:UtahLocality Name (eg, city) []:ProvoOrganization Name (eg, company) [Internet Widgits Pty Ltd]:UtopiaOrganizational Unit Name (eg, section) []:Common Name (eg, YOUR name) []:imap.utopia.netEmail Address []:

Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:An optional company name []:Request is in newreq.pem, private key is in newkey.pemroot@mangus:~# /usr/lib/ssl/misc/CA.pl -signreqUsing configuration from /usr/lib/ssl/openssl.cnfEnter pass phrase for ./demoCA/private/cakey.pem:Check that the request matches the signatureSignature okCertificate Details: Serial Number: e1:d8:25:12:be:99:bc:ac Validity Not Before: May 31 15:19:55 2012 GMT Not After : May 31 15:19:55 2013 GMT Subject: countryName = US stateOrProvinceName = Utah localityName = Provo organizationName = Utopia commonName = imap.utopia.net X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: AB:D4:28:4A:D1:44:79:FE:91:94:29:0E:DB:D0:50:57:A4:CA:DC:E0 X509v3 Authority Key Identifier: keyid:01:A9:62:BC:2B:5C:A9:8D:7C:80:9C:55:B9:1B:1C:B5:2E:92:27:16


1 out of 1 certificate requests certified, commit? [y/n]yWrite out database with 1 new entriesData Base UpdatedSigned certificate is in newcert.pemroot@mangus:~# mv newcert.pem imapd-cert.pemroot@mangus:~# mv newkey.pem imapd-key.pem


Test Page 38

Q: Go through the same steps to create a certificate (and key) for pop3.utopia.net. Rename the resulting filesto pop3d-cert.pem and pop3d-key.pem.


Test Page 39

A:root@mangus:~# /usr/lib/ssl/misc/CA.pl -newreq-nodesGenerating a 1024 bit RSA private key..........++++++.....................++++++writing new private key to 'newkey.pem'-----You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [AU]:USState or Province Name (full name) [Some-State]:UtahLocality Name (eg, city) []:ProvoOrganization Name (eg, company) [Internet Widgits Pty Ltd]:UtopiaOrganizational Unit Name (eg, section) []:Common Name (eg, YOUR name) []:pop3.utopia.netEmail Address []:

Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:An optional company name []:Request is in newreq.pem, private key is in newkey.pemroot@mangus:~# /usr/lib/ssl/misc/CA.pl -signreqUsing configuration from /usr/lib/ssl/openssl.cnfEnter pass phrase for ./demoCA/private/cakey.pem:Check that the request matches the signatureSignature okCertificate Details: Serial Number: e1:d8:25:12:be:99:bc:ae Validity Not Before: May 31 18:37:31 2012 GMT Not After : May 31 18:37:31 2013 GMT Subject: countryName = US stateOrProvinceName = Utah localityName = Provo organizationName = Utopia commonName = pop3.utopia.net X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: 89:46:BF:81:EC:CF:08:AC:3A:7E:95:C2:39:53:41:E1:F3:07:17:C1 X509v3 Authority Key Identifier: keyid:01:A9:62:BC:2B:5C:A9:8D:7C:80:9C:55:B9:1B:1C:B5:2E:92:27:16


1 out of 1 certificate requests certified, commit? [y/n]yWrite out database with 1 new entriesData Base UpdatedSigned certificate is in newcert.pemroot@mangus:~# mv newcert.pem pop3d-cert.pemroot@mangus:~# mv newkey.pem pop3d-key.pem


Test Page 40

3.2.5.2. Setting up SMTPS

Q: Move the smtpd-key.pem file into the /etc/postfix directory. This key file contains sensitiveinformation. It should only accessible by root. Adjust permissions accordingly.

A:root@mangus:~# mv newkey.pem /etc/postfix/smtpd-key.pemroot@mangus:~# chmod 600 /etc/postfix/smtpd-key.pem

Q: Postfix needs to have two certificates handy: The certificate holding the public key matching the private keyto make ciphered communications and the certificate from the CA that has signed the server's certificate,especially if the CA is a private one (self-signed certificates). It prefers to have both in one file. Concatenate(in that order) the smtpd-cert.pem certificate and the CA certificate (./demoCA/cacert.pemwith default settings) to create /etc/postfix/smtpd-cert.pem. The certificate file holds publicinformation. However it should not be world-writable. Adjust permissions accordingly.

A:root@mangus:~# cat smtpd-cert.pem demoCA/cacert.pem > /etc/postfix/smtpd-cert.pemroot@mangus:~# chmod 644 /etc/postfix/smtpd-cert.pem

Q: Configure Postfix to use the key and certificate you've just created. Enable SMTPS as an optional feature:The server should provide SMTPS upon client request, not force it.

A: Add the following lines to /etc/postfix/main.cf:

smtpd_tls_cert_file = /etc/postfix/smtpd-cert.pemsmtpd_tls_key_file = /etc/postfix/smtpd-key.pemsmtpd_tls_security_level = may

Q: Restart the service.

A:root@mangus:~# /etc/init.d/postfix restart

Q: Use telnet to check the STARTTLS capability of your server.

A:root@mangus:~# telnet localhost smtpTrying 127.0.0.1...Connected to localhost.Escape character is '^]'.220 mangus.utopia.net ESMTP PostfixEHLO wrk250-mangus.utopia.net250-PIPELINING250-SIZE 10240000250-VRFY250-ETRN250-STARTTLS250-AUTH LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM250-AUTH=LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM250-ENHANCEDSTATUSCODES250-8BITMIME250 DSNSTARTTLS220 2.0.0 Ready to start TLS


Test Page 41

3.2.5.3. Setting up IMAPS

If you're running a Debian-based system, you need to install the courier-imap-ssl package before goingany further.

Q: Courier's imapd supports SSL/TLS. However, it expects the key and the certificate to be in the samefile. Concatenate imapd-key.pem and imapd-cert.pem to /etc/courier/imapd.pem. This fileholds sensitive information. It should only accessible by root. Adjust permissions accordingly.

A:root@mangus:~# cat imapd-key.pem imapd-cert.pem > /etc/courier/imapd.pem root@mangus:~# chmod 600 /etc/courier/imapd.pem

Q: Edit courier-imap-ssl configuration file to ensure it uses that the right certificate file.

A:root@mangus:~# grep ^TLS_CERTFILE /etc/courier/imapd-ssl TLS_CERTFILE=/etc/courier/imapd.pem

Q: Restart the courier-imap-ssl service.

A:root@mangus:~# /etc/init.d/courier-imap-ssl restart

Q: Use the openssl s_client command to test the SSL connection to the imapd running on port 993. You usethe -CAfile openssl switch to provide your CA certificate file.


Test Page 42

A:root@mangus:~# openssl s_client -connect imap.utopia.net:993 \>-CAfile /root/demoCA/cacert.pem CONNECTED(00000003)depth=1 /C=US/ST=Utah/O=Utopia/CN=ca.utopia.netverify return:1depth=0 /C=US/ST=Utah/L=Provo/O=Utopia/CN=imap.utopia.netverify return:1---Certificate chain 0 s:/C=US/ST=Utah/L=Provo/O=Utopia/CN=imap.utopia.net i:/C=US/ST=Utah/O=Utopia/CN=ca.utopia.net---Server certificate-----BEGIN CERTIFICATE-----MIIClTCCAf6gAwIBAgIJAOHYJRK+mbysMA0GCSqGSIb3DQEBBQUAMEUxCzAJBgNVBAYTAlVTMQ0wCwYDVQQIEwRVdGFoMQ8wDQYDVQQKEwZVdG9waWExFjAUBgNVBAMTDWNhLnV0b3BpYS5uZXQwHhcNMTIwNTMxMTUxOTU1WhcNMTMwNTMxMTUxOTU1WjBXMQswCQYDVQQGEwJVUzENMAsGA1UECBMEVXRhaDEOMAwGA1UEBxMFUHJvdm8xDzANBgNVBAoTBlV0b3BpYTEYMBYGA1UEAxMPaW1hcC51dG9waWEubmV0MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCyG9VZCGyL5+5TYpPOYCm6dgbYVCufCOACrTUCau2QDXuQVN201qniiNI4tX+Mvo7fkgtJtUv6wbpdNlK++Okk0z12Fa2AiZfqiEVm2kRiMt7/rhYQY/CRnn6uC8Z7sOBshclr4c3d41gE5LzbYErhsZD6K2pfnGWvRibyeOSS+wIDAQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUq9QoStFEef6RlCkO29BQV6TK3OAwHwYDVR0jBBgwFoAUAalivCtcqY18gJxVuRsctS6SJxYwDQYJKoZIhvcNAQEFBQADgYEAJgtF9XHVcbFS+QJ2Rm9oxSdE1yXg4vjYPWE0EvjkHFLZdmPK7V1DMbKhP84rNDpu+dxG+xgjP2fCPNhWgZwnaVIFYbBjrar743uX8QVeYtMfF99KxnKUlBk8hL/aMov4Rr/grBvzIZ5AyTVFTh6syeZKN/r70DDly1fvErtjWbA=-----END CERTIFICATE-----subject=/C=US/ST=Utah/L=Provo/O=Utopia/CN=imap.utopia.netissuer=/C=US/ST=Utah/O=Utopia/CN=ca.utopia.net---No client certificate CA names sent---SSL handshake has read 834 bytes and written 319 bytes---New, TLSv1/SSLv3, Cipher is AES256-SHAServer public key is 1024 bitSecure Renegotiation IS supportedCompression: NONEExpansion: NONESSL-Session: Protocol : TLSv1 Cipher : AES256-SHA Session-ID: D5295F6E30A195A52F56FBB1F381A472251B4D336755967509214F5BEE85ACB8 Session-ID-ctx: Master-Key: C6FF20D5A2A212D9C1134A40617F20E91D675F2D9E2D290ADEE6C1F77B6C4869C53E882E7770D2102CB1FCAFA0253BDA Key-Arg : None Start Time: 1338490087 Timeout : 300 (sec) Verify return code: 0 (ok)---* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES [...] 01 LOGIN supinfo supinfo01 OK LOGIN Ok.02 LIST "" ** LIST (\HasNoChildren) "." "INBOX.Trash"* LIST (\HasNoChildren) "." "INBOX.Drafts"* LIST (\Marked \HasChildren) "." "INBOX"02 OK LIST completed03 SELECT INBOX* FLAGS (\Draft \Answered \Flagged \Deleted \Seen \Recent)* OK [PERMANENTFLAGS (\* \Draft \Answered \Flagged \Deleted \Seen)] Limited* 2 EXISTS* 0 RECENT* OK [UIDVALIDITY 1338302965] Ok* OK [MYRIGHTS "acdilrsw"] ACL03 OK [READ-WRITE] Ok04 FETCH 1 ALL* 1 FETCH (FLAGS (\Seen) INTERNALDATE "29-May-2012 17:51:29 +0200" RFC822.SIZE 895 ENVELOPE ("Tue, 29 May 2012 17:51:37 +0200" "Hi from Workstation" (("supinfo" NIL "supinfo" "debian-master.localdomain")) (("supinfo" NIL "supinfo" "debian-master.localdomain")) (("supinfo" NIL "supinfo" "debian-master.localdomain")) ((NIL NIL "supinfo" "utopia.net")) NIL NIL NIL "<[email protected]>"))04 OK FETCH completed.05 LOGOUT* BYE Courier-IMAP server shutting down05 OK LOGOUT completedclosed

3.2.5.4. Setting up POP3 over SSL

If you're running a Debian-based system, you need to install the courier-pop-ssl package before going anyfurther.


Test Page 43

Q: Courier's pop3d supports SSL/TLS. However, it expects the key and the certificate to be in the samefile. Concatenate pop3d-key.pem and pop3d-cert.pem to /etc/courier/pop3d.pem. This fileholds sensitive information. It should only accessible by root. Adjust permissions accordingly.

A:root@mangus:~# cat pop3d-key.pem pop3d-cert.pem > /etc/courier/pop3d.pem root@mangus:~# chmod 600 /etc/courier/pop3d.pem

Q: Edit courier-pop-ssl configuration file to ensure it uses that the right certificate file.

A:root@mangus:~# grep ^TLS_CERTFILE /etc/courier/pop3d-ssl TLS_CERTFILE=/etc/courier/pop3d.pem

Q: Restart the courier-imap-ssl service.

A:root@mangus:~# /etc/init.d/courier-pop-ssl restart

Q: Use the openssl s_client command to test the SSL connection to the popd running on port 995. You use the-CAfile openssl switch to provide your CA certificate file.

Note

Don't use the RETR POP3 command in capitals: openssl s_client will understand this first capitalR as a renogiate command. Either use retr in lowercase or put a space before RETR. For moreinformation, look at the "CONNECTED COMMANDS" section of s_client man page.


Test Page 44

A:root@mangus:~# openssl s_client -connect pop3.utopia.net:995 \>-CAfile /root/demoCA/cacert.pem CONNECTED(00000003)depth=1 /C=US/ST=Utah/O=Utopia/CN=ca.utopia.netverify return:1depth=0 /C=US/ST=Utah/L=Provo/O=Utopia/CN=pop3.utopia.netverify return:1---Certificate chain 0 s:/C=US/ST=Utah/L=Provo/O=Utopia/CN=pop3.utopia.net i:/C=US/ST=Utah/O=Utopia/CN=ca.utopia.net---Server certificate-----BEGIN CERTIFICATE-----MIIClTCCAf6gAwIBAgIJAOHYJRK+mbyuMA0GCSqGSIb3DQEBBQUAMEUxCzAJBgNVBAYTAlVTMQ0wCwYDVQQIEwRVdGFoMQ8wDQYDVQQKEwZVdG9waWExFjAUBgNVBAMTDWNhLnV0b3BpYS5uZXQwHhcNMTIwNTMxMTgzNzMxWhcNMTMwNTMxMTgzNzMxWjBXMQswCQYDVQQGEwJVUzENMAsGA1UECBMEVXRhaDEOMAwGA1UEBxMFUHJvdm8xDzANBgNVBAoTBlV0b3BpYTEYMBYGA1UEAxMPcG9wMy51dG9waWEubmV0MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCtERDr0IKAnnvmO8cm4lH2MYkFipe3h0QTDfyjdXyatuDYGCi0LYAZTpSn8WwtOQnldH8XT0hhjE2uHND1xznGK6hNMoNPA/ysrPIeTNkvNfzvT2t0iCqteLftKaozvxJ1bCEJGrIjyTbw2X7nX452kCL0S51N1iVKRN9O+x73aQIDAQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUiUa/gezPCKw6fpXCOVNB4fMHF8EwHwYDVR0jBBgwFoAUAalivCtcqY18gJxVuRsctS6SJxYwDQYJKoZIhvcNAQEFBQADgYEADDCJFnpv0dQvjUqzu8mhXJ+Rpw4EG6qtgHjy+RAs+r6f7gi+rOMp8rB2+6VkZbyN1DwGIhlisGvA0qVx00kt+su834I1085jpyTknSFqYv6mdxoQtvKmR3BPF7gpKk0YDqsIlxLDOn1Vo0i0SwMNMA+8Rgubmj3RyZSy0tauAzk=-----END CERTIFICATE-----subject=/C=US/ST=Utah/L=Provo/O=Utopia/CN=pop3.utopia.netissuer=/C=US/ST=Utah/O=Utopia/CN=ca.utopia.net---No client certificate CA names sent---SSL handshake has read 834 bytes and written 319 bytes---New, TLSv1/SSLv3, Cipher is AES256-SHAServer public key is 1024 bitSecure Renegotiation IS supportedCompression: NONEExpansion: NONESSL-Session: Protocol : TLSv1 Cipher : AES256-SHA Session-ID: 7F0CB5AFC7977A25057C6E3D0E1978D5E074500D289D37CCCCDF02A9524A124E Session-ID-ctx: Master-Key: C34D4789EA167F63D425C19365B180DD3568D48DE17A9443CDBBFE545B72E382AEBD08979887D412DC68CC617628F953 Key-Arg : None Start Time: 1338489795 Timeout : 300 (sec) Verify return code: 0 (ok)---+OK Hello there.USER supinfo+OK Password required.PASS supinfo+OK logged in.LIST+OK POP3 clients that break here, they violate STD53.1 8952 673.retr 1+OK 895 octets follow.Return-Path: <[email protected]>X-Original-To: [email protected]: [email protected]: from debian-master.localdomain (unknown [192.168.82.142]) by mangus.utopia.net (Postfix) with ESMTP id 313BF530A8 for <[email protected]>; Tue, 29 May 2012 17:51:29 +0200 (CEST)Received: from supinfo by debian-master.localdomain with local (Exim 4.72) (envelope-from <[email protected]>) id 1SZOht-0003PK-Kl for [email protected]; Tue, 29 May 2012 17:51:37 +0200Message-Id: <[email protected]>Date: Tue, 29 May 2012 17:51:37 +0200To: [email protected]: Hi from WorkstationUser-Agent: Heirloom mailx 12.4 7/29/08MIME-Version: 1.0Content-Type: text/plain; charset=us-asciiContent-Transfer-Encoding: 7bitFrom: supinfo <[email protected]>

Everything is up and running.quit+OK Bye-bye.closed


Test Page 45

3.2.6. Webmail

In the previous sections, you've configured a SMTP server with standard remote access protocols such as IMAPand POP3. You've lately added security on top of these protocols by enableing their "over SSL" counterparts. In thissection, you're going to install a Webmail to let your users work with their email directly from the web, withouta dedicated email client.

The Webmail is no more than a "regular" mail client: It makes uses of the standard POP3 and IMAP protocols.Therefore, it can run on any machine that can communicate with the mail server. In this lab, you're going to install iton the same machine as the mail server to minimize the need of additional virtual machines. However, if you reallywant to test operations on a separate machine, feel free to do so. Just adapt IP adresses to your specific scenario.

Q: Install the squirrelmail package.

A:root@mangus:~# apt-get install squirrelmail

Q: Configure Apache2 to serve content from /usr/share/squirrelmail. You can either directly includethe /etc/squirrelmail/apache.conf file shipped with squirrelmail in apache configuration orcreate a virtual host on your own. Don't forget to restart the service to apply your modifications.

A:root@mangus:~# echo "Include /etc/squirrelmail/apache.conf" >> /etc/apache2/apache2.conf root@mangus:~# /etc/init.d/apache2 restart

Q: Run the squirrelmail-configure command and adjust the settings to your setup. Be sure to set the domainto utopia.net in the "Server Settings" section.

A:root@mangus:~# squirrelmail-configureSquirrelMail Configuration : Read: config.php (1.4.0)---------------------------------------------------------Main Menu --1. Organization Preferences2. Server Settings3. Folder Defaults4. General Options5. Themes6. Address Books7. Message of the Day (MOTD)8. Plugins9. Database10. Languages

D. Set pre-defined settings for specific IMAP servers

C Turn color onS Save dataQ Quit

Command >> D

SquirrelMail Configuration : Read: config.php---------------------------------------------------------While we have been building SquirrelMail, we have discovered somepreferences that work better with some servers that don't work sowell with others. If you select your IMAP server, this option willset some pre-defined settings for that server.


Test Page 46

Please note that you will still need to go through and make sureeverything is correct. This does not change everything. There areonly a few settings that this will change.

Please select your IMAP server: bincimap = Binc IMAP server courier = Courier IMAP server cyrus = Cyrus IMAP server dovecot = Dovecot Secure IMAP server exchange = Microsoft Exchange IMAP server hmailserver = hMailServer macosx = Mac OS X Mailserver mercury32 = Mercury/32 uw = University of Washington's IMAP server gmail = IMAP access to Google mail (Gmail) accounts

quit = Do not change anythingCommand >> courier

imap_server_type = courier default_folder_prefix = INBOX. trash_folder = Trash sent_folder = Sent draft_folder = Drafts show_prefix_option = false default_sub_of_inbox = falseshow_contain_subfolders_option = false optional_delimiter = . delete_folder = true

Press any key to continue...

SquirrelMail Configuration : Read: config.php (1.4.0)---------------------------------------------------------Main Menu --1. Organization Preferences2. Server Settings3. Folder Defaults4. General Options5. Themes6. Address Books7. Message of the Day (MOTD)8. Plugins9. Database10. Languages

D. Set pre-defined settings for specific IMAP servers

C Turn color onS Save dataQ Quit

Command >> 2

SquirrelMail Configuration : Read: config.php (1.4.0)


Test Page 47

---------------------------------------------------------Server Settings

General-------1. Domain : trim(implode('', file('/etc/'.(file_exists('/etc/mailname')?'mail':'host').'name')))2. Invert Time : false3. Sendmail or SMTP : SMTP

A. Update IMAP Settings : localhost:143 (courier)B. Update SMTP Settings : localhost:25

R Return to Main MenuC Turn color onS Save dataQ Quit

Command >> 1

The domain name is the suffix at the end of all email addresses. Iffor example, your email address is [email protected], then your domainwould be example.com.

[trim(implode('', file('/etc/'.(file_exists('/etc/mailname')?'mail':'host').'name')))]: utopia.net

SquirrelMail Configuration : Read: config.php (1.4.0)---------------------------------------------------------Server Settings

General-------1. Domain : utopia.net2. Invert Time : false3. Sendmail or SMTP : SMTP

A. Update IMAP Settings : localhost:143 (courier)B. Update SMTP Settings : localhost:25

R Return to Main MenuC Turn color onS Save dataQ Quit

Command >> Q

You have not saved your data.Save? [Y/n]: YData saved in config.php

Exiting conf.pl.You might want to test your configuration by browsing tohttp://your-squirrelmail-location/src/configtest.phpHappy SquirrelMailing!


Test Page 48

Q: Point a browser or any http client to http://127.0.0.1/squirrelmail/src/configtest.php to check the configuration. This URL is valid if you've used the apache.conf file thatcomes with squirrelmail. If you've created a virtual host or a different configuration, your milage may vary.

A:root@mangus:~# apt-get install elinkssupinfo@mangus:~$ elinks http://127.0.0.1/squirrelmail/src/configtest.php

Q: From your host operating system, point your browser at http://vm-ip/squirrelmail/ (adjust theURL as needed) and try to login as supinfo/supinfo.

A:

3.3. Virtual domains

In the previous section, you've setup a complete e-mail system for the utopia.net domain. This setup was a"canonical" setup: Mail addresses were bound to system users. Although it's possible to have more than onecanonical domain attached to the mail server, it's not the most flexible setup: If you have more than one canonicaldomain, you've still the same user set: [email protected] and [email protected] are indeed the same userand therefore mailbox.

In this section, you're going to alter the configuration to host virtual domains. Each virtual domain has its ownset of users and mailboxes.


Test Page 49

3.3.1. Utopia.net goes virtual

In this section, you're going to tune Postfix configuration to make utopia.net a virtual domain.

3.3.1.1. Filesystem: Prepare virtual mailboxes

In this section, you're going to create necessary directory adjust permissions for virtual domains to operate.

Q: Each virtual domain user mailboxes must go in a dedicated directory under the virtual "mail root". Thisdirectory is usually named after the domain itself. Provided that the virtual mail root is /var/spool/mail, create a directory for the utopia.net domain.

A:root@mangus:~# mkdir /var/spool/mail/utopia.net

Q: Each "virtual" user (domain mailbox) will automatically have his mailbox directory created under the domaindirectory where it belongs. as you knoz Unix files and directories are always owned by a user and a group.When mail addresses matches system users, their mailboxes and files are just owned by these system usersuid and gid. However, with virtual domains enabled there is no system account associated with each mailboxand therefore no "natural" ownership policy. Moreover, the imap and pop3 deamons expects mailboxes tobo owned by the matching user and its primary group. When an user authenticates on the imap or pop3deamon the undertakes this very user identity, that is switching to this user effective uid and primary gid.As an example when you issue a AUTH supinfo supinfo when talking with the imapd, the imapd switches itseffective uid to supinfo and effective gid to supinfo's primary group.

The only satisfactory solution to these issues is to create a special user and a special group associated withvirtual users mail delivery. All virtual users will be bound to this uid/gid couple to solve the ownershipproblem.

Create a vmail system group and a vmail system user for this task. As these user and group are not "real"users, they should get a uid/gid in the appropriate range. Additionaly, the user should get /bin/false as itsshell, the virtual mail root as its home and of course the epony;ous group as primary group.

A:root@mangus:~# groupadd -g 111 vmailroot@mangus:~# useradd -u 107 -d /var/spool/mail/ -s /bin/false -g vmail vmail

Q: Change ownership of the utopia.net virtual mail directory to vmail:vmail.

Q:root@mangus:~# chown vmail:vmail /var/spool/mail/utopia.net

3.3.1.2. Postfix

In this section, you're going to alter postfix configuration to move utopia.net from canonical to virtual.

3.3.1.2.1. Virtual mailboxes mapping

In this section you're going to map mail addresses to mailbox directories.

Q: Create a /etc/postfix/vmailboxes file that maps supinfo, sarah and bob accounts from theutopia.net domain to their mailboxes stored under the domain directory. All relative path are relative tothe virtual mail root that will be /var/spool/mail. You want mail to be delivered in the maildir format:Use the appropriate syntax!

A: Create /etc/postfix/vmailboxes as follows:


Test Page 50

# utopia.net [email protected] utopia.net/supinfo/[email protected] utopia.net/bob/[email protected] utopia.net/sarah/

Warning

Be sure to add the trailing / to get mail delivered into maildir format.

Q: Generate the binary hash table file from your plain-text mapping file.

A:root@mangus:~# postmap /etc/postfix/vmailboxes

3.3.1.2.2. Server configuration

Q: Alter the server configuration to have:

• utopia.net as a virtual domain

• /var/spool/mail as the virtual mail domains root.

• /etc/postfix/vmailboxes as the virtual mail addresses mapping file

• vmail as the uid and gid used for virtual domains delivery

A: Your configuration should look like the following:


Test Page 51

smtpd_banner = $myhostname ESMTPbiff = no


mydestination = magnus localhost mangus.localdomain localhost.localdomainhome_mailbox = Maildir/

inet_interfaces = allmynetworks_style = host

virtual_mailbox_domains = utopia.netvirtual_mailbox_base = /var/spool/mailvirtual_mailbox_maps = hash:/etc/postfix/vmailboxes

virtual_uid_maps = static:107virtual_gid_maps = static:111

smtpd_sasl_auth_enable = yesbroken_sasl_auth_clients = yes

smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination

smtpd_tls_cert_file = /etc/postfix/smtpd-cert.pemsmtpd_tls_key_file = /etc/postfix/smtpd-key.pemsmtpd_tls_security_level = may

3.3.1.2.3. Testing the virtual domain

Q: Try to send mails to supinfo, bob and/or sarah.

A:root@mangus:~# mail [email protected]: Test virtualSome content.EOTroot@mangus:~# mail [email protected]: Virtual testMail received ?.EOT

Q: Check if your mails have reached the new mailboxes.


Test Page 52

A:root@mangus:~# find /var/spool/mail//var/spool/mail//var/spool/mail/utopia.net/var/spool/mail/utopia.net/sarah/var/spool/mail/utopia.net/sarah/tmp/var/spool/mail/utopia.net/sarah/new/var/spool/mail/utopia.net/sarah/new/1338567473.V801I531b3M813160.mangus/var/spool/mail/utopia.net/sarah/cur/var/spool/mail/utopia.net/supinfo/var/spool/mail/utopia.net/supinfo/tmp/var/spool/mail/utopia.net/supinfo/new/var/spool/mail/utopia.net/supinfo/new/1338567341.V801I531a8M199840.mangus/var/spool/mail/utopia.net/supinfo/cur

3.3.1.3. Courier

In the previous section, you've tuned postfix configuration to host virtual domains. The server is now acceptingincoming mail for virtual users mapped in the /etc/postfix/vmailboxes file. However the SMTPd cannotauthenticate you anymore, because it relies on the IMAP authentication that relies in turn on PAM that only knowsabout system users.

In this section, you'll configure Courier authentication layer to use virtual users.

3.3.1.3.1. Courier authentication daemon

Q: Edit the courier authentication daemon configuration to use the authuserdb authentication module (only).

A: Alter /etc/courier/authdaemonrc as follows:

authmodulelist="authuserdb"

Q: Restart courier authentication daemon

A:root@mangus:~# /etc/init.d/courier-authdaemon restart

3.3.1.3.2. Virtual user accounts

Virtual users are "created" by two complementary methods:

• Email address to mailbox mapping in postfix configuration

• Credentials to mailbox mapping in courier configuration

The first one tells postfix where to deliver messages whereas the second tells how to let people access thesemailboxes.

Q: Use the userdb command to create virtual users matching bob, supinfo and sarah. Be sure to

• Use full mail addresses as login names

• Bind them to the vmail uid and gid

• Set both their home and mail properties to their mailbox


Test Page 53

It doesn't matter if the mailbox directory hasn't been created yet (if the user hasn't received any message).The mailbox is VIRTUAL_MAIL_ROOT/DOMAIN_NAME/USERNAME, as typed in the postfix mapping file.

A:root@mangus:~# userdb [email protected] set uid=107 gid=111 \>home=/var/spool/mail/utopia.net/supinfo mail=/var/spool/mail/utopia.net/supinforoot@mangus:~# userdb [email protected] set uid=107 gid=111 \>home=/var/spool/mail/utopia.net/bob mail=/var/spool/mail/utopia.net/bobroot@mangus:~# userdb [email protected] set uid=107 gid=111 \ >home=/var/spool/mail/utopia.net/bob mail=/var/spool/mail/utopia.net/sarah

Q: Set a random password for all these users.

A:root@mangus:~# userdbpw -md5 | userdb [email protected] set systempwPassword: Reenter password: root@mangus:~# userdbpw -md5 | userdb [email protected] set systempwPassword: Reenter password: root@mangus:~# userdbpw -md5 | userdb [email protected] set systempwPassword: Reenter password:

Q: Rebuild the user database

A:root@mangus:~# makeuserdb

3.3.1.3.3. Adjusting Cyrus-SASL options

Q: Usernames are now in the "[email protected]" form. In the default configuration, Cyrus-sasl's saslauthdsplits the username in two parts and passes the login as "user" to the authentication backend. In our case,the authentication backup is the IMAP server, which expects logins in the "[email protected]" form. Use thesaslauthd man page to find which option tells the authentication deamon not to split the login(search forthe realm keyword). Adjusts saslauthd startup options and restart the daemon.

A: The switch is -r. Edit /etc/default/saslauthd as follows:

OPTIONS="-r c -m /var/spool/postfix/var/run/saslauthd"

And restart the service:

root@mangus:~# /etc/init.d/saslauthd restart

3.3.1.3.4. Tests

Q: Try to establish an IMAP session as [email protected], using SSL or not, at your option.


Test Page 54

A:supinfo@mangus:~$ telnet localhost imapTrying 127.0.0.1...Connected to localhost.Escape character is '^]'.* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double Precision See COPYING for distribution information.01 LOGIN [email protected] supinfo* BYE [ALERT] Fatal error: Account's mailbox directory is not owned by the correct uid or gid: Connection closed by foreign host.supinfo@mangus:~$ telnet localhost imapTrying 127.0.0.1...Connected to localhost.Escape character is '^]'.* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double Precision See COPYING for distribution information.01 LOGIN [email protected] supinfo* BYE [ALERT] Fatal error: Account's mailbox directory is not owned by the correct uid or gid: Connection closed by foreign host.supinfo@mangus:~$ telnet localhost imapTrying 127.0.0.1...Connected to localhost.Escape character is '^]'.* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double Precision See COPYING for distribution information.01 LOGIN [email protected] supinfo01 OK LOGIN Ok.02 LIST "" ** LIST (\Marked \HasNoChildren) "." "INBOX"02 OK LIST completed03 SELECT INBOX* FLAGS (\Draft \Answered \Flagged \Deleted \Seen \Recent)* OK [PERMANENTFLAGS (\* \Draft \Answered \Flagged \Deleted \Seen)] Limited* 1 EXISTS* 1 RECENT* OK [UIDVALIDITY 1338813376] Ok* OK [MYRIGHTS "acdilrsw"] ACL03 OK [READ-WRITE] Ok04 FETCH 1 ALL* 1 FETCH (FLAGS (\Recent) INTERNALDATE "01-Jun-2012 18:15:41 +0200" RFC822.SIZE 560 ENVELOPE ("Fri, 01 Jun 2012 18:15:41 +0200" "Test virtual" (("root" NIL "root" "mangus.localdomain")) (("root" NIL "root" "mangus.localdomain")) (("root" NIL "root" "mangus.localdomain")) ((NIL NIL "supinfo" "utopia.net")) NIL NIL NIL "<[email protected]>"))04 OK FETCH completed.

Q: Try to authenticate against the SMTPD as [email protected].


Test Page 55

A:supinfo@mangus:~$ printf '\[email protected]\0supinfo' | base64AHN1cGluZm9AdXRvcGlhLm5ldABzdXBpbmZvsupinfo@mangus:~$ telnet localhost smtpTrying 127.0.0.1...Connected to localhost.Escape character is '^]'.220 mangus.localdomain ESMTPEHLO ts250-mangus.localdomain250-PIPELINING250-SIZE 10240000250-VRFY250-ETRN250-STARTTLS250-AUTH LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM250-AUTH=LOGIN CRAM-MD5 DIGEST-MD5 PLAIN NTLM250-ENHANCEDSTATUSCODES250-8BITMIME250 DSNAUTH PLAIN AHN1cGluZm9AdXRvcGlhLm5ldABzdXBpbmZv235 2.7.0 Authentication successfulQUIT221 2.0.0 ByeConnection closed by foreign host.

Q: Try to authenticate as [email protected] using the webmail.

A:

3.3.2. Adding virtual domains

In the previous section, you've configured your mail system to host utopia.net as a virtual domain. In this section,you'll see how easy it is to host a new domain with the same system.

3.3.2.1. DNS

The first step is to make the domain's MX records point to our mail server. In this lab, we are going to setup a newzone in our DNS for newdomain.com which will figure the domain to add.

Q: Add a newdomain.com zone in the DNS with the MX record resolving to magnus ip address. Also add smtp,pop and imap A records to the domain. You can just copy the utopia.net zone file and do appropriatemodifications.

A: Add the zone declaration in the DNS configuration:

zone "newdomain.com" { type master; file "/etc/bind/db.newdomain.com";};


Test Page 56

And the zone file itself:

$TTL 604800@ IN SOA newdomain.com. root.newdomain.com. ( 20120604 ; Serial 604800 ; Refresh 86400 ; Retry 2419200 ; Expire 604800 ) ; Negative Cache TTL;@ IN NS ns@ IN MX 10 mail.newdomain.com.

ns IN A 192.168.82.156mail IN A 192.168.82.156imap IN CNAME mailpop3 IN CNAME mailsmtp IN CNAME mail

Q: Restart the service and check that the MX record is resolved to the local machine.

A: root@mangus:~# /etc/init.d/bind9 restartroot@mangus:~# dig MX newdomain.com

; <<>> DiG 9.7.3 <<>> MX newdomain.com;; global options: +cmd;; Got answer:;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50546;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 2

;; QUESTION SECTION:;newdomain.com. IN MX

;; ANSWER SECTION:newdomain.com. 604800 IN MX 10 mail.newdomain.com.

;; AUTHORITY SECTION:newdomain.com. 604800 IN NS ns.newdomain.com.

;; ADDITIONAL SECTION:mail.newdomain.com. 604800 IN A 192.168.82.156ns.newdomain.com. 604800 IN A 192.168.82.156

;; Query time: 0 msec;; SERVER: 192.168.82.156#53(192.168.82.156);; WHEN: Mon Jun 4 15:50:57 2012;; MSG SIZE rcvd: 101

3.3.2.2. Filesystem:

Q: Create a directory to store newdomain.com users mailboxes, just like you did for utopia.net. Rememberthat the virtual mail domains root is /var/spool/mail. Also make sure this directory is owned by thevirtual mail user and group.


Test Page 57

A:root@mangus:~# mkdir /var/spool/mail/newdomain.comroot@mangus:~# chown vmail:vmail /var/mail/newdomain.com

3.3.2.3. Postfix configuration

Q: Add newdomain.com to the list of virtual hosted domains.

A: Edit postfix configuration as follows:

virtual_mailbox_domains = utopia.net newdomain.com

Q: Add virtual mailbox mappings for:

• [email protected]



Don't forget to rebuild the mapping file afterwards

A: Your /etc/postfix/vmailboxes file should look like the following:

# utopia.net [email protected] utopia.net/supinfo/[email protected] utopia.net/bob/[email protected] utopia.net/sarah/

# newdomain.com [email protected] newdomain.com/john/[email protected] newdomain.com/glenn/[email protected] newdomain.com/brian/

Warning

Don't forget to trailing slash to get the mail delivered in the maildir format that Courier'sdaemons expect.

Re-hash the mappings:

root@mangus:~# postmap /etc/postfix/vmailboxes

Q: Restart the service.

A:root@mangus:~# /etc/init.d/postfix restart


Test Page 58

3.3.2.4. Registering users

In this section, you'll register virtual users into courier and bind them to the virtual addresses you've mapped inpostfix configuration.

Q: Use the userdb command to create virtual users matching john, glenn and brian. Be sure to

• Use full mail addresses as login names

• Bind them to the vmail uid and gid

• Set both their home and mail properties to their mailbox

It doesn't matter if the mailbox directory hasn't been created yet (if the user hasn't received any message).The mailbox is VIRTUAL_MAIL_ROOT/DOMAIN_NAME/USERNAME, as typed in the postfix mapping file.

A:root@mangus:~# userdb [email protected] set uid=107 gid=111 \>home=/var/spool/mail/newdomain.com/john mail=/var/spool/mail/newdomain.com/johnroot@mangus:~# userdb [email protected] set uid=107 gid=111 \>home=/var/spool/mail/newdomain.com/glenn mail=/var/spool/mail/newdomain.com/glennroot@mangus:~# userdb [email protected] set uid=107 gid=111 \>home=/var/spool/mail/newdomain.com/brian mail=/var/spool/mail/newdomain.com/brian

Q: Set a random password for all these users.

A: root@mangus:~# echo "supinfo" | userdbpw -md5 | userdb [email protected] set systempwroot@mangus:~# echo "supinfo" | userdbpw -md5 | userdb [email protected] set systempwroot@mangus:~# echo "supinfo" | userdbpw -md5 | userdb [email protected] set systempw

Q: Rebuild the user database

A:root@mangus:~# makeuserdb

3.3.2.5. Tests

Q: Try to open an IMAP session as [email protected]. Does it works? Why?

A:supinfo@mangus:~$ telnet imap.newdomain.com imapTrying 192.168.82.156...Connected to mail.newdomain.com.Escape character is '^]'.* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double Precision See COPYING for distribution information.01 LOGIN [email protected] supinfo* BYE Temporary problem, please try again laterConnection closed by foreign host.

It doesn't works, because glenn mailbox hasn't been created yet. It will be automatically created by postfixif he gets mail.

Q: Fix the problem, and try again.


Test Page 59

A:supinfo@mangus:~$ telnet imap.newdomain.com imapTrying 192.168.82.156...Connected to mail.newdomain.com.Escape character is '^]'.* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double Precision See COPYING for distribution information.01 LOGIN [email protected] supinfo* BYE Temporary problem, please try again laterConnection closed by foreign host.supinfo@mangus:~$ mail [email protected]: Welcome to our greate mail systemThis welcome mail will automatically create your mailbox.EOTsupinfo@mangus:~$ telnet imap.newdomain.com imapTrying 192.168.82.156...Connected to mail.newdomain.com.Escape character is '^]'.* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double PrecisionSee COPYING for distribution information.01 LOGIN [email protected] supinfo01 OK LOGIN Ok.

Q: Try to read your mail as glenn using the webmail.

A:


Test Page 60

4. iSCSIIn this section, you're going to experiment with iSCSI. Working with iSCSI takes at least two virtual machines: Amachine that exports devices, the target, and a machine that uses these exported devices, the initiator.

Prepare two virtual machines, with two network interfaces: One on the NAT'ed virtual network and the second onthe host-only network. Be sure to add two more hard drives to the machine you're going to use as the iSCSI target.

4.1. Simple iSCSI

In this section, you're going to export a pair of devices through iSCSI. I'm going to assume the you've prepared thevirtual machines with appropriate devices as stated above.

4.1.1. Target setup

In this section, you're going to configure the virtual machine that will export devices. You'll export the additionaltwo disks you've added to the virtual machine. Be sure to pick the right one.

Q: Set the machine hostname to "san".

A:root@debian-master:~# echo "san" > /etc/hostnameroot@debian-master:~# /etc/init.d/hostname.sh

Q: Initialize your two additional disks: Create a single partition on each one and format it as ext3.

A:root@san:~# fdisk /dev/sdb Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-1044, default 1): Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-1044, default 1044): Using default value 1044

Command (m for help): wroot@san:~# fdisk /dev/sdc Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-1044, default 1): Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-1044, default 1044): Using default value 1044

Command (m for help): wroot@san:~# mkfs.ext3 /dev/sdb1 root@san:~# mkfs.ext3 /dev/sdc1


Test Page 61

Q: Install appropriate packages to act as a target.

A:root@san:~# apt-get install iscsitargetroot@san:~# apt-get install iscsitarget-dkms

Note


Q: Export your devices as disk1 and disk2 in the utopia.net domain. Use the UUID's rather than devices names:Devices names are connection-dependant whereas the UUID's will never change. Protect your devices withsanuser/sanuser credentials.

A:root@san:~# ls /dev/disk/by-uuid/ -ltotal 0lrwxrwxrwx 1 root root 10 Jun 12 13:41 0d249d58-fa1e-4e09-ae98-ea777460224e -> ../../sda2lrwxrwxrwx 1 root root 10 Jun 12 13:41 44d22a03-f7ea-44c0-ae72-9eb1465f51f5 -> ../../sda1lrwxrwxrwx 1 root root 10 Jun 12 13:47 5ee38f27-1d89-4ca6-94f6-c2e10cab88b3 -> ../../sdc1lrwxrwxrwx 1 root root 10 Jun 12 13:47 d7a134e3-bf82-4702-9ae1-bec7513baa7b -> ../../sdb1

Edit your /etc/iet/ietd.conf as follow:

Target iqn.2012.06.net.utopia:disk1 IncomingUser sanuser sanuser Lun 0 Path=/dev/disk/by-uuid/d7a134e3-bf82-4702-9ae1-bec7513baa7b,Type=fileio

Target iqn.2012.06.net.utopia:disk2 IncomingUser sanuser sanuser Lun 0 Path=/dev/disk/by-uuid/5ee38f27-1d89-4ca6-94f6-c2e10cab88b3,Type=fileio

Q: (Re)start the service.

A: If you're running Debian, enable the target first:

root@san:~# echo "ISCSITARGET_ENABLE=true" > /etc/default/iscsitarget

And then restart the service:

root@san:~# /etc/init.d/iscsitarget restart

4.1.2. Initiator setup

In this section, you're going to configure the second virtual machine as an iSCSI initiator that connects the targetand mount devices.

Q: Set the machine hostname to "initiator".

A:root@debian-master:~# echo "initiator" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh

Q: Install the iscsi initiator userland tools.


Test Page 62

A:root@initiator:~# apt-get install open-iscsi

Q: Start the iSCSI client deamon.

A:root@initiator:~# /etc/init.d/open-iscsi start

Q: Try to discover targets exported from the first virtual machine.

A:root@initiator:~# iscsiadm -m discovery -t st -p 192.168.82.164192.168.82.164:3260,1 iqn.2012.06.net.utopia:disk1192.168.82.164:3260,1 iqn.2012.06.net.utopia:disk2

Q: Try to have the local iSCSI node to login to the first exported target. Does it works? Why?

A:root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk1" \>--portal "192.168.82.164:3260" --loginLogging in to [iface: default, target: iqn.2012.06.net.utopia:disk1, portal: 192.168.82.164,3260]iscsiadm: Could not login to [iface: default, target: iqn.2012.06.net.utopia:disk1, portal: 192.168.82.164,3260]:

It doesn't work because you need to authenticate to the target as sanuser/sanuser to access this resource.

Q: Try again while supplying the right credentials for this target.

A:root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk1" \>-p "192.168.82.164:3260" --op=update -n node.session.auth.username -v sanuserroot@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk1" \>-p "192.168.82.164:3260" --op=update -n node.session.auth.password -v sanuserroot@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk1" \>--portal "192.168.82.164:3260" --loginLogging in to [iface: default, target: iqn.2012.06.net.utopia:disk1, portal: 192.168.82.164,3260]Login to [iface: default, target: iqn.2012.06.net.utopia:disk1, portal: 192.168.82.164,3260]: successful

Q: Edit the local node configuration to supply sanuser/sanuser credentials by default. Also enable CHAPauthentication.

A: Edit /etc/iscsi/iscsid.conf as follows:

node.session.auth.authmethod = CHAP

node.session.auth.username = sanusernode.session.auth.password = sanuser

Q: Try to connect to the second target. Does it works? Why?

A:root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk2" \>--portal "192.168.82.164:3260" --loginLogging in to [iface: default, target: iqn.2012.06.net.utopia:disk2, portal: 192.168.82.164,3260]Login to [iface: default, target: iqn.2012.06.net.utopia:disk2, portal: 192.168.82.164,3260]: successful

It works because the expected credentials have been sent using the default values.

Q: Mount the first disk, try to create a file on it and then unmount the remote device.


Test Page 63

A:root@initiator:~# mount \>/dev/disk/by-path/ip-192.168.82.164\:3260-iscsi-iqn.2012.06.net.utopia\:disk1-lun-0 /mnt/root@initiator:~# echo "Hello SAN" > /mnt/testroot@initiator:~# umount /mnt/

Q: Restart the daemon. Are the devices still present? Why?

A:root@initiator:~# /etc/init.d/open-iscsi restartroot@initiator:~# ls /dev/disk/by-path/pci-0000:00:07.1-scsi-1:0:0:0 pci-0000:00:10.0-scsi-0:0:0:0 pci-0000:00:10.0-scsi-0:0:0:0-part1 pci-0000:00:10.0-scsi-0:0:0:0-part2

The remote devices didn't survived the deamon restart, because the node.startup value is set to itsdefault, manual. Therefore devices are never automatically attached are created.

Q: Adjust the node configuration to have the devices mounted at deamon startup.

A:root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk1" \>-p "192.168.82.164:3260" --op=update -n node.startup -v automaticroot@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:disk2" \>-p "192.168.82.164:3260" --op=update -n node.startup -v automaticroot@initiator:~# /etc/init.d/open-iscsi restartLogin to [iface: default, target: iqn.2012.06.net.utopia:disk1, portal: 192.168.82.164,3260]: successfulLogin to [iface: default, target: iqn.2012.06.net.utopia:disk2, portal: 192.168.82.164,3260]: successful

Q: Modify the default settings to have any newly created device automatically attached as well.

A: Edit the /etc/iscsi/iscsid.conf file as well:

node.startup = automatic

4.2. Multipath iSCSI

In this section, you're going to configure multipath iSCSI. Multipath iSCSI provides network link redondancy and letyour devices survive a link failure. In real-world setups, the complete link should be redundant, including differentnetwork cards, but also different switches, like in the follwing schema:


Test Page 64

4.2.1. Target setup

Q: Enable and configure the second NIC. You can either use DHCP or set a fixed IP addresses. Of course a real-life SAN server should rather have fixed IP addresses. However for a lab environment there is no practicaldifference.

A: Edit /etc/network/interfaces as follows:

allow-hotplug eth1iface eth1 inet staticaddress 172.17.2.11netmask 255.255.255.0

And apply your modifications:

root@san:~# ifup eth1

Note

This is Debian-specific. This is also a fixed-address configuration. You can perfectly use a DHCP-based setup instead.

4.2.2. Initiator setup

Q: Configure your second network interface to be on the same network as the first virtual machine secondvirtual interface you've just configured.

A: Edit /etc/network/interfaces as follows:


Test Page 65

allow-hotplug eth1iface eth1 inet staticaddress 172.17.2.22netmask 255.255.255.0

And apply your modifications:

root@initiator:~# ifup eth1

Q: List exported devices from the first virtual machine. What do you notice?

A:root@initiator:~# iscsiadm -m discovery -t st -p 192.168.82.164192.168.82.164:3260,1 iqn.2012.06.net.utopia:disk1172.17.2.11:3260,1 iqn.2012.06.net.utopia:disk1192.168.82.164:3260,1 iqn.2012.06.net.utopia:disk2172.17.2.11:3260,1 iqn.2012.06.net.utopia:disk2

Each device appears twice, once per IP.

Q: Login to the targets on the second IP address.

A:root@initiator:~# iscsiadm -m node -T "iqn.2012.06.net.utopia:disk1" \>-p "172.17.2.11:3260" --loginroot@initiator:~# iscsiadm -m node -T "iqn.2012.06.net.utopia:disk2" \>-p "172.17.2.11:3260" --login

Q: Lis all devices present in /dev/disk/by-path/ using long listing mode. What do you notice?

A:root@initiator:~# ls /dev/disk/by-path/ -ltotal 0lrwxrwxrwx 1 root root 9 Jun 12 18:00 ip-172.17.2.11:3260-iscsi-iqn.2012.06.net.utopia:disk1-lun-0 -> ../../sddlrwxrwxrwx 1 root root 9 Jun 12 18:01 ip-172.17.2.11:3260-iscsi-iqn.2012.06.net.utopia:disk2-lun-0 -> ../../sdelrwxrwxrwx 1 root root 9 Jun 12 17:09 ip-192.168.82.164:3260-iscsi-iqn.2012.06.net.utopia:disk1-lun-0 -> ../../sdclrwxrwxrwx 1 root root 9 Jun 12 17:09 ip-192.168.82.164:3260-iscsi-iqn.2012.06.net.utopia:disk2-lun-0 -> ../../sdblrwxrwxrwx 1 root root 9 Jun 12 13:41 pci-0000:00:07.1-scsi-1:0:0:0 -> ../../sr0lrwxrwxrwx 1 root root 9 Jun 12 13:41 pci-0000:00:10.0-scsi-0:0:0:0 -> ../../sdalrwxrwxrwx 1 root root 10 Jun 12 13:41 pci-0000:00:10.0-scsi-0:0:0:0-part1 -> ../../sda1lrwxrwxrwx 1 root root 10 Jun 12 13:41 pci-0000:00:10.0-scsi-0:0:0:0-part2 -> ../../sda2

You know have two devices per target, one per path.

4.2.2.1. Using multipath

You have two distinct network paths to reach each of your targets. Each network path is represented by a device.It means that from the system perspective, each path is something different. If you want to use both at the sametime, one solution could be to mount these devices in separate mount points. However, only a few filesystemdesigned on purpose actually supports multi-mounting. At this point having two path but two devices seems prettyuseless. What you want is to have only one device and the system using the two path indinstinctively.

The achieve this goal, you need an additional piece of software that will tell the system that these two devicesare actually the same.

Q: Install the multipath tools package.

A:root@initiator:~# apt-get install multipath-tools


Test Page 66

Q: Ensure that the device mapper kernel module is loaded.

A:root@initiator:~# modprobe dm_mod

Q: Restart the multipath deamon and check the multipath status.

A:root@initiator:~# /etc/init.d/multipath-tools restartroot@initiator:~# multipath -lmpath1 (14945540000000000745c758a8fcf684b36429b04a630ef63) dm-1 IET,VIRTUAL-DISKsize=8.0G features='0' hwhandler='0' wp=rw|-+- policy='round-robin 0' prio=-1 status=active| `- 15:0:0:0 sdc 8:32 active undef running`-+- policy='round-robin 0' prio=-1 status=enabled `- 17:0:0:0 sdd 8:48 active undef runningmpath0 (14945540000000000a27ba0f0a1070b8e757e06570954193d) dm-0 IET,VIRTUAL-DISKsize=8.0G features='0' hwhandler='0' wp=rw|-+- policy='round-robin 0' prio=-1 status=active| `- 16:0:0:0 sdb 8:16 active undef running`-+- policy='round-robin 0' prio=-1 status=enabled `- 18:0:0:0 sde 8:64 active undef running

Q: Mount the mpath0 device and create a file on the filesystem.

A:root@initiator:~# mount /dev/mapper/mpath0 /mnt/root@initiator:~# echo "Hello multipath" > /mnt/dummy

Q: Bring one of your interfaces down and try to read the file.

A:root@initiator:~# ifconfig eth1 downroot@initiator:~# cat /mnt/dummy Hello multipath


Test Page 67

5. DRBD: Distributed Replicated Block DeviceIn this Lab, you're going to experiment with the distributed replicated block device. This in-kernel subsystem allowsadministrator to build what one could call networked RAID-1.

You'll need a total of three virtual machines, each one with an additional disk that will represent the volume beingreplicated. The first section will only use two virtual machines whereas the second will demonstrate resourcestacking with three nodes. Two of these nodes will also need additional network interfaces. However as theseinterfaces would be useless for this first part of this lab, there is no need to add them now. You'll be givenappropriate instructions in the begining of the second section.

Prepare two virtual machines with an additional disk drive. The disk drive must be of the same size on both nodes.A size of 200MB should be sufficient.

This lab is about DRBD. DRBD sits at the block layer, therefore it has now knowledge of the above layers, i.efilesystems. This lab will not feature any filesystem-related operations.s

5.1. Two-nodes

In this section, you're going to setup a two-node replicated block device, like in the following picture:

5.1.1. Preparing the nodes.

Q: Set hostnames on both nodes: drbd-node1 and drbd-node2.

A:root@debian-master:~# echo "drbd-node1" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh

And:

root@debian-master:~# echo "drbd-node2" > /etc/hostname root@debian-master:~# /etc/init.d/hostname.sh

Q: If you're running a Debian-based system, check you've the following source and update your packagedatabase.


Test Page 68

deb http://repo.labo-linux.fr/debian/ binary/

A:root@drbd-node1:~# echo \>"deb http://repo.labo-linux.fr/debian/ binary/" >> /etc/apt/sources.listroot@drbd-node1:~# apt-get update

And:

root@drbd-node2:~# echo \>"deb http://repo.labo-linux.fr/debian/ binary/" >> /etc/apt/sources.listroot@drbd-node2:~# apt-get update

Q: Install the drbd 8.4 userland tools package and kernel module on the two nodes. If you're running a Debian-based distro, the package name are drbd8-utils and drbd8-module-2.6.32-5-686.

A:root@drbd-node1:~# root@drbd-node1:~# apt-get install drbd8-module-2.6.32-5-686 drbd8-utils

And:

root@drbd-node2:~# apt-get install drbd8-module-2.6.32-5-686 drbd8-utils

Q: Be sure to have DRBD >= 8.4 for either the kernel module and the userland tools. Double-check the version.

A:root@drbd-node2:~# modinfo drbdfilename: /lib/modules/2.6.32-5-686/updates/drbd.koalias: block-major-147-*license: GPLversion: 8.4.1root@drbd-node2:~# drbdadm --version DRBDADM_BUILDTAG=GIT-hash:\ 91b4c048c1a0e06777b5f65d312b38d47abaea80\ build\ by\ supinfo@debian-dev\,\ 2012-06-14\ 12:35:11iDRBDADM_API_VERSION=1DRBD_KERNEL_VERSION_CODE=0x000000DRBDADM_VERSION_CODE=0x080401DRBDADM_VERSION=8.4.1

Q: Load the drbd kernel module (on both nodes).

A:root@drbd-node1:~# modprobe drbd

And

root@drbd-node2:~# modprobe drbd

Q: On both nodes, create a single partition from the additional disk. This partition will be the replicated device.


Test Page 69

A:root@drbd-node1:~# fdisk /dev/sdb Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-204, default 1): Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-204, default 204): Using default value 204

Command (m for help): Command (m for help): wThe partition table has been altered!

Calling ioctl() to re-read partition table.Syncing disks.

And:

root@drbd-node2:~# fdisk /dev/sdb Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-1044, default 1): Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-1044, default 1044): Using default value 1044



5.1.2. Configuring replication

The drbd configuration files must be the same on both nodes. It means you can write your initial configurationon any node and then copy it to the other over SSH.

5.1.2.1. Single-Primary setup

Q: Create a "storage" replication resource configuration with the following properties:

• Exposed through the /dev/drbd1 device

• Replicates the /dev/sdb1 device

• Synchronous replication


Test Page 70

• metadata stored on the same device as the actual data.

A: Edit /etc/drbd.d/storage.res as follows:

resource storage { device /dev/drbd1; disk /dev/sdb1; meta-disk internal;

on drbd-node1 { address 192.168.82.168:7788; } on drbd-node2 { address 192.168.82.156:7788; }}

Note

As the default settings in /etc/drbd.d/global_common.conf set Protocol C globally, thereis no need to state it again at the resource level. However you can still explicitly mention it. Itwill do nothing more but won't harm your configuration either.

Q: Copy your configuration to the other node.

A:root@drbd-node1:~# scp /etc/drbd.d/storage.res [email protected]:/etc/drbd.d/

Q: Initialize the metadata and bring the resource up on both nodes.

A:root@drbd-node1:~# drbdadm create-md storage --== Thank you for participating in the global usage survey ==--The server's response is:

you are the 5015th user to install this versionWriting meta data...initializing activity logNOT initializing bitmapNew drbd meta data block successfully created.successroot@drbd-node1:~# drbdadm up storage

and on the second node:


Test Page 71

root@drbd-node2:~# drbdadm create-md storage --== Thank you for participating in the global usage survey ==--The server's response is:

you are the 16339th user to install this versionWriting meta data...initializing activity logNOT initialized bitmapNew drbd meta data block successfully created.successroot@drbd-node2:~# drbdadm up storage

Q: What's the current DRBD state? Why?

A:root@drbd-node2:~# cat /proc/drbd version: 8.3.7 (api:88/proto:86-91)srcversion: EE47D8BF18AC166BE219757

1: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r---- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:8385604

And/Or

root@drbd-node2:~# drbd-overview 1:storage/0 Connected Secondary/Secondary Inconsistent/Inconsistent C r-----

Both nodes are in the secondary state, and the replicated device is considered to be in the inconsistentstate. As we've just created the volume this is perfectly normal: We've still have to tell DRBD which deviceis the original master and enable initial synchronization.

Q: Set drbd-node1 as primary and force a synchronization.

A:root@drbd-node1:~# drbdadm primary --force storage

Q: Check DRBD status. What do you notice?

A:root@drbd-node1:~# drbd-overview 1:storage/0 Connected Primary/Secondary UpToDate/UpToDate C r-----

The DRBD volume is now up to date.

5.1.2.1.1. Tests

Q: On the first node, write a FOO string directly to the DRBD device. This string will be used as a synchronisationreference.

A:root@drbd-node1:~# echo "FOO" > /dev/drbd/by-res/storage/0

Q: On the second node, try to read the first 4 bytes in ASCII mode (If you don't know how to do that, read theod man page) from the DRBD device. Does it works? Why?


Test Page 72

A:root@drbd-node2:~# od -c -N 4 /dev/drbd/by-res/storage/0od: /dev/drbd/by-res/storage/0: Wrong medium type

It doesn't work because the DRBD driver doesn't expose data on the two nodes while running in primary/secondary mode.

Q: Switch nodes status (first node to secondary and second node to primary) and try again.

A:root@drbd-node1:~# drbdadm secondary storage

And:

root@drbd-node2:/home/supinfo# drbdadm primary storageroot@drbd-node2:/home/supinfo# od -c -N 4 /dev/drbd/by-res/storage0000000 F O O \n0000004

5.1.2.2. Dual-primary setup

Q: Temporary enable the dual-primary mode on for the "storage" resource.

A:root@drbd-node1:~# drbdadm net-options --protocol=C --allow-two-primaries storage

Note

Dual-primary is only available using protocol C and thus is a C-protocol option. That's why youneed to explicitly set C protocol, even if you're already using it.

Q: Make the two nodes primary.

A:root@drbd-node1:~# drbdadm primary storage

And

root@drbd-node2:~# drbdadm primary storage

Q: Check the resource status.

A:root@drbd-node2:~# drbd-overview 1:storage/0 Connected Primary/Primary UpToDate/UpToDate C r-----

Q: Write a random string on the device from a node and check the replication on the other.

A:root@drbd-node1:~# echo "BAR" > /dev/drbd/by-res/storage/0


Test Page 73

And

root@drbd-node2:~# od -c -N 4 /dev/drbd/by-res/storage/00000000 B A R \n0000004

Q: Put one of the nodes in secondary mode and restore the original configuration by re-reading the config fileon both nodes.

A:root@drbd-node1:~# drbdadm secondary storageroot@drbd-node1:~# drbdadm adjust storage

and

root@drbd-node2:~# drbdadm adjust storage

Q: Check resource status.

A:root@drbd-node2:~# drbd-overview 1:storage/0 Connected Primary/Secondary UpToDate/UpToDate C r-----

5.1.3. Dealing with split-brain

In this section, you're going to manually resolve a split brain case. Split-brain occurs when the link between thetwo nodes fails and when for some reason (manual, cluster-driven, ...) the two nodes become primary in the meantime. The following picture illustrate such a situation:

Q: Break the link between the two nodes: Unplug the virtual interace.

Q: Write a different random string on the device on each node.

A:root@drbd-node1:~# echo "NEW1" > /dev/drbd/by-res/storage/0

And

root@drbd-node2:~# echo "NEW2" > /dev/drbd/by-res/storage/0

Q: Re-plug the virtual network interface and check the device status.


Test Page 74

A:root@drbd-node1:~# drbd-overview 1:storage/0 StandAlone Primary/Unknown UpToDate/DUnknown r-----

And

root@drbd-node2:~# drbd-overview 1:storage/0 StandAlone Primary/Unknown UpToDate/DUnknown r-----

Q: The resource is now in split-brain state. You must decide which copy of the data you want to keep (survivor)and which one to discard (victim). Select a node and discard its data.

A:root@drbd-node2:~# drbdadm secondary storageroot@drbd-node2:~# drbdadm connect --discard-my-data storage

I've choosen to discard data from node2. You can perfectly discard data from node1 if you wish.

Q: Connect the other node back, and check resource status.

A:root@drbd-node1:~# drbdadm connect storageroot@drbd-node1:~# drbd-overview 1:storage/0 Connected Primary/Secondary UpToDate/UpToDate C r-----

Q: Check the device content: It should now contain the data from the "surviving" node.

A:root@drbd-node1:~# od -c -N 4 /dev/drbd/by-res/storage/00000000 N E W 10000004

5.2. Three-nodes

In this section, you're going to add a supplementary replica to the existing DRBD array. However this third nodewill not behave like the first two. The first two nodes are using protocol C: A write is considered complete onlywhen the other node has actually written the data to the disk. The third node will "emulate" an off-site replicaconnected through a relatively low bandwith WAN. This node will use protocol A: A write is considered complete assoon as the sync data has been pushed in the socket buffer (and of course written to the local disk). The followingpicture illustrate the network setup:


Test Page 75

The WAN will be simulated by an additional host-only network link. Add a NIC to drbd-node1 and connect it tothe host-only network.

You'll also need to prepare your third virtual machine as follows:

• Hostname: drbd-node3

• Additional single-partitioned 200 MB hard drive.

• Virtual NIC connected to the host-only network.

Also make sure drbd-node2 and drbd-node3 can communicate through the host-only network: Configure theinterfaces to either get addresses via DHCP or set fixed IP addresses. Also make sure to add the updated drbdpackages source as you've done before. Install appropriate packages as well. You might need to temporary switchyour VNIC from host-only to NAT to perform this step.

The rest of this lab assumes the above elements have been correctly configured.

Q: Add a new stacked resource to your configuration. This resource is stacked on top of storage and will bebound to /dev/drbd5 and listen to the host-only NIC of drbd-node2. On drbd-node3 this resource is alsobound to /dev/drbd5 and uses the /dev/sdb1 device. As this is a stacked resources, it also have internalmetadata.

Also add the follwing disk parameters to your stacked resource:

disk { disk-barrier no; disk-flushes no; md-flushes no;}

These parameters will prevent a deadlock issue you can encounter on VMware virtual machines.

What can you say about this configuration?

A: Edit /etc/drbd.d/storage.res as follows:


Test Page 76


on drbd-node1 { address 192.168.82.168:7788; }

on drbd-node2 { address 192.168.82.156:7788; }}

resource storage-m { net { protocol A; }

disk { disk-barrier no; disk-flushes no; md-flushes no; }

stacked-on-top-of storage { device /dev/drbd5; address 172.17.2.129:7788; }

on drbd-node3 { device /dev/drbd5; disk /dev/sdb1; address 172.17.2.128:7788; meta-disk internal; }}

The stacked resource is bound to drbd-node2 NIC. Therefore it will only work when the node2 is the primarynode. It would be better to have the stacked resource bound to an IP address that always "moves" to theprimary node. That's what cluster software (like pacemake) are used for.

Q: Replicate the configuration on all the three nodes.

A:root@drbd-node1:~# scp /etc/drbd.d/storage.res [email protected]:/etc/drbd.d/root@drbd-node2:~# scp /etc/drbd.d/storage.res [email protected]:/etc/drbd.d/

Q: Ensure that drbd-node2 has primary role. Set it as needed.


Test Page 77

A:root@drbd-node2:~# drbdadm role storageSecondary/Primaryroot@drbd-node1:~# drbdadm secondary storageroot@drbd-node2:~# drbdadm primary storageroot@drbd-node2:~# drbdadm role storagePrimary/Secondary

Q: From drbd-node2, initialize the meta-data on the stacked resource, and bring it up.

A:root@drbd-node2:~# drbdadm create-md --stacked storage-mmd_offset 213843968al_offset 213811200bm_offset 213803008

Found some data

==> This might destroy existing data! <==

Do you want to proceed?[need to type 'yes' to confirm] yes

Writing meta data...initializing activity logNOT initializing bitmapNew drbd meta data block successfully created.successroot@drbd-node2:~# drbdadm up --stacked storage-m

Q: Set drdb-node2 as primary for the stacked resource.

A:root@drbd-node2:~# drbdadm primary --stacked --force storage-m

Q: On drbd-node3, initialize the resource by creating the metadata and bring the resource up.

A:root@drbd-node3:/home/supinfo# drbdadm create-md storage-mroot@drbd-node3:/home/supinfo# drbdadm up storage-mroot@drbd-node3:/home/supinfo# drbd-overview 5:storage-m/0 Connected Secondary/Primary UpToDate/UpToDate A r-----

5.2.1. Tests

Q: On drbd-node2 write the "STK" string to the drbd device (not the stacked one).

A:root@drbd-node2:~# echo "STK" > /dev/drbd/by-res/storage/0

Q: On drbd-node2, check the replication on the stacked device. Due to the very low amount of data, you mayhave to wait a couple of seconds to get it replicated.


Test Page 78

A:root@drbd-node2:~# od -c -N 4 /dev/drbd/by-res/storage-m 0000000 N E W 10000004root@drbd-node2:~# od -c -N 4 /dev/drbd/by-res/storage-m 0000000 N E W 10000004root@drbd-node2:~# od -c -N 4 /dev/drbd/by-res/storage-m 0000000 N E W 10000004root@drbd-node2:~# od -c -N 4 /dev/drbd/by-res/storage-m 0000000 N E W 10000004root@drbd-node2:~# od -c -N 4 /dev/drbd/by-res/storage-m 0000000 S T K \n0000004

Q: Speed up the replication process by remotely invalidate the stacked resource on drdb-node3 from drdb-node2, and then switch the stacked resource from primary to secondary (You want to read data on theresource on drbd-node3, so you'll have to set the resource as primary there, therefore you must demoteit on drbd-node2 before).

A:root@drbd-node2:~# drbdadm invalidate-remote --stacked storage-mroot@drbd-node2:~# drbdadm secondary --stacked storage-m

Q: On drdb-node3, promote the resource and read the first bytes to see if the "STK" string you've originalywritten to the underlying resource has been replicated over here.

A:root@drbd-node3:~# drbdadm primary storage-m root@drbd-node3:~# od -c -N 4 /dev/drbd/by-res/storage-m 0000000 S T K \n0000004

Q: Demote the resource on drbd-node3 and promote it again on drbd-node2 to ensure proper replicationoperations.

A:root@drbd-node3:~# drbdadm secondary storage-mroot@drbd-node2:~# drbdadm primary --stacked storage-m


Test Page 79

6. High Availability

6.1. High available website

In this section, you're going to build a two-nodes cluster solution that provides high-availability for a website. Ofcourse the two-node is just an arbitrary number choosed to minimize the amount of resources needed to run thislab. You can expand this number at will to scale the cluster up.

The website data will be stored on an NFS volume mounted from the active node, like in the following schema:

Note

This is a functional schema. All hosts are connected to the same network.

You'll need at least three virtual machines: Two cluster nodes and an NFS server. The client machine can be yourhost operating system, or an additional virtual machine, at your option.

Prepare your virtual machines and ensure that they are connected to the same network and see each other. Hereis a list of hostnames for these machines:

• nfs-server

• cluster-node1

• cluster-node2

6.1.1. The NFS server and web content

Q: Create a /exports/docroot/index.php file with some php code that shows the following message(hostname being the hostname of the machine where the code runs):

Hello from <hostname>

A:root@nfs-server:~# mkdir -p /exports/wwwroot@nfs-server:~# echo '<?php echo "Hello from ".gethostname()."\n" ?>' > /exports/www/index.php


Test Page 80

Q: Install appropriate packages to serve content over NFS (as needed).

A:root@nfs-server:~# apt-get install nfs-kernel-server

Note


Q: Export /exports/www in read-only for all hosts.

A:root@nfs-server:~# echo "/exports/www *(ro,no_subtree_check)" >> /etc/exports root@nfs-server:~# exportfs -a

6.1.2. Cluster nodes

6.1.2.1. Installing services

Before actually setting up the cluster engine that will manage nodes and fail nodes over, you need to configureactual the services that needs to be managed. In this section, you'll configure Apache to serve content and thencopy over the configuration to the other node. You can do the inital configuration on whichever node you want.

Q: Install the Apache webserver and its php module. Restart the service to ensure proper php module loading.

A:root@cluster-node1:~# apt-get install apache2 libapache2-mod-php5root@cluster-node2:~# /etc/init.d/apache2 restart

Note


Q: Add line to your fstab to mount the NFS exported directory from nfs-server to /var/www, and performthe mount.

A: In /etc/fstab:

192.168.82.156:/exports/www /var/www/ nfs defaults 0 0

root@cluster-node1:~# mount -a

Q: Check if the website is working.


Test Page 81

A:Hello from cluster-node1root@cluster-node1:~# curl http://localhostHello from cluster-node1root@cluster-node1:~# umount /var/www/

Q: Mirror the configuration on the second node.

A:root@cluster-node2:~# apt-get install apache2 libapache2-mod-php5root@cluster-node2:~# /etc/init.d/apache2 restartroot@cluster-node2:~# echo \>"192.168.82.156:/exports/www /var/www/ nfs defaults 0 0" > /etc/fstab root@cluster-node2:~# mount -aroot@cluster-node2:~# curl http://localhost/Hello from cluster-node2

Q: The service will be managed by the cluster. It must not be started during the init process. Remove theapache2 service from your runlevels (on both nodes).

A:root@cluster-node1:~# insserv -r apache2root@cluster-node2:~# insserv -r apache2

6.1.2.2. Configuring the cluster

In this section you're going to install and configure the cluster engine itself.

Q: Install pacemaker on both nodes.

A:root@cluster-node1:~# apt-get install pacemakerroot@cluster-node2:~# apt-get install pacemaker

Q: If you're running a Debian-based system ,enable the service in /etc/default/corosync.

A:root@cluster-node1:~# echo "START=yes" > /etc/default/corosyncroot@cluster-node2:~# echo "START=yes" > /etc/default/corosync

Q: Choose a private multicast address (begins with 238 or 239) and configure your two nodes to use thataddress for internal cluster communication. Also configure the cluster engine on both nodes to bind to theright network interface.

A: Edit /etc/corosync/corosync.conf interface section on node1:

interface { # The following values need to be set based on your environment ringnumber: 0 bindnetaddr: 192.168.82.0 mcastaddr: 239.12.13.14 mcastport: 5405}

and on node2 (or copy it over using ssh):


Test Page 82


Q: Use corosync-keygen to create a private key for your cluster. Remember that corosync-keygen needsentropy. Run a background task that writes the content of your hard drive to /dev/null to generateentropy. Don't forget to kill the task when corosync-keygen is done. Copy the key over to the second node.

A:root@cluster-node1:~# cat /dev/sda > /dev/null& corosync-keygen [1] 3594Corosync Cluster Engine Authentication key generator.Gathering 1024 bits for key from /dev/random.Press keys on your keyboard to generate entropy.Press keys on your keyboard to generate entropy (bits = 184).Press keys on your keyboard to generate entropy (bits = 248).Press keys on your keyboard to generate entropy (bits = 312).Press keys on your keyboard to generate entropy (bits = 376).Press keys on your keyboard to generate entropy (bits = 440).Press keys on your keyboard to generate entropy (bits = 504).Press keys on your keyboard to generate entropy (bits = 568).Press keys on your keyboard to generate entropy (bits = 632).Press keys on your keyboard to generate entropy (bits = 696).Press keys on your keyboard to generate entropy (bits = 760).Press keys on your keyboard to generate entropy (bits = 824).Press keys on your keyboard to generate entropy (bits = 888).Press keys on your keyboard to generate entropy (bits = 952).Press keys on your keyboard to generate entropy (bits = 1016).Writing corosync key to /etc/corosync/authkey.root@cluster-node1:~# kill %1[1]+ Terminated cat /dev/sda > /dev/nullroot@cluster-node1:~# scp /etc/corosync/authkey [email protected]:/etc/corosync

Q: On both nodes, (re) start the service.

A:root@cluster-node1:~# /etc/init.d/corosync restartroot@cluster-node2:~# /etc/init.d/corosync restart

Q: From any node, check the cluster status. You might have to wait a couple of seconds before the clustercames up.

root@cluster-node1:~# crm_mon --one-shot============Last updated: Tue Jun 26 12:52:37 2012Stack: openaisCurrent DC: cluster-node1 - partition with quorumVersion: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b2 Nodes configured, 2 expected votes0 Resources configured.============

Online: [ cluster-node1 cluster-node2 ]


Test Page 83

6.1.2.3. Configuring resources

In this section, you're going to configure cluster-managed resources and cluster internal settings.

Q: As your cluster only has two nodes, the quorum property doesn't make sense (quroum will be lost if oneout of the two nodes if lost). Configure the cluster to ignore quorum. Also disable the STONITH feature.

A:root@cluster-node1:~# crmcrm(live)# configureINFO: building help indexcrm(live)configure# property stonith-enabled="false" no-quorum-policy="ignore"crm(live)configure# commitcrm(live)configure# bye

Q: Choose an unallocated IP address within the virtual network pool. Create a primitive web-ip IPaddr resourceto handle it.

A:root@cluster-node1:~# crmcrm(live)# configurecrm(live)configure# primitive web-ip ocf:heartbeat:IPaddr params ip=192.168.82.200crm(live)configure# commit

Q: Add a lsb:apache2 resource named web-server. This resource will handle the start/stop of the apache webserver on the node its active/not active.

A:crm(live)configure# primitive web-server lsb:apache2crm(live)configure# commit

Q: Group the two previous resources in the website group.

A:crm(live)configure# group website web-ip web-server crm(live)configure# commit

Q: Check the cluster status and the resources location

A:crm(live)# status============Last updated: Tue Jun 26 13:23:11 2012Stack: openaisCurrent DC: cluster-node1 - partition with quorumVersion: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b2 Nodes configured, 2 expected votes1 Resources configured.============


Resource Group: website web-ip (ocf::heartbeat:IPaddr): Started cluster-node1 web-server (lsb:apache2): Started cluster-node1

Q: From another machine, try to access the website using the "web-ip" address.


Test Page 84

A:samuel@chickamauga ~ $ curl http://192.168.82.200/Hello from cluster-node1

Q: Manually migrate the resource group website to the other node and try to access the website again.

A:crm(live)# resourcecrm(live)resource# migrate website cluster-node2crm(live)resource# cd ..crm(live)# status============Last updated: Tue Jun 26 13:26:01 2012Stack: openaisCurrent DC: cluster-node1 - partition with quorumVersion: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b2 Nodes configured, 2 expected votes1 Resources configured.============


Resource Group: website web-ip (ocf::heartbeat:IPaddr): Started cluster-node2 web-server (lsb:apache2): Started cluster-node2

From another computer in the network:

samuel@chickamauga ~ $ curl http://192.168.82.200/Hello from cluster-node2

Q: Still from another computer in the network (a virtual machine or your host operating system), run a taskthat regulary queries the web server (the watch command can help). While the monitoring is going, unplugthe virtual interface of the cluster node where the group is currently running and see what happens. Plugthe virtual interface back while still monitoring the web site. What do you notice ?

A:samuel@chickamauga ~ $ watch -n1 curl -s http://192.168.82.200/

Shows

Every 1.0s: curl -s http://192.168.82.200/ Tue Jun 26 11:34:03 2012

Hello from cluster-node2

Unplug cluster-node2 virtual iface:



Plug the interface back:


Test Page 85



The resource comes back on the node where it was as soon as the failed node comes back online again.

Q: Configure the cluster to prevent it to migrate resources back automatically by setting the resource-sticknessdefault to 1000. Also make sure that no "location" constraints tries to put the website group to a particularnode.

A:crm(live)configure# delete cli-prefer-websitecrm(live)configure# rsc_defaults resource-stickiness="1000"crm(live)configure# commit

Q: Do the same test again (unplug and plug back the "active" node while monitoring the website) and seewhat happens.

A:samuel@chickamauga ~ $ watch -n1 curl -s http://192.168.82.200/

Shows



Unplug cluster-node2 virtual iface:



Plug the interface back:



The resource now stays where it is.

6.2. High available iSCSI-based SAN

In this section, you're going to make DRBD and Pacemaker work together to provide a high-available iSCSI SAN.DRBD will ensure data replication between the two nodes where as Pacemaker will manage Primary/Secondarystates and the iSCSI service (and IP) when a node becomes unavailable. The final setup will look like the following:


Test Page 86

You'll need three virtual machines: Two acting as cluster/drbd nodes and the third being the iSCSI initiator thatmounts the exported device. Prepare three virtual machines with the following hostnames:

• node1

• node2

• initiator

The DRBD nodes, node1 and node2 will need an additional hard drive to create the replicated volume. Add a 1GBhard drive to these virtual machines.

Also add an additional virtual NIC (host-only virtual network) on node1 and node2. Configure these NIC with fixedIP addresses: 172.17.2.11 for node1 and 172.17.2.22 for node2.

6.2.1. Setting up DRBD

Q: Add the linux lab repository and install appropriate packages on both nodes (Refer to the DRBD lab). Onceinstalled, probe the module.

A:root@node1:~# echo "deb http://repo.labo-linux.fr/debian/ binary/" >> /etc/apt/sources.listroot@node1:~# apt-get updateroot@node1:~# apt-get install drbd8-module-2.6.32-5-686 drbd8-utilsroot@node1:~# modprobe drbdroot@node2:~# echo "deb http://repo.labo-linux.fr/debian/ binary/" >> /etc/apt/sources.listroot@node2:~# apt-get updateroot@node2:~# apt-get install drbd8-module-2.6.32-5-686 drbd8-utilsroot@node2:~# modprobe drbd

Q: Prepare the additional disk (on both nodes): Create a single partition on it.


Test Page 87

A:root@node1:~# fdisk /dev/sdb Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-130, default 1): Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-130, default 130): Using default value 130


Calling ioctl() to re-read partition table.Syncing disks.root@node2:~# fdisk /dev/sdb Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-130, default 1): Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-130, default 130): Using default value 130

Command (m for help): wThe partition table has been altered!


Q: Configure DRBD: Create a storage resource that replicates over the 172.17.2.0/24 network.

A: Create the /etc/drbd.d/storage.res file on both nodes with the following content:


on node1 { address 172.17.2.11:7788; }

on node2 { address 172.17.2.22:7788; }}


Test Page 88

Q: Initialze the device and bring it up (on both nodes).

A:root@node1:~# drbdadm create-md storageWriting meta data...initializing activity logNOT initializing bitmapNew drbd meta data block successfully created.successroot@node1:~# drbdadm up storageroot@node2:~# drbdadm create-md storageWriting meta data...initializing activity logNOT initializing bitmapNew drbd meta data block successfully created.successroot@node2:~# drbdadm up storage

Q: Check the current drbd status (on any node).

A:root@node2:~# drbd-overview 1:storage/0 Connected Secondary/Secondary Inconsistent/Inconsistent C r-----

Q: Do the first forced replication by setting node1 as the primary node.

A:root@node1:~# drbdadm primary --force storage

Q: Check the current DRBD status from node1. What do you notice?

A:root@node1:~# drbd-overview 1:storage/0 SyncSource Primary/Secondary UpToDate/Inconsistent C r----- [====>...............] sync'ed: 27.1% (762524/1044124)K

NoteThe volume is sync'ing.

Q: Create an ext3 filesystem on the DRBD device, from the node that as the primary role.

A:root@node1:~# mkfs.ext3 /dev/drbd/by-res/storage/0

6.2.2. Setting up iSCSI

Q: Install appropriate packages to run a iSCSI target on the two nodes (Refer to the iSCSI lab as needed). If you'rerunning a Debian-based system, don't forget to enable the service in /etc/default/iscsitarget.

A:root@node1:~# apt-get install iscsitarget iscsitarget-dkmsroot@node1:~# echo "ISCSITARGET_ENABLE=true" > /etc/default/iscsitarget root@node2:~# apt-get install iscsitarget iscsitarget-dkmsroot@node2:~# echo "ISCSITARGET_ENABLE=true" > /etc/default/iscsitarget


Test Page 89

NoteThis is Debian-specific

Q: Configure the iscsi target to export the drbd device (on both node) as iqn.2012.06.net.utopia:storage. Betteruse the per-resource special file from /dev/drbd/by-res rather than the numbered ones such as /dev/drbd1.

A: Edit /etc/iet/ietd.conf as follows(on both nodes):

Target iqn.2012.06.net.utopia:storage Lun 0 Path=/dev/drbd/by-res/storage/0,Type=fileio

Q: The service will be managed by the cluster: Remove it from the nodes' runlevels.

A:root@node1:~# insserv -r iscsitargetroot@node2:~# insserv -r iscsitarget

6.2.3. Setting up the cluster

6.2.3.1. Configuring the cluster

Q: Install pacemaker on both nodes.

A:root@node1:~# apt-get install pacemakerroot@node2:~# apt-get install pacemaker

Q: If you're running a Debian-based system ,enable the service in /etc/default/corosync.

A:root@node1:~# echo "START=yes" > /etc/default/corosyncroot@node2:~# echo "START=yes" > /etc/default/corosync

Q: Choose a private multicast address (begins with 238 or 239) and configure your two nodes to use thataddress for internal cluster communication. Also configure the cluster engine on both nodes to bind to theright network interface.

A: Edit /etc/corosync/corosync.conf interface section on node1:


and on node2 (or copy it over using ssh):


Test Page 90


Q: Use corosync-keygen to create a private key for your cluster. Remember that corosync-keygen needsentropy. Run a background task that writes the content of your hard drive to /dev/null to generateentropy. Don't forget to kill the task when corosync-keygen is done. Copy the key over to the second node.

A:root@node1:~# cat /dev/sda > /dev/null& corosync-keygen [1] 3594Corosync Cluster Engine Authentication key generator.Gathering 1024 bits for key from /dev/random.Press keys on your keyboard to generate entropy.Press keys on your keyboard to generate entropy (bits = 184).Press keys on your keyboard to generate entropy (bits = 248).Press keys on your keyboard to generate entropy (bits = 312).Press keys on your keyboard to generate entropy (bits = 376).Press keys on your keyboard to generate entropy (bits = 440).Press keys on your keyboard to generate entropy (bits = 504).Press keys on your keyboard to generate entropy (bits = 568).Press keys on your keyboard to generate entropy (bits = 632).Press keys on your keyboard to generate entropy (bits = 696).Press keys on your keyboard to generate entropy (bits = 760).Press keys on your keyboard to generate entropy (bits = 824).Press keys on your keyboard to generate entropy (bits = 888).Press keys on your keyboard to generate entropy (bits = 952).Press keys on your keyboard to generate entropy (bits = 1016).Writing corosync key to /etc/corosync/authkey.root@node1:~# kill %1[1]+ Terminated cat /dev/sda > /dev/nullroot@node1:~# scp /etc/corosync/authkey [email protected]:/etc/corosync

Q: On both nodes, (re) start the service.

A:root@node1:~# /etc/init.d/corosync restartroot@node2:~# /etc/init.d/corosync restart

Q: From any node, check the cluster status. You might have to wait a couple of seconds before the clustercames up.

root@node1:~# crm_mon --one-shot============Last updated: Tue Jun 26 19:16:45 2012Stack: openaisCurrent DC: node1 - partition with quorumVersion: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b2 Nodes configured, 2 expected votes0 Resources configured.============

Online: [ node2 node1 ]


Test Page 91

6.2.3.2. Creating resources

Q: As your cluster only has two nodes, the quorum property doesn't make sense (quroum will be lost if oneout of the two nodes if lost). Configure the cluster to ignore quorum. Also disable the STONITH feature. Setthe resource stickiness to 1000.

A:root@node1:~# crmcrm(live)# configureINFO: building help indexcrm(live)configure# property stonith-enabled="false" no-quorum-policy="ignore"crm(live)configure# rsc_defaults resource-stickiness="1000"crm(live)configure# commitcrm(live)configure# bye

Q: Choose an unallocated IP address within the virtual network pool. Create a primitive "san-ip" IPaddrresource to handle it.

A:root@node1:~# crmcrm(live)# configurecrm(live)configure# primitive san-ip ocf:heartbeat:IPaddr params ip=192.168.82.200crm(live)configure# commit

Q: Add a drbddisk "drbd-service" resource that will manage the storage volume.

A:crm(live)configure# primitive drdb-service heartbeat:drbddiskcrm(live)configure# commit

Q: Add a iscsitarget "iscsi-service" resource to manage the iscsitarget service.

A:crm(live)configure# primitive iscsi-service lsb:iscsitargetcrm(live)configure# commit

Q: Group all these resource in a "san" group.

A:crm(live)configure# group san san-ip drbd-service iscsi-servicecrm(live)configure# commit

Q: Check the cluster status


Test Page 92

A:crm(live)# status============Last updated: Wed Jun 27 12:31:31 2012Stack: openaisCurrent DC: node1 - partition with quorumVersion: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b2 Nodes configured, 2 expected votes1 Resources configured.============


Resource Group: san san-ip (ocf::heartbeat:IPaddr): Started node2 drdb-service (heartbeat:drbddisk): Started node2 iscsi-service (lsb:iscsitarget): Started node2

6.2.3.3. Testing the High availability

Q: On the "initiator" virual machine, install all appropriate packages to connect to iscsi devices.

A:root@initiator:~# apt-get install open-iscsi

Note


Q: Do a iSCSI exported devices discovery on the san-ip you've chosen.

A:root@initiator:~# iscsiadm -m discovery -t st -p 192.168.82.200192.168.82.156:3260,1 iqn.2012.06.net.utopia:storage172.17.2.22:3260,1 iqn.2012.06.net.utopia:storage192.168.82.200:3260,1 iqn.2012.06.net.utopia:storage

Q: Connect to the exported device using the san-ip portal.

A:root@initiator:~# iscsiadm -m node --targetname "iqn.2012.06.net.utopia:storage" \>--portal "192.168.82.200:3260" --loginLogging in to [iface: default, target: iqn.2012.06.net.utopia:storage, portal: 192.168.82.200,3260]Login to [iface: default, target: iqn.2012.06.net.utopia:storage, portal: 192.168.82.200,3260]: successful

Q: Mount the filesystem on /mnt.

A:root@initiator:~# mount \>/dev/disk/by-path/ip-192.168.82.200\:3260-iscsi-iqn.2012.06.net.utopia\:storage-lun-0 /mnt/

Q: Create a file on that filesystem, with some random content in it.


Test Page 93

A:root@initiator:~# echo "highly available content" > /mnt/highly-available-file

Q: Disconnect the node where the SAN is running from the network it uses to communictate with the client.

A:root@node1:~# crm_mon --one-shot============Last updated: Wed Jun 27 12:45:09 2012Stack: openaisCurrent DC: node1 - partition with quorumVersion: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b2 Nodes configured, 2 expected votes1 Resources configured.============


Resource Group: san san-ip (ocf::heartbeat:IPaddr): Started node2 drdb-service (heartbeat:drbddisk): Started node2 iscsi-service (lsb:iscsitarget): Started node2

The SAN services are running on node2. Just disconnect this node.

Q: Try to read your file from the initiator.

A:root@initiator:~# cat /mnt/highly-available-file highly available content


Test Page 94

7. SELinuxIn this lab, you're going to work with SELinux. You only need a single virtual machine, in its standard configuration.

7.1. Enabling SELinux

Q: Install SELinux packages on your system as well as the auditing daemon.

A:root@debian-selinux:~# apt-get install selinux-policy-default selinux-basics selinux-utils auditd

Note


Q: Check current SELinux status

A:root@debian-selinux:~# sestatus SELinux status: disabled

Q: Activate SELinux using the selinux-activate script and proceed to reboot. Note this script is specific toDebian-based systems.

A:root@debian-selinux:~# selinux-activate Activating SE LinuxGenerating grub.cfg ...Found linux image: /boot/vmlinuz-2.6.32-5-686Found initrd image: /boot/initrd.img-2.6.32-5-686doneSE Linux is activated. You may need to reboot now.

Note

This is Debian specific

Q: Check current SELinux status. What's the current mode? What does it means?

A:root@debian-selinux:~# sestatus SELinux status: enabledSELinuxfs mount: /selinuxCurrent mode: permissiveMode from config file: permissivePolicy version: 24Policy from config file: default


Test Page 95

SELinux is in permissive mode. It means that all denials will be logged but not applied. e.i your system isrunning as it would without SELinux being enabled.

Q: Show your current SELinux context. What does it means ?

A:root@debian-selinux:~# id -Zunconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

You currently have unconfined role and domain. It means that SELinux "rules" won't apply to you and tothe process you'll run.

Q: List all running processes and their security context. What do you notice ?

A:root@debian-selinux:~# ps aux -ZLABEL USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMANDsystem_u:system_r:init_t:s0 root 1 0.1 0.2 2032 680 ? Ss 16:36 0:01 init [2] system_u:system_r:kernel_t:s0 root 2 0.0 0.0 0 0 ? S 16:36 0:00 [kthreadd]system_u:system_r:kernel_t:s0 root 3 0.0 0.0 0 0 ? S 16:36 0:00 [migration/0]system_u:system_r:portmap_t:s0 daemon 780 0.0 0.1 1808 496 ? Ss 16:36 0:00 /sbin/portmapsystem_u:system_r:rpcd_t:s0 statd 792 0.0 0.3 1936 768 ? Ss 16:36 0:00 /sbin/rpc.statdsystem_u:system_r:auditd_t:s0 root 949 0.0 0.3 11732 848 ? S<sl 16:36 0:00 /sbin/auditdsystem_u:system_r:audisp_t:s0 root 951 0.0 0.2 10052 736 ? S<sl 16:36 0:00 /sbin/audispdsystem_u:system_r:kernel_t:s0 root 953 0.0 0.0 0 0 ? S 16:36 0:00 [kauditd]system_u:system_r:syslogd_t:s0 root 955 0.0 0.6 27684 1756 ? Sl 16:36 0:00 /usr/sbin/rsyslogd -c4system_u:system_r:apmd_t:s0 root 992 0.0 0.2 1704 592 ? Ss 16:36 0:00 /usr/sbin/acpidsystem_u:system_r:crond_t:s0-s0:c0.c1023 daemon 1004 0.0 0.1 2160 416 ? Ss 16:36 0:00 /usr/sbin/atdsystem_u:system_r:sshd_t:s0-s0:c0.c1023 root 1433 0.0 0.3 5496 976 ? Ss 16:37 0:00 /usr/sbin/sshdunconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 root 1504 0.0 0.4 3872 1044 pts/0 R+ 16:55 0:00 ps aux -Z

Note

Reduced output

Processes running in kernel space runs within the kernel_t SElinux domain whereas daemons runs with adedicated domain. User (incl. root) processes runs in the unconfined domain.

Q: Switch to the enforcing mode.

A:root@debian-selinux:~# setenforce 1

Note

This won't survive a reboot.

7.2. Working with booleans

Q: List all available booleans.


Test Page 96

A:root@debian-selinux:~# getsebool -a[...]user_direct_mouse --> offuser_dmesg --> offuser_manage_dos_files --> onuser_ping --> offuser_rw_noexattrfile --> offuser_tcp_server --> offuser_ttyfile_stat --> off

Q: The user_dmesg boolean is set to off/false. User should't be able to use the dmesg command. Try to runit as supinfo. Do you get the expected result? Why?

A:supinfo@debian-selinux:~$ dmesg | tail -n 4[ 5.296247] type=1400 audit(1341585414.507:4): avc: denied { append } for pid=618 comm="mount" name="mtab" dev=sda1 ino=268024 scontext=system_u:system_r:mount_t:s0 tcontext=system_u:object_r:etc_t:s0 tclass=file[ 5.341905] loop: module loaded[ 6.352018] eth0: link up[ 17.482012] eth0: no IPv6 routers present

Although the user_dmesg boolean has been set to prevent users from using the dmesg command, you'veno problem to run it as supinfo. Maybe the Unix user -> SELinux user mapping as something to do with that.

Q: Check supinfo's current security context and existing Unix -> SELinux user mappings. What do you notice.

A:supinfo@debian-selinux:~$ id -Zunconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023root@debian-selinux:~# semanage login -l

Login Name SELinux User MLS/MCS Range

__default__ unconfined_u s0-s0:c0.c1023 root unconfined_u s0-s0:c0.c1023 system_u system_u s0-s0:c0.c1023

Both root and regular users (__default__) are mapped to the unconfined_u SELinux user. Therefore, from aSELinux perspective, regular users cannot be more restricted than root.

Q: Change the default user mapping from unconfined_u to user_u for the default policy.

A:root@debian-selinux:~# semanage login -m -S default -r s0 -s "user_u" __default__

Q: Check your security context and try to dmesg again as supinfo (login/logout as needed).

A:supinfo@debian-selinux:~$ id -Zuser_u:user_r:user_t:s0supinfo@debian-selinux:~$ dmesg klogctl: Permission denied

The restriction now works as expected.

Q: Turn the user_dmesg boolean on and try to dmesg again as supinfo.


Test Page 97

A:root@debian-selinux:~# setsebool user_dmesg 1supinfo@debian-selinux:~$ dmesg | tail -n 2[ 3904.943279] SELinux: 6 users, 6 roles, 1209 types, 46 bools, 1 sens, 1024 cats[ 3904.943285] SELinux: 77 classes, 27184 rules

Q: Do the same with the user_ping boolean.

A:supinfo@debian-selinux:~$ ping google.frping: icmp open socket: Permission deniedsupinfo@debian-selinux:~$ ping google.frPING google.fr (173.194.67.94) 56(84) bytes of data.64 bytes from wi-in-f94.1e100.net (173.194.67.94): icmp_req=1 ttl=128 time=36.0 ms

Q: Try to su to root and to halt the system. What happens? Why?

A:supinfo@debian-selinux:~$ suPassword: root@debian-selinux:/home/supinfo# haltshutdown: warning: cannot open /var/run/shutdown.pidshutdown: /dev/initctl: Permission deniedshutdown: cannot execute /sbin/initshutdown: /dev/initctl: Permission deniedroot@debian-selinux:/home/supinfo# id -Zuser_u:user_r:user_t:s0

Even if the su command successuflly gives you id 0, you're still in the user_t context. This context cannot domuch things, and especially not halting the system. You can still get the unconfined_t context by openinga session as root from the login prompt.

Q: Change back the default mapping to unconfined_u.

A:root@debian-selinux:~# id -Zunconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023root@debian-selinux:~# semanage login -m -S default -r s0 -s "unconfined_u" __default__

7.3. Using audit2allow

Be sure your system is in enforcing mode before going any further.

Q: Set the default user mapping to user_u.

A:root@debian-selinux:~# semanage login -m -S default -r s0 -s "user_u" __default__

Q: On a second console, open a session as supinfo and su to root. Check you've user_r role.

A:supinfo@debian-selinux:~$ suPassword: root@debian-selinux:/home/supinfo# id -Zuser_u:user_r:user_t:s0

Q: On the first console erase blank off the /var/log/audit/audit.log file.

A:root@debian-selinux:/home/supinfo# cat /dev/null > /var/log/audit/audit.log


Test Page 98

Q: Disable all active dontaudit rules. This will make your upcoming task a lot easier.

A:root@debian-selinux:~/local# semodule -DB

Q: Setup One: On the console were you're logged as root with the user_r role, try to halt the system using the/sbint/init 0 command. This will try to perform the appropriate syscalls and fail generating 'denied' avc logmessages for any syscall that have not been allowed yet.

Note

You'll have to do this step more than once.

A:root@debian-selinux:~# /sbin/init 0bash: /sbin/init: Permission denied

Q: On the first console, use audit2allow to generate a set of rules allowing what has just been denied (haltingthe system). The rules belong to a module named local. Save the rules in a local.te file.

A:root@debian-selinux:~# audit2allow -m local -i /var/log/audit/audit.log > local.te

Q: Ensure the generated rules only refer to user_t domain processes halting the system.

A: The local.te file should have the following content:

module local 1.0;

require { type init_exec_t; type var_run_t; type syslogd_t; type initctl_t; type devlog_t; type user_t; class file { write getattr read unlink open execute execute_no_trans }; class capability { setuid dac_override }; class fifo_file { write open }; class sock_file write; class unix_dgram_socket sendto; class dir { write remove_name };}

#============= user_t ==============allow user_t devlog_t:sock_file write;allow user_t init_exec_t:file { read execute open getattr execute_no_trans };allow user_t initctl_t:fifo_file { write open };allow user_t self:capability { setuid dac_override };allow user_t syslogd_t:unix_dgram_socket sendto;allow user_t var_run_t:dir { write remove_name };allow user_t var_run_t:file { read write getattr unlink open };


Test Page 99

Note

You won't get this result in the very first step. This is perfectly normal. New syscalls will become"available" as you allow them. If syscall B depends on syscall A success' it won't be tried untilsyscall A got allowed. You'll allow syscall A in the first run and syscall B in the second try, andso on.

Q: Build the module into local.mod.

A:root@debian-selinux:~# checkmodule -M -m -o local.mod local.techeckmodule: loading policy configuration from local.techeckmodule: policy configuration loadedcheckmodule: writing binary representation (version 10) to local.mod

Q: Package the module into local.pp.

A:root@debian-selinux:~# semodule_package -o local.pp -m local.mod

Q: Load the module package.

A:root@debian-selinux:~# semodule -i local.pp

Q: Try to halt the system again from the console where you're running under the user_t domain. If it doesn'twork, start again from step one.

A:root@debian-selinux:/home/supinfo# id -Zuser_u:user_r:user_t:s0root@debian-selinux:/home/supinfo# /sbin/init 0

7.4. Creating a policy

In this section, you're going to create a new policy and set appropriate labels on executables and files to confinea process: The apache web server.

Ensure that users are mapped to the unconfined domain before going any further. Refer to previous questions tochange the default user mapping if necessary.

Q: Install SELinux policy developpement packages.

A:root@debian-selinux:~# apt-get install selinux-policy-dev

Note

This is Debian-Specific


Test Page 100

Q: Install Apache2 on your system.

A:root@debian-selinux:~# apt-get install apache2

NoteThis is Debian specific.

Q: Ensure the service is running and check its process security context. What do you notice? Stop the service.

A:root@debian-selinux:~# /etc/init.d/apache2 restartroot@debian-selinux:~# ps aux -Z | grep apache2unconfined_u:unconfined_r:unconfined_t:s0 root 2808 0.0 1.0 5428 2708 ? Ss 14:50 0:00 /usr/sbin/apache2 -k startunconfined_u:unconfined_r:unconfined_t:s0 www-data 2814 0.0 0.7 5200 1832 ? S 14:50 0:00 /usr/sbin/apache2 -k startunconfined_u:unconfined_r:unconfined_t:s0 www-data 2815 0.0 0.8 226840 2284 ? Sl 14:50 0:00 /usr/sbin/apache2 -k startunconfined_u:unconfined_r:unconfined_t:s0 www-data 2816 0.0 0.9 226840 2304 ? Sl 14:50 0:00 /usr/sbin/apache2 -k startunconfined_u:unconfined_r:unconfined_t:s0 root 2873 0.0 0.2 3300 748 pts/0 S+ 14:50 0:00 grep apache2root@debian-selinux:~# /etc/init.d/apache2 stop

The service is running unconfined.

Q: Use the policygentool command to generate a policy module called apache2 for the /usr/bin/apache2binary.

A:root@debian-selinux:~/apache2# policygentool apache2 /usr/sbin/apache2

This tool generate three files for policy development, A Type Enforcement (te)file, a File Context (fc), and a Interface File(if). Most of the policy ruleswill be written in the te file. Use the File Context file to associate filepaths with security context. Use the interface rules to allow other protecteddomains to interact with the newly defined domains.

After generating these files use the /usr/share/selinux/POLICY-NAME/include/Makefile tocompile your policy package. Then use the semodule tool to load it.

# /usr/bin/policygentool myapp /usr/bin/myapp# echo 'HEADERDIR:=/usr/share/selinux/refpolicy-targeted/include' > Makefile# echo 'include $(HEADERDIR)/Makefile' >> Makefile# make# semodule -l myapp.pp# restorecon -R -v /usr/bin/myapp "all files defined in myapp.fc"

Now you can turn on permissive mode, start your application and avc messageswill be generated. You can use audit2allow to help translate the avc messagesinto policy.

# setenforce 0# /etc/init.d/myapp start# audit2allow -R -i /var/log/audit/audit.log

Return to continue:

If the module uses pidfiles, what is the pidfile called?/var/run/apache2.pidIf the module uses logfiles, where are they stored?/var/log/apache2(/.*)?If the module has var/lib files, where are they stored?

Does the module have a init script? [yN]yDoes the module use the network? [yN]y

Q: Create a link (or copy) the Makefile supplied by the SELinux policy developpment package into the currentdirectory. Use your package management system to find where is that Makefile.


Test Page 101

A:root@debian-selinux:~/apache2# ln -s /usr/share/doc/selinux-policy-dev/examples/Makefile

Q: Compile and load the apache2 policy module you've created with policygentool.

A:root@debian-selinux:~/apache2# makeroot@debian-selinux:~/apache2# semodule -i apache2.pp

Q: Relabel the actual apache2 service binary to apache2_exect_t.

A:root@debian-selinux:~# ls -lZ /usr/sbin/apache2lrwxrwxrwx. 1 root root system_u:object_r:bin_t:s0 33 Jun 28 14:14 /usr/sbin/apache2 -> ../lib/apache2/mpm-worker/apache2root@debian-selinux:~# chcon -t apache2_exec_t /usr/lib/apache2/mpm-worker/apache2 root@debian-selinux:~# ls -lZ /usr/lib/apache2/mpm-worker/apache2 -rwxr-xr-x. 1 root root system_u:object_r:apache2_exec_t:s0 406784 Apr 1 08:40 /usr/lib/apache2/mpm-worker/apache2

Or:

root@debian-selinux:/home/supinfo# chcon -t apache2_exec_t /usr/sbin/apache2root@debian-selinux:/home/supinfo# ls -lZ /usr/lib/apache2/mpm-worker/apache2 -rwxr-xr-x. 1 root root system_u:object_r:apache2_exec_t:s0 406784 Apr 1 08:40 /usr/lib/apache2/mpm-worker/apache2root@debian-selinux:/home/supinfo# ls -lZ /usr/sbin/apache2lrwxrwxrwx. 1 root root system_u:object_r:bin_t:s0 33 Jun 28 14:14 /usr/sbin/apache2 -> ../lib/apache2/mpm-worker/apache2

Q: Relabel anything below /var, in verbose mode. What do you notice?

A:root@debian-selinux:~/apache2# restorecon -vR /var/restorecon reset /var/log/apache2 context system_u:object_r:var_log_t:s0->system_u:object_r:apache2_var_log_t:s0restorecon reset /var/log/apache2/other_vhosts_access.log context user_u:object_r:var_log_t:s0->system_u:object_r:apache2_var_log_t:s0restorecon reset /var/log/apache2/error.log context user_u:object_r:var_log_t:s0->system_u:object_r:apache2_var_log_t:s0restorecon reset /var/log/apache2/access.log context user_u:object_r:var_log_t:s0->system_u:object_r:apache2_var_log_t:s0

Files got relabled according to context definitions from apache2.fc.

Q: Restart the service and check the process context. What do you notice?

A:root@debian-selinux:~/apache2# /etc/init.d/apache2 restartroot@debian-selinux:~/apache2# ps aux -Z | grep apache2unconfined_u:system_r:apache2_t:s0 root 3071 0.0 1.0 5428 2708 ? Ss 14:56 0:00 /usr/sbin/apache2 -k startunconfined_u:system_r:apache2_t:s0 www-data 3076 0.0 0.7 5200 1832 ? S 14:56 0:00 /usr/sbin/apache2 -k startunconfined_u:system_r:apache2_t:s0 www-data 3077 0.0 0.8 226840 2284 ? Sl 14:56 0:00 /usr/sbin/apache2 -k startunconfined_u:system_r:apache2_t:s0 www-data 3078 0.0 0.9 226840 2304 ? Sl 14:56 0:00 /usr/sbin/apache2 -k start

The service is now running in the apache2_t confined domain.

Q: Set SELinux to enforcing mode.

A:root@debian-selinux:~/apache2# setenforce 1

Q: Try to restart apache2. Does it works? Why?

A:root@debian-selinux:~/apache2# /etc/init.d/apache2 restart

It'll fail, because some apache2 syscalls will got denied by SELinux. In the permissive mode, SELinux onlylogs denied action while in the enforcing mode it logs denied actions and actually denies them.

Q: Use the avc log messages to add appropriate allow directives in the apache2.te file. This involves trying tostart the service check denied actions from the logs, adding appropriate directives in the apache2.te file,rebuilding the policy, reloading it and trying to start the service again. Repeat until you've allowed all neededactions. audit2allow can also help you in this task.

A: Here is a working apache2.te:


Test Page 102

policy_module(apache2,1.0.0)

require { type sysctl_t; type sysctl_kernel_t;}

########################################## Declarations#

type apache2_t;type apache2_exec_t;domain_type(apache2_t)init_daemon_domain(apache2_t, apache2_exec_t)

# pid filestype apache2_var_run_t;files_pid_file(apache2_var_run_t)

# log filestype apache2_var_log_t;logging_log_file(apache2_var_log_t)

########################################## apache2 local policy## Check in /etc/selinux/refpolicy/include for macros to use instead of allow rules.

# Some common macros (you might be able to remove some)files_read_etc_files(apache2_t)libs_use_ld_so(apache2_t)libs_use_shared_libs(apache2_t)miscfiles_read_localization(apache2_t)## internal communication is often done using fifo and unix sockets.allow apache2_t self:fifo_file { read write };allow apache2_t self:unix_stream_socket create_stream_socket_perms;

# pid fileallow apache2_t apache2_var_run_t:file manage_file_perms;allow apache2_t apache2_var_run_t:sock_file manage_file_perms;allow apache2_t apache2_var_run_t:dir rw_dir_perms;files_pid_filetrans(apache2_t,apache2_var_run_t, { file sock_file })

# log filesallow apache2_t apache2_var_log_t:file { create_file_perms append };allow apache2_t apache2_var_log_t:sock_file create_file_perms;allow apache2_t apache2_var_log_t:dir { rw_dir_perms setattr };logging_log_filetrans(apache2_t,apache2_var_log_t,{ sock_file file dir })

## Networking basics (adjust to your needs!)sysnet_dns_name_resolve(apache2_t)corenet_tcp_sendrecv_all_if(apache2_t)corenet_tcp_sendrecv_all_nodes(apache2_t)corenet_tcp_sendrecv_all_ports(apache2_t)corenet_non_ipsec_sendrecv(apache2_t)corenet_tcp_connect_http_port(apache2_t)#corenet_tcp_connect_all_ports(apache2_t)## if it is a network daemon, consider these:#corenet_tcp_bind_all_ports(apache2_t)#corenet_tcp_bind_all_nodes(apache2_t)allow apache2_t self:tcp_socket { listen accept };

#Manual entriesallow apache2_t http_port_t:tcp_socket name_bind;allow apache2_t node_t:tcp_socket node_bind;


Test Page 103

allow apache2_t self:capability { kill net_bind_service setgid setuid chown };allow apache2_t self:process { sigchld sigkill sigstop signull signal fork};allow apache2_t self:sem { create setattr read destroy write unix_write };allow apache2_t sysctl_t:dir {read search};allow apache2_t sysctl_kernel_t:file {read};allow apache2_t sysctl_kernel_t:dir {read search};

# Init script handlinginit_use_fds(apache2_t)init_use_script_ptys(apache2_t)domain_use_interactive_fds(apache2_t)

Q: Try to get the default "It Works" content from the web server. Does it works? Why?

A:root@debian-selinux:~/apache2# curl http://localhost/<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"><html><head><title>403 Forbidden</title></head><body><h1>Forbidden</h1><p>You don't have permission to access /on this server.</p><hr><address>Apache/2.2.16 (Debian) Server at localhost Port 80</address></body></html>

It doesn't works. If you look at the avc log, you'll see that the apache2 process got denied access to thedirectories and files representing web content. This is perfectly normal: You didn't have authorized that yet.

Q: Add a line in apache2.fc that assigns the apache2_sys_content_t context to all files in /var/www andbelow.

A: Your apache2.fc file should look like this one:

# apache2 executable will have:# label: system_u:object_r:apache2_exec_t# MLS sensitivity: s0# MCS categories: <none>

/usr/sbin/apache2 -- gen_context(system_u:object_r:apache2_exec_t,s0)/var/run/apache2.pid gen_context(system_u:object_r:apache2_var_run_t,s0)/var/log/apache2(/.*)? gen_context(system_u:object_r:apache2_var_log_t,s0)/var/www(/.*)? gen_context(system_u:object_r:apache2_sys_content_t,s0)

Q: Also edit apache2.te to allow it to browse and read apache2_sys_content_t tagged directories/files.

A: Add the following to apache2.te:

type apache2_sys_content_t;

#content to serveallow apache2_t apache2_sys_content_t:dir {getattr search};allow apache2_t apache2_sys_content_t:file {open read getattr};

Q: Compile and load the policy. Relabel /var. Switch enforcing mode off and on before/after relabeling. Youhave to do that (mode switching) because you've only authorized the apache2_sys_contetn_t domain tomanipulate /var/www. Even unconfined_t can't do anything on these files due to the lack of authorizations.


Test Page 104

The alternative to switching SELinux off/on before/after relabeling is to add explicit authorizations forcontexts like unconfined_t and setfiles_t.

A:root@debian-selinux:~/apache2# make && semodule -i apache2.pproot@debian-selinux:~/apache2# setenforce 0root@debian-selinux:~/apache2# restorecon -Rv /var/root@debian-selinux:~/apache2# setenforce 1

Q: Try to query to local web server again.

A:root@debian-selinux:~/apache2# curl http://localhost/<html><body><h1>It works!</h1><p>This is the default web page for this server.</p><p>The web server software is running but no content has been added, yet.</p></body></html>


Test Page 105

8. Quality Of Service8.1. Bandwidth fairness

In this section, you're in a scenario where your router provide internet access for clients in the private network.However, with the default configuration on client can get all the bandwith and starve others.

For this scenario, you'll need three virtual machines, connected like in the following schema:

Prepare these three virtual machines with appropriate hostnames and network configuration. Note that the routerwill need two NIC: The first one plugged to the NAT'ed virtual network and the second one plugged to the host-only virtual network.

8.1.1. Routing/NAT

Q: Enable routing on debian-router.

A:root@router:~# echo 1 > /proc/sys/net/ipv4/ip_forward

NoteThis won't survive a reboot. You can also edit the sysctl.conf file to ensure persistance.

Q: On Debian-router, create a rule that NAT all incoming traffic from the host-only network to the VMwareNAT'ed virtual network.

A:root@router:~# iptables -t nat -A POSTROUTING -s 172.17.2.0/24 -j MASQUERADE

Q: From the client machines, try to reach something on the internet (You might need to adjust DNS settingsbefore doing so).

A:root@client1:~# echo "nameserver 8.8.8.8" > /etc/resolv.confsupinfo@client1:~$ ping google.comPING google.com (173.194.34.38) 56(84) bytes of data.64 bytes from par03s03-in-f6.1e100.net (173.194.34.38): icmp_req=1 ttl=127 time=23.1 msroot@client2:~# echo "nameserver 8.8.8.8" > /etc/resolv.conf supinfo@client2:~$ ping google.comPING google.com (173.194.34.38) 56(84) bytes of data.64 bytes from par03s03-in-f6.1e100.net (173.194.34.38): icmp_req=1 ttl=127 time=25.4 ms


Test Page 106

Q: On the router, add a traffic control rule that limits the bandwidth available to clients to 100 kbits. This willhelp simulate a low performance link and enlighten how a client can eat all the bandwidth.

A:root@router:~# tc qdisc add dev eth1 root handle 1: htb default 10root@router:~# tc class add dev eth1 parent 1:0 classid 1:10 htb rate 100kbit ceil 100kbit

8.1.2. Natural bandwidth distribution

Q: On both clients, start a big HTTP download, like the full source code of the latest Linux kernel. What doyou notice ?

A:supinfo@client1:~$ wget http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2--2012-07-11 17:21:20-- http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2Resolving www.kernel.org... 149.20.4.69, 149.20.20.133Connecting to www.kernel.org|149.20.4.69|:80... connected.HTTP request sent, awaiting response... 200 OKLength: 80960577 (77M) [application/x-bzip2]Saving to: linux-3.5-rc6.tar.bz2.11

0% [ ] 39,359 3.40K/s eta 6h 46m

And:

supinfo@client2:~$ wget http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2--2012-07-11 17:21:21-- http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2Resolving www.kernel.org...

The first download uses all the available bandwidth: The second client can't even perform a responsive DNSlookup.

8.1.3. Fair bandwidth distribution

Q: On the router, add a traffic control rule that adds fairness queuing with a hash function updated every 10seconds.

A:root@router:~# tc qdisc add dev eth1 parent 1:10 handle 110: sfq perturb 10

Q: Try to start a big download on both clients again and see what happens.

A:supinfo@client1:~$ wget http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2--2012-07-11 17:29:09-- http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2Resolving www.kernel.org... 149.20.20.133, 149.20.4.69Connecting to www.kernel.org|149.20.20.133|:80... connected.HTTP request sent, awaiting response... 200 OKLength: 80960577 (77M) [application/x-bzip2]Saving to: ###linux-3.5-rc6.tar.bz2.12###

0% [ ] 54,799 2.96K/s eta 5h 59m

And:

supinfo@client2:~$ wget http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2--2012-07-11 17:29:07-- http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.5-rc6.tar.bz2Resolving www.kernel.org... 149.20.20.133, 149.20.4.69Connecting to www.kernel.org|149.20.20.133|:80... connected.HTTP request sent, awaiting response... 200 OKLength: 80960577 (77M) [application/x-bzip2]Saving to: ###linux-3.5-rc6.tar.bz2.7###

0% [ ] 51,603 2.43K/s eta 8h 11m


Test Page 107

The bandwidth is now fairly shared by both clients.

8.2. Traffic Shaping

In this section, you're going to configure the router/gateway to shape traffic according to the following schema:

Remember that you're shapping outgoing traffic. For example the HTTP traffic shaping rule will by applied torequests made by clients, not incoming server response. Your router has two interfaces: One is connected to theinternet and the other one is connected to the private client's network. By applying traffic control rules to theprivate network interface, you're limiting outgoing rate to that network and thus its downloading capacity. In thisscenario, you want to throttle and control outgoing traffic to the internet. Therefore, you'll apply these rules tothe interface connected to the internet.

8.2.1. Creating the tree

Q: Delete any pre-existing root qdisc on eth0 iface.

A:root@router:~# tc qdisc del dev eth0 root

Q: Apply a root HTB qdisc on eth0. This qdisc should have handle 1:0 and redirect traffic to the class havingthe 1:40 handle.

A:root@router:~# tc qdisc add dev eth0 root handle 1:0 htb default 40

Q: Add the 1:1 htb class to the tree, using values from the schema. This class represent the "full power" of thevirtual uplink you're working with. Note that this link is symmetrical.

A:root@router:~# tc class add dev eth0 parent 1:0 classid 1:1 htb rate 4096 ceil 4096

Q: Add the "interactive" class as well, with a priority of 1.


Test Page 108

A:root@router:~# tc class add dev eth0 parent 1:1 classid 1:10 \>htb rate 1024 ceil 2048 burst 32k prio 1

Q: Finally add the leaf of the tree branch: A SFQ queue discipline that will ensure fair bandwidth distributionbetween connections.

A:root@router:~# tc qdisc add dev eth0 parent 1:10 handle 110 sfq

Q: Perform analog steps for other branches.Don't forget to increment priority when going from leftmost torightmost tree branches.

A:root@router:~# tc class add dev eth0 parent 1:1 classid 1:20 \>htb rate 256 ceil 4096 burst 512 prio 2root@router:~# tc qdisc add dev eth0 parent 1:20 handle 120 sfqroot@router:~# tc class add dev eth0 parent 1:1 classid 1:30 \>htb rate 2048 ceil 4096 burst 1024 prio 3root@router:~# tc qdisc add dev eth0 parent 1:30 handle 130 sfqroot@router:~# tc class add dev eth0 parent 1:1 classid 1:40 \>htb rate 768 ceil 4096 burst 1024 prio 4root@router:~# tc qdisc add dev eth0 parent 1:40 handle 140 sfq

Q: Use tc to show classes and qdiscs in your tree.

A:root@router:~# tc class show dev eth0class htb 1:1 root rate 4096bit ceil 4096bit burst 1600b cburst 1600b class htb 1:10 parent 1:1 leaf 110: prio 1 rate 1024bit ceil 2048bit burst 32Kb cburst 1600b class htb 1:20 parent 1:1 leaf 120: prio 2 rate 256bit ceil 4096bit burst 512b cburst 1600b class htb 1:30 parent 1:1 leaf 130: prio 3 rate 2048bit ceil 4096bit burst 1Kb cburst 1600b class htb 1:40 parent 1:1 leaf 140: prio 4 rate 768bit ceil 4096bit burst 1023b cburst 1600b root@router:~# tc qdisc show dev eth0qdisc htb 1: root refcnt 2 r2q 10 default 40 direct_packets_stat 467qdisc sfq 110: parent 1:10 limit 127p quantum 1514b qdisc sfq 120: parent 1:20 limit 127p quantum 1514b qdisc sfq 130: parent 1:30 limit 127p quantum 1514b qdisc sfq 140: parent 1:40 limit 127p quantum 1514b

8.2.2. Marking packets

In the previous section, you've set a traffic control tree. However at the moment all packets are going to the defaultclass (1:40). You know need to tell the system how to put packets into classes. This will be done using the firewallto mark packets and then tc rules to put packets having a given mark to a given class.

Packet marks are integers. You're going to mark packets with the minor number of the class they belong.

Q: Use iptable's connection tracking module to mark packets going to the DNS and NTP udp ports as "10".

A:root@router:~# iptables -t mangle -A POSTROUTING -o eth0 -p udp -m multiport \>--dports 53,123 -j CONNMARK --set-mark 10

Q: Do the same with packets going to the port 80: They must be marked 30.

A:root@router:~# iptables -t mangle -A POSTROUTING -o eth0 -p tcp \>--dport 80 -j CONNMARK --set-mark 30

Q: Add a rule that propagate connection marks to actual packets.

A:root@router:~# iptables -t mangle -A POSTROUTING -j CONNMARK --restore-mark


Test Page 109

Q: Now add a last-in-order rule that will mark (actual packet mark, not connection tracking mark) TCP packetshaving the ACK flag (and only this flag) set to 20. Only select "empty" (no data) packets: Their size is between40 and 64 bytes.

A:root@router:~# iptables -t mangle -A POSTROUTING -p tcp \>--tcp-flags URG,ACK,PSH,RST,SYN,FIN ACK -m length --length 40:64 -j MARK --set-mark 20

8.2.3. Glue

You know have a traffic control tree with classes in one hand and packets marked with the minor of these classesin the other hand. The last step is to glue this together by add filters to the traffic control tree to redirect packetswith a certain mark to the matching class.

Attach all filters to the root (1:0) qdisc.

Q: Add a filter that uses the fw module to selects packets having the 10 mark and sends them to the 1:10 class.This filter has a priority of one.

A:root@router:~# tc filter add dev eth0 parent 1:0 protocol ip prio 1 handle 10 fw flowid 1:10

Q: Add filters for the 2 remaining mark/classes. Don't forget to increment to filter priority for each new filter.

A:root@router:~# tc filter add dev eth0 parent 1:0 protocol ip prio 2 handle 20 fw flowid 1:20root@router:~# tc filter add dev eth0 parent 1:0 protocol ip prio 3 handle 30 fw flowid 1:30


Test Page 110

9. IPSec9.1. Host to host

In this chapter, you're going to setup a IPSec connection between two network hosts and experiment with variouskeying mecanisms. Your network setup will look like the following schema:

Prepare two virtual machines with appropriate netwok configuration (these hosts should be able to talk to eachother) and hostnames.

9.1.1. Static Keying

Q: Install ipsec-tools on both virtual machines.

A:root@debian-ipsec1:~# apt-get install ipsec-toolsroot@debian-ipsec2:~# apt-get install ipsec-tools

Note

This is Debian-specific.

Q: Generate two 128 bit keys for AH hmac-md5 encryption.

A:root@debian-ipsec1:~# dd if=/dev/urandom count=1 bs=16 2>/dev/null | xxd -p4e331c97f663cd11e4f9885995020588root@debian-ipsec1:~# dd if=/dev/urandom count=1 bs=16 2>/dev/null | xxd -pa9e6d3bf3bc69d7ed89dc88b97b0e669

Q: Generate two 192 bit keys for ESP 3des-cbc encryption.

A:root@debian-ipsec1:~# dd if=/dev/urandom count=1 bs=24 2>/dev/null | xxd -pcca621f764a640fb8efaa7527df2fdb6ad7163eb99dd299froot@debian-ipsec1:~# dd if=/dev/urandom count=1 bs=24 2>/dev/null | xxd -p5f6cf7dabd5a5f7704a98979ee9511f8af3e02436960cb06

Q: Configure IPSec to use these keys for communications between your two hosts. Don't write security policiesyet, just declare these keys as back and forth communication keys between these machines.

A: Add the following lines to /etc/ipsec-tools.conf, on both machines (replace IP addresses by thoseof your hosts):


Test Page 111

# AH SA's add 192.168.82.156 192.168.82.180 ah 0x200 -A hmac-md5 0x4e331c97f663cd11e4f9885995020588; add 192.168.82.180 192.168.82.156 ah 0x300 -A hmac-md5 0xa9e6d3bf3bc69d7ed89dc88b97b0e669;

# ESP SA'sadd 192.168.82.156 192.168.82.180 esp 0x201 -E 3des-cbc0xcca621f764a640fb8efaa7527df2fdb6ad7163eb99dd299f;add 192.168.82.180 192.168.82.156 esp 0x301 -E 3des-cbc0x5f6cf7dabd5a5f7704a98979ee9511f8af3e02436960cb06;

Q: Add security policies requiring all traffic (incoming and outgoing) to be encapsulated by IPSec in transportmode, using ESP+AH flavor.

A: Edit /etc/ipsec-tools.conf as follows. On debian-ipsec1 (192.168.82.156):

spdadd 192.168.82.156 192.168.82.180 any -P out ipsec esp/transport//require ah/transport//require;

spdadd 192.168.82.180 192.168.82.156 any -P in ipsec esp/transport//require ah/transport//require;

On debian-ipsec2 (192.168.82.180):

spdadd 192.168.82.156 192.168.82.180 any -P in ipsec esp/transport//require ah/transport//require;

spdadd 192.168.82.180 192.168.82.156 any -P out ipsec esp/transport//require ah/transport//require;

Q: Ensure IPSec is not active on any host.

A:root@debian-ipsec1:~# /etc/init.d/setkey stopFlushing IPsec SA/SP database: done.root@debian-ipsec2:~# /etc/init.d/setkey stopFlushing IPsec SA/SP database: done.

Q: Try to ping from one host to another while tcpdump'ing on the other. What do you notice?

A:root@debian-ipsec1:~# ping 192.168.82.180PING 192.168.82.180 (192.168.82.180) 56(84) bytes of data.64 bytes from 192.168.82.180: icmp_req=1 ttl=64 time=3.64 ms64 bytes from 192.168.82.180: icmp_req=2 ttl=64 time=0.430 ms64 bytes from 192.168.82.180: icmp_req=3 ttl=64 time=0.292 ms64 bytes from 192.168.82.180: icmp_req=4 ttl=64 time=0.314 ms


Test Page 112

On debian-ipsec2:

root@debian-ipsec2:~# tcpdump -i eth0 -n 'host 192.168.82.156' tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes17:45:06.993143 ARP, Request who-has 192.168.82.180 tell 192.168.82.156, length 4617:45:06.993247 ARP, Reply 192.168.82.180 is-at 00:0c:29:e9:6e:d7, length 2817:45:06.993802 IP 192.168.82.156 > 192.168.82.180: ICMP echo request, id 4215, seq 1, length 6417:45:06.993893 IP 192.168.82.180 > 192.168.82.156: ICMP echo reply, id 4215, seq 1, length 6417:45:07.991638 IP 192.168.82.156 > 192.168.82.180: ICMP echo request, id 4215, seq 2, length 6417:45:07.991696 IP 192.168.82.180 > 192.168.82.156: ICMP echo reply, id 4215, seq 2, length 6417:45:08.990532 IP 192.168.82.156 > 192.168.82.180: ICMP echo request, id 4215, seq 3, length 6417:45:08.990576 IP 192.168.82.180 > 192.168.82.156: ICMP echo reply, id 4215, seq 3, length 6417:45:09.989538 IP 192.168.82.156 > 192.168.82.180: ICMP echo request, id 4215, seq 4, length 6417:45:09.989591 IP 192.168.82.180 > 192.168.82.156: ICMP echo reply, id 4215, seq 4, length 64

The ICMP protocol can be read clearly over the wire.

Q: Enable IPSec on both machines.

A:root@debian-ipsec1:~# /etc/init.d/setkey startroot@debian-ipsec2:~# /etc/init.d/setkey start

Q: Try to ping from one host to another while tcpdump'ing on the other. What do you notice?

A:root@debian-ipsec1:~# ping 192.168.82.180PING 192.168.82.180 (192.168.82.180) 56(84) bytes of data.64 bytes from 192.168.82.180: icmp_req=1 ttl=64 time=0.403 ms64 bytes from 192.168.82.180: icmp_req=2 ttl=64 time=0.417 ms64 bytes from 192.168.82.180: icmp_req=3 ttl=64 time=0.877 ms64 bytes from 192.168.82.180: icmp_req=4 ttl=64 time=0.432 ms

On debian-ipsec2:

root@debian-ipsec2:~# tcpdump -i eth0 -n 'host 192.168.82.156' tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes17:48:52.358512 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x00000200,seq=0x4): ESP(spi=0x00000201,seq=0x4), length 8817:48:52.358598 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x00000300,seq=0x4): ESP(spi=0x00000301,seq=0x4), length 8817:48:53.357539 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x00000200,seq=0x5): ESP(spi=0x00000201,seq=0x5), length 8817:48:53.357608 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x00000300,seq=0x5): ESP(spi=0x00000301,seq=0x5), length 8817:48:54.356752 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x00000200,seq=0x6): ESP(spi=0x00000201,seq=0x6), length 8817:48:54.356901 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x00000300,seq=0x6): ESP(spi=0x00000301,seq=0x6), length 8817:48:55.357142 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x00000200,seq=0x7): ESP(spi=0x00000201,seq=0x7), length 8817:48:55.357211 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x00000300,seq=0x7): ESP(spi=0x00000301,seq=0x7), length 88

Packets are now encrypted.

9.1.2. Dynamic keying with racoon

9.1.2.1. Using a pre-shared key

Q: Remove the static keys you've added previously from IPSec configuration (on both machines). Don't removesecurity policies: Only remove security associations.

A: Remove (or comment out) this part of /etc/ipsec-tools.conf:


Test Page 113

# AH SA's add 192.168.82.156 192.168.82.180 ah 0x200 -A hmac-md5 0x4e331c97f663cd11e4f9885995020588; add 192.168.82.180 192.168.82.156 ah 0x300 -A hmac-md5 0xa9e6d3bf3bc69d7ed89dc88b97b0e669;

# ESP SA'sadd 192.168.82.156 192.168.82.180 esp 0x201 -E 3des-cbc0xcca621f764a640fb8efaa7527df2fdb6ad7163eb99dd299f;add 192.168.82.180 192.168.82.156 esp 0x301 -E 3des-cbc0x5f6cf7dabd5a5f7704a98979ee9511f8af3e02436960cb06;

Q: Install racoon on both machines.

A:root@debian-ipsec1:~# apt-get install racoonroot@debian-ipsec1:~# apt-get install racoon

Q: On both machines, configure racoon with the following settings:

• IKE Settings (for all potential communication partners) :

• Main mode

• Use 3des for encryption and sha1 for hashing.

• Use pre shared keys found in /etc/racoon/psk.txt

• IPSec SA's settings (for all hosts).

• Same algorithms as for IKE.

• Enable compression

A: Edit /etc/racoon/racoon.conf as follows (on both nodes):

path pre_shared_key "/etc/racoon/psk.txt";

remote anonymous { exchange_mode main;

proposal { encryption_algorithm 3des; hash_algorithm sha1; authentication_method pre_shared_key; dh_group 2; }}

sainfo anonymous { encryption_algorithm 3des; authentication_algorithm hmac_sha1; compression_algorithm deflate ;}


Test Page 114

Q: Set the psk to use for communications between these hosts to "supinfoipsecaccess" (without quotes).

A:root@debian-ipsec1:~# echo "192.168.82.180 supinfoipsecaccess" >> /etc/racoon/psk.txtroot@debian-ipsec2:~# echo "192.168.82.156 supinfoipsecaccess" >> /etc/racoon/psk.txt

Q: Restart the whole IPSec userland stack on both machines.

A:root@debian-ipsec1:~# /etc/init.d/setkey restart && /etc/init.d/racoon restartroot@debian-ipsec2:~# /etc/init.d/setkey restart && /etc/init.d/racoon restart

Q: Try to ping from one host to another while tcpdump'ing on the other.

A:root@debian-ipsec1:~# ping 192.168.82.180PING 192.168.82.180 (192.168.82.180) 56(84) bytes of data.64 bytes from 192.168.82.180: icmp_req=2 ttl=64 time=0.602 ms64 bytes from 192.168.82.180: icmp_req=3 ttl=64 time=0.405 ms64 bytes from 192.168.82.180: icmp_req=4 ttl=64 time=0.437 ms

On debian-ipsec2:

root@debian-ipsec2:~# tcpdump -i eth0 -n 'host 192.168.82.156' tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes18:32:01.578661 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x0c401987,seq=0x4): ESP(spi=0x051bcd76,seq=0x4), length 10018:32:01.578770 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x0c244255,seq=0x4): ESP(spi=0x0da4c8b8,seq=0x4), length 10018:32:02.577657 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x0c401987,seq=0x5): ESP(spi=0x051bcd76,seq=0x5), length 10018:32:02.577733 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x0c244255,seq=0x5): ESP(spi=0x0da4c8b8,seq=0x5), length 10018:32:03.576624 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x0c401987,seq=0x6): ESP(spi=0x051bcd76,seq=0x6), length 100

9.1.2.2. Using X.509 certificates

9.1.2.2.1. Creating certificates

In this section, you're going to create two certificates/key pair, one for each host. You can directly use the opensslcommand or make use of the CA.pl script that comes with the openssl package, at your option. This documentwill illustrate the second option.

Q: On one of the machines, create a /root/ca directory and call CA.pl -newca from that directory. You canget the full path to CA.pl by using your distro package management system. Type-in appropriate values whenrequested. You can refer to the OpenSSL lab of the "System Fundamentals" course.


Test Page 115

A:root@debian-ipsec1:~# mkdir caroot@debian-ipsec1:~# cd caroot@debian-ipsec1:~/ca# /usr/lib/ssl/misc/CA.pl -newcaCA certificate filename (or enter to create)

Making CA certificate ...Generating a 1024 bit RSA private key...++++++..++++++writing new private key to './demoCA/private/cakey.pem'Enter PEM pass phrase: supinfoVerifying - Enter PEM pass phrase: supinfo-----You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [AU]:USState or Province Name (full name) [Some-State]:UtahLocality Name (eg, city) []:ProvoOrganization Name (eg, company) [Internet Widgits Pty Ltd]:UtopiaOrganizational Unit Name (eg, section) []:Common Name (eg, YOUR name) []:ca.utopia.netEmail Address []:[email protected]

Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:An optional company name []:Using configuration from /usr/lib/ssl/openssl.cnfEnter pass phrase for ./demoCA/private/cakey.pem: supinfoCheck that the request matches the signatureSignature okCertificate Details: Serial Number: c8:d9:f1:ca:a1:6a:bb:bd Validity Not Before: Jul 23 14:41:22 2012 GMT Not After : Jul 23 14:41:22 2015 GMT Subject: countryName = US stateOrProvinceName = Utah organizationName = Utopia commonName = ca.utopia.net emailAddress = [email protected] X509v3 extensions: X509v3 Subject Key Identifier: 55:59:38:CB:31:43:EB:CA:56:01:6B:D1:D4:C8:02:4A:14:E1:34:94 X509v3 Authority Key Identifier: keyid:55:59:38:CB:31:43:EB:CA:56:01:6B:D1:D4:C8:02:4A:14:E1:34:94 DirName:/C=US/ST=Utah/O=Utopia/CN=ca.utopia.net/[email protected] serial:C8:D9:F1:CA:A1:6A:BB:BD

X509v3 Basic Constraints: CA:TRUECertificate is to be certified until Jul 23 14:41:22 2015 GMT (1095 days)

Write out database with 1 new entriesData Base Updated

Q: Create a new signing request for debian-ipsec1 using CA.pl -newreq.


Test Page 116

A:root@debian-ipsec1:~/ca# /usr/lib/ssl/misc/CA.pl -newreqGenerating a 1024 bit RSA private key........++++++.....++++++writing new private key to 'newkey.pem'Enter PEM pass phrase: supinfoVerifying - Enter PEM pass phrase: supinfo-----You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [AU]:USState or Province Name (full name) [Some-State]:UtahLocality Name (eg, city) []:ProvoOrganization Name (eg, company) [Internet Widgits Pty Ltd]:UtopiaOrganizational Unit Name (eg, section) []:Common Name (eg, YOUR name) []:debian-ipsec1Email Address []:

Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:An optional company name []:Request is in newreq.pem, private key is in newkey.pem

Q: Sign the request using CA.pl -sign.

A:root@debian-ipsec1:~/ca# /usr/lib/ssl/misc/CA.pl -signUsing configuration from /usr/lib/ssl/openssl.cnfEnter pass phrase for ./demoCA/private/cakey.pem:Check that the request matches the signatureSignature okCertificate Details: Serial Number: c8:d9:f1:ca:a1:6a:bb:be Validity Not Before: Jul 23 14:45:58 2012 GMT Not After : Jul 23 14:45:58 2013 GMT Subject: countryName = US stateOrProvinceName = Utah localityName = Provo organizationName = Utopia commonName = debian-ipsec1 X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: 1E:5B:8F:12:22:E1:61:EA:59:26:26:BD:D3:A3:2B:5F:65:05:F8:40 X509v3 Authority Key Identifier: keyid:55:59:38:CB:31:43:EB:CA:56:01:6B:D1:D4:C8:02:4A:14:E1:34:94

Certificate is to be certified until Jul 23 14:45:58 2013 GMT (365 days)Sign the certificate? [y/n]:y

1 out of 1 certificate requests certified, commit? [y/n]yWrite out database with 1 new entriesData Base UpdatedSigned certificate is in newcert.pem

Q: Use the openssl command to read RSA-encrypted key from newkey.pem and output it in clear form todebian-ipsec1.key.


Test Page 117

A:root@debian-ipsec1:~/ca# openssl rsa -in newkey.pem -out debian-ipsec1.keyEnter pass phrase for newkey.pem: supinfowriting RSA key

Q: Rename the certificate to debian-ipsec1.crt, and copy both key and certificate to the /etc/racoon/certs directory (create it as needed).

A:root@debian-ipsec1:~/ca# mv newcert.pem debian-ipsec1.crtroot@debian-ipsec1:~/ca# mkdir /etc/racoon/certsroot@debian-ipsec1:~/ca# cp debian-ipsec1.* /etc/racoon/certs

Q: Follow the same steps to create a certificate/key pair for debian-ipsec2.


Test Page 118

A:root@debian-ipsec1:~/ca# /usr/lib/ssl/misc/CA.pl -newreqGenerating a 1024 bit RSA private key.++++++...............................++++++writing new private key to 'newkey.pem'Enter PEM pass phrase: supinfoVerifying - Enter PEM pass phrase: supinfo-----You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [AU]:USState or Province Name (full name) [Some-State]:UtahLocality Name (eg, city) []:ProvoOrganization Name (eg, company) [Internet Widgits Pty Ltd]:UtopiaOrganizational Unit Name (eg, section) []:Common Name (eg, YOUR name) []:debian-ipsec2Email Address []:

Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:An optional company name []:Request is in newreq.pem, private key is in newkey.pemroot@debian-ipsec1:~/ca# /usr/lib/ssl/misc/CA.pl -signUsing configuration from /usr/lib/ssl/openssl.cnfEnter pass phrase for ./demoCA/private/cakey.pem: supinfoCheck that the request matches the signatureSignature okCertificate Details: Serial Number: c8:d9:f1:ca:a1:6a:bb:bf Validity Not Before: Jul 23 14:54:41 2012 GMT Not After : Jul 23 14:54:41 2013 GMT Subject: countryName = US stateOrProvinceName = Utah localityName = Provo organizationName = Utopia commonName = debian-ipsec2 X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: 0D:3D:47:4D:9F:CA:BE:A4:52:EE:4F:FD:EF:7B:21:CD:BC:2A:89:CE X509v3 Authority Key Identifier: keyid:55:59:38:CB:31:43:EB:CA:56:01:6B:D1:D4:C8:02:4A:14:E1:34:94

Certificate is to be certified until Jul 23 14:54:41 2013 GMT (365 days)Sign the certificate? [y/n]:y

1 out of 1 certificate requests certified, commit? [y/n]yWrite out database with 1 new entriesData Base UpdatedSigned certificate is in newcert.pemroot@debian-ipsec1:~/ca# openssl rsa -in newkey.pem -out debian-ipsec2.keyEnter pass phrase for newkey.pem:writing RSA keyroot@debian-ipsec1:~/ca# mv newcert.pem debian-ipsec2.crt


Test Page 119

Q: Copy the certificate/key pair to debian-ipsec2 /etc/racoon/certs.

A:root@debian-ipsec2:~# mkdir /etc/racoon/certsroot@debian-ipsec1:~/ca# scp debian-ipsec2.* [email protected]:/etc/racoon/certs/[email protected]'s password: debian-ipsec2.crt 100% 3131 3.1KB/s 00:00debian-ipsec2.key 100% 887 0.9KB/s 00:00

Q: Copy the certification authority certificate to /etc/racoon/certs as well (on both machines).

A:root@debian-ipsec1:~/ca# cp demoCA/cacert.pem /etc/racoon/certs/CA.crtroot@debian-ipsec1:~/ca# scp demoCA/cacert.pem [email protected]:/etc/racoon/certs/[email protected]'s password: cacert.pem 100% 3371 3.3KB/s 00:00

Q: In the /etc/racoon/certs (on both machines), create a symlink key-hash.0 that links to the actualcertificate. You can get the key hash by using the openssl x509 -noout -hash -in <filename> command.

A:root@debian-ipsec1:/etc/racoon/certs# ln -s CA.crt `openssl x509 -noout -hash -in CA.crt`.0root@debian-ipsec2:/etc/racoon/certs# ln -s CA.crt `openssl x509 -noout -hash -in CA.crt`.0

9.1.2.2.2. Configuring racoon

Q: Configure racoon IKE to:

• Look for certificates in /etc/racoon/certs.

• Use certificate and keys you've created.

• Use the certificate common name as the identifier for both local and remote machines.

• Use a RSA signature as the authentication method.

A: Edit /etc/racoon/racoon.conf as follows. On debian-ipsec1:


Test Page 120

path pre_shared_key "/etc/racoon/psk.txt";path certificate "/etc/racoon/certs";

remote anonymous { exchange_mode main;

certificate_type x509 "debian-ipsec1.crt" "debian-ipsec1.key"; my_identifier asn1dn; peers_identifier asn1dn;

proposal { encryption_algorithm 3des; hash_algorithm sha1; authentication_method rsasig; dh_group 2; }}

sainfo anonymous { encryption_algorithm 3des; authentication_algorithm hmac_sha1; compression_algorithm deflate ;}

Same configuration on debian-ipsec2 with debian-ipsec2.crt and debian-ipsec2.key instead of the abovevalues.



Q: Try to ping from one host to another while tcpdump'ing on the other.

A:root@debian-ipsec1:/etc/racoon# ping 192.168.82.180PING 192.168.82.180 (192.168.82.180) 56(84) bytes of data.64 bytes from 192.168.82.180: icmp_req=1 ttl=64 time=0.469 ms64 bytes from 192.168.82.180: icmp_req=2 ttl=64 time=0.376 ms64 bytes from 192.168.82.180: icmp_req=3 ttl=64 time=0.463 ms

On debian-ipsec2:

root@debian-ipsec2:/etc/racoon# tcpdump -i eth0 -n 'host 192.168.82.156' tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes19:16:02.587220 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x036acca3,seq=0xc): ESP(spi=0x0d19434d,seq=0xc), length 10019:16:02.587335 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x05900922,seq=0xc): ESP(spi=0x027aa409,seq=0xc), length 10019:16:03.586149 IP 192.168.82.156 > 192.168.82.180: AH(spi=0x036acca3,seq=0xd): ESP(spi=0x0d19434d,seq=0xd), length 10019:16:03.586237 IP 192.168.82.180 > 192.168.82.156: AH(spi=0x05900922,seq=0xd): ESP(spi=0x027aa409,seq=0xd), length 100

9.2. Network to network

In this section, you're going to leverage your existing setup to make the host-to-host setup into a tunneled network-to-network configuration, as in the following schema:


Test Page 121

You'll need two more virtual machines and to alter you existing virtual machines configuration. Add a NIC to eachone. This NIC must be connected to a host-only virtual network. The 172.17.1.0/24 and 172.17.2.0/24 networksmust be separate virtual networks. Use the vmware virtual network editor to create enough dedicated host-onlyvirtual networks.

Make sure all virtual machines have the correct hostname and that debian-ipsec2 and debian-net2 cancommunicate over the 172.17.2.0/24 network. Do the same verifications for the 172.17.1.0/24 network.

debian-net1 and debian-net2 should also have respectively debian-ipsec1 and debian-ipsec2 as their defaultgateway. The following questions assumes that all these networking settings have been correctly configured.

Q: Enable packet routing on debian-ipsec1 and debian-ipsec2.

A:root@debian-ipsec1:~# echo 1 > /proc/sys/net/ipv4/ip_forwardroot@debian-ipsec2:~# echo 1 > /proc/sys/net/ipv4/ip_forward

Q: On both IPSec endpoints, add SA's to specify that all traffic going from or to the 172.17.X.0/24 networkbehind the local or remote endpoint should go through the IPSec tunnel working in ESP mode.

A: Edit /etc/ipsec-tools.conf as follows. On debian-ipsec1:

spdadd 172.17.1.0/24 172.17.2.0/24 any -P out ipsec esp/tunnel/192.168.82.156-192.168.82.180/require;

spdadd 172.17.2.0/24 172.17.1.0/24 any -P in ipsec esp/tunnel/192.168.82.180-192.168.82.156/require;

On debian-ipsec2:

spdadd 172.17.2.0/24 172.17.1.0/24 any -P out ipsec esp/tunnel/192.168.82.180-192.168.82.156/require;

spdadd 172.17.1.0/24 172.17.2.0/24 any -P in ipsec esp/tunnel/192.168.82.156-192.168.82.180/require;

Q: On both endpoints, stop the whole IPSec userland stack.

A:root@debian-ipsec1:~# /etc/init.d/setkey stop && /etc/init.d/racoon stoproot@debian-ipsec2:~# /etc/init.d/setkey stop && /etc/init.d/racoon stop

Q: From debian-net1, try to ping debian-net2. Does it works? Why?


Test Page 122

A:supinfo@debian-net1:~$ ping 172.17.2.128PING 172.17.2.128 (172.17.2.128) 56(84) bytes of data.

It doesn't work, because even if debian-net1 as debian-ipsec1 as the default router, there is no routebetween debian-ipsec1 and debian-ipsec2 saying that debian-ipsec2 is a router for 172.17.2.0/24. Thereforedebian-ipsec1 will send the packet to its default gateway which is not debian-ipsec2.



Q: Try to ping again between debian-net1 and debian-net2. Does it works? Why?

A:supinfo@debian-net1:~$ ping 172.17.2.128PING 172.17.2.128 (172.17.2.128) 56(84) bytes of data.64 bytes from 172.17.2.128: icmp_req=3 ttl=62 time=7.24 ms64 bytes from 172.17.2.128: icmp_req=4 ttl=62 time=1.98 ms

It works, because the packets are now forwarded to the other endpoint through the IPSec tunnel, despitethe lack of routing table entry.


Test Page 123

10. RADIUSIn this Lab, you're going to setup a RADIUS authentication server and an AS to demonstrate its features. The AS willbe a dd-wrt based router that will run a Chillispot captive portal. Users plugged to this router will automaticallyland on a connection page the first time they try to access a web page. They will enter their credentials and therouter will try these credentials against the RADIUS server which will in turn grant or deny access depending onthe credentials validity.

The network topology will look like the following schema:

You'll need two "regular" (based on debian-master or your favorite Linux distro) virtual machines for debian-radius and debian-netclient plus a special dd-wrt vmware image. You'll find this image on courses (http://courses.supinfo.com/), along with other Linux resources.

Prepare and configure these virtual machines as in the topology schema. As dd-wrt will distribute IP leases toclients and get its "WAN" ip from dhcp, disable VMware built-in dhcp for the host-only network and make sureit's enabled for the NAT'ed network.

The dd-wrt virtual machine as two network interfaces already configured to be on the right network: The firstinterface is to be connected on the "WAN" (VMware NAT'ed vnet) and the second to the host-only network. Makesure these interfaces as connected as expected.

10.1. Setting up the RADIUS Server

10.1.1. RADIUS

Q: Install the freeradius package.

A:root@debian-radius:~# apt-get install freeradius

Note


Q: Configure freeradius to accpet clients from the VMware shared network with the "supinfo-radius" secret.Also set the same secret for the "localhost" client.


Test Page 124

A: Edit (add) /etc/freeradius/clients.conf as follows:

client localhost {[...] secret = supinfo-radius[...]}client 192.168.82.0/24 { secret = supinfo-radius shortname = vmware-nated}

Q: Add a user record to your radius flat file user database: The username is your Open Campus ID and thepassword is "supinfo".

A: Edit (add) /etc/freeradius/users as follows:

40793 Cleartext-Password := "supinfo"

Q: Make sure the radius service is not running. Using two shells, run the service binary in the foreground indebug mode and do a test run using radtest.

A: In a first shell:

root@debian-radius:~# /etc/init.d/freeradius stoproot@debian-radius:~# freeradius -X[...]Listening on proxy address * port 1814Ready to process requests.

In a second shell:

supinfo@debian-radius:~$ radtest 40793 supinfo localhost 0 supinfo-radiusSending Access-Request of id 228 to 127.0.0.1 port 1812 User-Name = "40793" User-Password = "supinfo" NAS-IP-Address = 127.0.1.1 NAS-Port = 0rad_recv: Access-Accept packet from host 127.0.0.1 port 1812, id=228, length=20


A:root@debian-radius:~# /etc/init.d/freeradius start

10.1.2. Chillispot CGI

Unfortunately, the chillispot version that comes with dd-wrt doesn'r fully run on the router: Only the deamon runsthere. The other part, the login cgi, must be installed (along with a webserver) on another machine. For this Lab,I've choosen to run minimize the number of virtual machines and run the CGI on the same server as Radius. You'llnow configure debian-radius to run that CGI.

Q: Install the apache2 webserver.


Test Page 125

A:root@debian-radius:~# apt-get install apache2

Note


Q: Download and install the chillispot debian package from http://www.chillispot.info/download/chillispot_1.0_i386.deb.

A:root@debian-radius:~# wget http://www.chillispot.info/download/chillispot_1.0_i386.debroot@debian-radius:~# dpkg -i chillispot_1.0_i386.deb

Q: Use the hotspotlogin.cgi.gz file from the chillispot package (use your package system tools to locatethe file) to create /var/www/cgi-bin/hotspotlogin.cgi (create directories as needed).

A:root@debian-radius:~# mkdir /var/www/cgi-binroot@debian-radius:~# zcat \>/usr/share/doc/chillispot/hotspotlogin.cgi.gz > /var/www/cgi-bin/hotspotlogin.cgi

Q: Edit the CGI script to set the secret that will be shared between the CGI and the remote chillispot deamon.This secret will be supinfo-uam. Also enable user-password RADIUS authentication. Just open the script witha text editor and read the 30+ first lines to know how to perform this task. Everything is in the comments.

A: Edit(alter) the CGI as follows:

# Shared secret used to encrypt challenge with. Prevents dictionary attacks.# You should change this to your own shared secret.$uamsecret = "supinfo-uam";

# Uncomment the following line if you want to use ordinary user-password# for radius authentication. Must be used together with $uamsecret.$userpassword=1;

Q: Configure the default SSL vhost that come with the apache2 package (or create a new one, along withcertificates) to serve CGI's from /var/www/cgi-bin.

A: Edit(Alter) /etc/apache2/sites-available/default-ssl as follows:

[...] ScriptAlias /cgi-bin/ /var/www/cgi-bin/ <Directory "/var/www/cgi-bin"> [...] </Directory>[...]

Q: Enable appropriates modules and virtual hosts as needed.

A:root@debian-radius:~# a2enmod sslroot@debian-radius:~# a2ensite default-ssl


Test Page 126

Note


Q: Restart the Apache service.

A:root@debian-radius:~# /etc/init.d/apache2 restart

Q: Point a browser(curl works well for that) at the CGI, using HTTPS. You should get a message that says youcan only login through the chillispot deamon.

A:root@debian-radius:~# curl -k https://192.168.82.156/cgi-bin/hotspotlogin.cgi

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head> <title>ChilliSpot Login Failed</title> <meta http-equiv="Cache-control" content="no-cache"> <meta http-equiv="Pragma" content="no-cache"></head><body bgColor = '#c0d8f4'> <h1 style="text-align: center;">ChilliSpot Login Failed</h1> <center> Login must be performed through ChilliSpot daemon. </center></body></html>

10.2. Setting up dd-wrt

Start the dd-wrt virtual machine. If you've followed the previous instructions, dd-wrt has its second interfaceconnected to the host-only virtual network. In its default configuration, dd-wrt will distribute IP addresses onthat network. Either use your host operating system to get an IP address via DHCP on this interface (or manuallyassign the host-only virtual host iface an IP address within the 192.168.1.0/24 network), or use another virtualmachine plugged on the host-only network. Once you've got an IP address, you can point your browser tohttp://192.168.1.1/.

Q: Go to the dd-wrt web admin interface and follow Setup -> Basic Setup and set the WAN connection type toDHCP. Don't forget to Save & Apply Settings afterwise.

A: Edit settings as follows:

Q: Still from the dd-wrt web admin interface, go through Services -> Hotspot. In this page, enable Chillispotand fill the parameters as follows:


Test Page 127

• Primary Radius Server: IP of debian-radius.

• DNS IP: IP of the VMware NAT'ed network DNS. You can find it on /etc/resolv.conf of any machineconnected to the VMware NAT'ed network, such as debian-radius.

• Remote network: Network were the clients are going to be. This setting will alter dd-wrt configuration.It'll then provide DHCP leases for that network instead of the default 192.168.82.1.0/24. You can leavethe default.

• Redirect URL: URL of the CGI login script you've configured in the previous part.

• Shared Key: Radius secret you've configured in section 1.1.

• UAM Secret: Shared secret between chillispot deamon and authentication CGI you've configured earlier.

Don't forget to Save & Apply Settings afterwise.

A: Edit settings as follows:

10.3. Tests with the client

Before going any further, make sure the client network interface is connected to the host-only network were dd-wrt also is. Please note and keep in mind that you'll have to temporarily switch this interface connection to NATif you need to access the internet before being able to authenticiate against chillispot (i.e install packages). Don'tforget to switch the interface back to host-only afterwise. Also remember that you'll have to bring the iface upand down (or run the DHCP client manually) each time you switch the connection.

Q: To test proper operation of this setup, you'll need a browser on the client. You can choose to install firefox(package name iceweasel in Debian) and run it through a SSH connection if you're running an X-capablehost operating system, install a complete GUI environment on debian-netclient, or install a text-mode webbroswser such as elinks. Make your choice and install appropriate packages. Be sure to have read the aboveinstructions about network connectivity.

A: Connect virtual network iface to NAT'ed network.


Test Page 128

root@debian-netclient:~# ifdown eth0 && ifup eth0root@debian-netclient:~# apt-get install iceweasel

Connect virtual NIC back to host-only network.

root@debian-netclient:~# ifdown eth0 && ifup eth0

Run firefox through ssh from host operating system.

Q: Using a browser running on debian-netclient, try to reach www.google.com. What happens? Why?

A: AS you're not identified yet, you are redirected to the chillispot login page:

Q: Use your RADIUS credentials to open a session. What happens?

A: You get a "session control window" that allows you do end it and you're redirected to the website you weretrying to reach (in this case www.google.com).


Test Page 129

11. PuppetIn this lab, you're going to puppetize various services.

You'll need two virtual machines. The first one will be your DNS server and your Puppet Master, the second onewill be your Puppet-managed node.

You'll be working with Bind9 and Puppet. These tools are not installed by default on all distros. Use yourpackage management system to install them as needed. On Debian-based systems, package names are bind9,puppetmaster and puppet.

Also set virtual machines hostnames to "puppetmaster" (DNS/Puppet Master) for the first one and"workstation" (Puppet-managed node) for the second one.

11.1. Preparing the DNS Server

Q: Configure your first virtual machine to act as a DNS server. This DNS server will host the utopia.net withtwo hosts:

• puppetmaster.utopia.net

• workstation.utopia.net

Resolve these names to the virtual machine's IP addresses.

A: Edit your DNS configuration as follows:

zone "utopia.net" { type master; file "/etc/bind/utopia.net";};

Create the /etc/bind/utopia.net zone file with the following content:

$TTL 604800@ IN SOA ns.utopia.net. root.utopia.net. ( 1 ; Serial 604800 ; Refresh 86400 ; Retry 2419200 ; Expire 604800 ) ; Negative Cache TTL;@ IN NS ns.utopia.net.ns IN A 192.168.0.25puppetmaster IN A 192.168.0.25workstation IN A 192.168.0.50

11.2. Working with Puppetmaster

Q: On the client, enter the puppet master FQDN into the adequate file.


Test Page 130

A:root@workstation:~# vim /etc/puppet/puppet.conf[main]server=puppetmaster.utopia.net

Q: Generate (and sign) the SSL certificate needed by Puppet for the workstation node.

A:root@workstation:~# puppetd --test --waitforcert 60

root@puppetmaster:~# puppetca --listroot@puppetmaster:~# puppetca --sign workstation.utopia.net

Q: Start the Puppet daemon on the workstation. On Debian-based systems, you might need to edit the /etc/default/puppet file.

A:root@workstation:~# /etc/init.d/puppet startStarting puppet agentpuppet not configured to start, please edit /etc/default/puppet to enable.

root@workstation:~# echo "START=yes" > /etc/default/puppetroot@workstation:~# /etc/init.d/puppet start

Enable Puppet files transfer by adding these two lines into /etc/puppet/manifests/site.pp (on theserver):

filebucket { main: server => puppetmaster.utopia.net }File { backup => main }

Q: Create the directory structure for a module called editor, a module called ssh and another called apache.

A:root@puppetmaster:~# cd /etc/puppet/modulesroot@puppetmaster:~# mkdir -p {editor,ssh,apache}/{manifests,files,templates}

Q: Write a Puppet program that makes sure the vim editor is installed, whereas nano has to be absent. Thisprogram is part of the editor module.

A:root@puppetmaster:~# vim /etc/puppet/modules/editor/init.pp

class ssh {

package { "vim": ensure => installed }

package { "nano": ensure => absent }}


Test Page 131

Q: Create a "basenode" node configuration spec in your Puppetmaster configuration. This node spec includesyour editor module.

A:root@puppetmaster:~# vim /etc/puppet/manifests/nodes.pp

node basenode {include editor}

Q: Alter your configuration to add a "workstation" node specification. This node spec inherits from the"basenode" spec.

A:root@puppetmaster:~# vim /etc/puppet/manifests/nodes.pp


node workstation.utopia.net inherits basenode {}

Q: Write a "ssh" Puppet class to manage SSH. This class must ensure SSH is installed in the latest version. SSHmust be running and launched during system boot. The SSH package is required for the service : it mustbe explicitily specified.

A:root@puppetmaster:~# vim /etc/puppet/modules/ssh/init.pp

class ssh {

package { "openssh-server": ensure => latest }

service { "ssh": ensure => running, enale => true, hasstatus => true, hasrestart => true, require => Package[openssh-server] }}

Q: Alter your configuration to manage SSH configuration using Puppet: Distribute a fixed default configurationfile with the following items:

• X Forwarding enabled

• SSH v2 only

• Direct root login forbidden

• sshd listens on 2222


Test Page 132

Be sure that any future change in this SSH configuration will result in a service restart. Of course, the SSHpackage is also required.

A: Edit /etc/puppet/modules/ssh/init.pp as follows:

class ssh {

package { "openssh-server": ensure => latest }

service { "ssh": ensure => running, enable => true, hasstatus => true, hasrestart => true, require => Package[openssh-server] } file { "/etc/ssh/sshd_config": owner => root, group => root, mode => 644, source => "puppet:///ssh/sshd_cfg", notify => Service['ssh'], require => Package['openssh-server'] }

}

Create /etc/puppet/modules/ssh/files/sshd_config from a default sshd configuration fileand alter it as follows:

[...]Port 2222Protocol 2PermitRootLogin yesX11Forwarding yes[...]

Q: Alter the configuration to include the ssh module in the workstation node spec. Restart the Puppetmasterand test your module on the node with puppetd --test.

A: Edit /etc/puppet/manifests/nodes.pp as follows:


Test Page 133


node workstation.utopia.net inherits basenode {include ssh}

root@puppetmaster:~# service puppetmaster restart

root@workstation:~# puppetd --test

Q: Create a "apache" Puppet class to manage tha Apache webserver. It will ensure that apache is installed inthe latest version and the daemon is launched. Imagine that you have also some servers using RedHat-baseddisitributions. You have to make sure that the package is installed and the daemon is launched, regardlessof the script name. Use only one class.

A: Edit /etc/puppet/modules/apache/init.pp as follows:

class apache {

case $operatingsystem { debian, ubuntu: { $apache_name = 'apache2' } fedora, redhat: { $apache_name = 'httpd' } }

package { "apache": name => $apache_name ensure => latest }

service { "apache": name => $apache_name ensure => running, enable => true, hasstatus => true, hasrestart => true, require => Package[$apache_name] }}

Q: Imagine that your server has more than one network interface. You want your webserver to only honorrequests from the eth0 interface. Use a Puppet template to distribute an Apache IP-based virtual host that"listens" on eth0 only.

Note

There is no need to cover all and every possible Apache install / configuration cases. Create aconfiguration that works on the distribution you're working on.


Test Page 134

A: Edit /etc/puppet/modules/apache/init.pp as follows:

class apache {

case $operatingsystem { debian, ubuntu: { $apache_name = 'apache2' } fedora, redhat: { $apache_name = 'httpd' } }

package { "apache": name => $apache_name ensure => latest }

service { "apache": name => $apache_name ensure => running, enable => true, hasstatus => true, hasrestart => true, require => Package[$apache_name] }

file { "/etc/apache2/sites-available/default": owner => root, group => root, mode => 644, content => template("apache/default.erb"), notify => Service['$apache_name'], require => Package['$apache_name'] }

file { "/etc/apache2/sites-enabled/default": ensure => link, target => '/etc/apache2/sites-available/default' }}

Create the template in /etc/puppet/modules/apache/templates:

<VirtualHost <%= ipaddress_eth0 %>:80> ServerAdmin webmaster@localhost Servername www.utopia.net DocumentRoot /var/www/ ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined</VirtualHost>

Q: Include the Apache module into the workstatio node spec. Restart the Puppetmaster and test your moduleon the node with puppetd --test.

A: Edit /etc/puppet/manifests/nodes.pp as follows:


Test Page 135


node workstation.utopia.net inherits basenode {include sshinclude apache}

root@puppetmaster:~# service puppetmaster restart

root@workstation:~# puppetd --test

02 - workbook - teachers edition

Documents

simple iscsi

multipath iscsi

mail domains

high available iscsibased

high available website

high availability

mail delivery

mail services