websphere edge server: working with web traffic express and network dispatcher

ibm.com/redbooks

WebSphere Edge Server: Working with Web Traffic Express andNetwork Dispatcher

Cristiane FerreiraAna Mostardinha

Byron Braswell

Configuration of WebSphere Edge Server

Scenarios for Web Traffic Express and Network Dispatcher

Details of latest enhancements

Front cover

http://www.redbooks.ibm.com/


WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

July 2001

International Technical Support Organization

SG24-6172-00

© Copyright International Business Machines Corporation 2001. All rights reserved.Note to U.S Government Users – Documentation related to restricted rights – Use, duplication or disclosure is subject torestrictions set forth in GSA ADP Schedule Contract with IBM Corp.

First Edition (July 2001)

This edition applies to Version 1.0.3 of WebSphere Edge Server for Multiplatforms for use with Windows NT, Windows 2000, AIX, Linux and Solaris operating systems.

This document created or updated on August 1, 2001.

Comments may be addressed to:IBM Corporation, International Technical Support OrganizationDept. HZ8 Building 662P.O. Box 12195Research Triangle Park, NC 27709-2195

When you send information to IBM, you grant IBM a non-exclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you.

Take Note! Before using this information and the product it supports, be sure to read the general information in “Special notices” on page 483.

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixThe team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixSpecial notice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiIBM trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiComments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Part 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Chapter 1. WebSphere Edge Server for Multiplatforms concepts . . . . . . . 31.1 WebSphere Edge Server for Multiplatforms . . . . . . . . . . . . . . . . . . . . . 41.2 Caching and filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.3 Load balancing and server monitoring capabilities . . . . . . . . . . . . . . . . 61.4 What is new in WebSphere Edge Server . . . . . . . . . . . . . . . . . . . . . . . 8

Chapter 2. What WebSphere Edge Server can do for you. . . . . . . . . . . . . 152.1 Building high performance Web sites . . . . . . . . . . . . . . . . . . . . . . . . . 162.2 Who can benefit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2.1 Content hosting Internet Service Providers. . . . . . . . . . . . . . . . . . . . 182.2.2 Corporate Web sites and content aggregators . . . . . . . . . . . . . . . . . 192.2.3 Backbone Internet Service Providers . . . . . . . . . . . . . . . . . . . . . . . . 212.2.4 Access Internet Service Providers . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Part 2. Network Dispatcher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Chapter 3. Network Dispatcher installation . . . . . . . . . . . . . . . . . . . . . . . . 273.1 Installing Network Dispatcher on AIX . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.1.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.1.2 Choosing the components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.1.3 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.1.4 Starting the software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.1.5 Uninstallation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.2 Installing Network Dispatcher on Windows NT . . . . . . . . . . . . . . . . . . 403.2.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403.2.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.2.3 Starting the software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513.2.4 Uninstallation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.3 Installing Network Dispatcher on Linux . . . . . . . . . . . . . . . . . . . . . . . . 553.3.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553.3.2 Choosing the components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

© Copyright IBM Corp. 2001 iii

3.3.3 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573.3.4 Starting the software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653.3.5 Uninstallation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Chapter 4. Network Dispatcher basic configuration . . . . . . . . . . . . . . . . . 694.1 Load balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

4.1.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704.1.2 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 714.1.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

4.2 High availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1044.2.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1064.2.2 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1074.2.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

4.3 Using Interactive Session Support . . . . . . . . . . . . . . . . . . . . . . . . . . 1274.3.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1284.3.2 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Chapter 5. Advanced features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1555.1 Remote configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1565.2 Collocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

5.2.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1595.2.2 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1605.2.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

5.3 Load balancing with fixed weights . . . . . . . . . . . . . . . . . . . . . . . . . . 1675.3.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1685.3.2 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1695.3.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

5.4 Rules-based load balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1755.4.1 Types of rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1765.4.2 How rules are evaluated . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1795.4.3 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1805.4.4 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1815.4.5 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

5.5 Wide area Network Dispatcher support . . . . . . . . . . . . . . . . . . . . . . 1915.5.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1925.5.2 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1945.5.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

5.6 Mutual high availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2065.6.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2075.6.2 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2085.6.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

5.7 Affinity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2315.7.1 Server Directed Affinity API to control client-server affinity . . . . . . . 232

iv WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

5.7.2 Cross port affinity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2335.7.3 Affinity address mask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2345.7.4 Rule affinity override . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2355.7.5 Cookie affinity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

5.8 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2375.8.1 Basic logging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2375.8.2 Binary logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

Part 3. Web Traffic Express . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

Chapter 6. Web Traffic Express installation . . . . . . . . . . . . . . . . . . . . . . . 2476.1 Installing Web Traffic Express on AIX. . . . . . . . . . . . . . . . . . . . . . . . 248

6.1.1 System hardware and software requirements. . . . . . . . . . . . . . . . . 2486.1.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2506.1.3 Starting and stopping Web Traffic Express. . . . . . . . . . . . . . . . . . . 2606.1.4 Uninstallation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

6.2 Installing Web Traffic Express on Windows NT . . . . . . . . . . . . . . . . 2626.2.1 System hardware and software requirements. . . . . . . . . . . . . . . . . 2626.2.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2636.2.3 Starting and stopping Web Traffic Express. . . . . . . . . . . . . . . . . . . 2726.2.4 Uninstallation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

6.3 Installing Web Traffic Express on Linux . . . . . . . . . . . . . . . . . . . . . . 2746.3.1 System hardware and software requirements. . . . . . . . . . . . . . . . . 2756.3.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2766.3.3 Starting and stopping Web Traffic Express. . . . . . . . . . . . . . . . . . . 2846.3.4 Uninstallation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

Chapter 7. Web Traffic Express basic configuration . . . . . . . . . . . . . . . . 2877.1 Basic configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

7.1.1 Configuration environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2887.1.2 Using the Configuration and Administration forms . . . . . . . . . . . . . 2897.1.3 Web Traffic Express Administration settings. . . . . . . . . . . . . . . . . . 2917.1.4 Configuring the basic settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2937.1.5 Configuration of the ibmproxy.conf file . . . . . . . . . . . . . . . . . . . . . . 307

7.2 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3097.2.1 Configuring the browser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3097.2.2 Testing the access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

Chapter 8. Advanced features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3138.1 Security enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

8.1.1 Handling header information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3148.1.2 Basic user authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3208.1.3 User authentication using LDAP . . . . . . . . . . . . . . . . . . . . . . . . . . . 3268.1.4 Blocking URLs using the CyberPatrol plug-in . . . . . . . . . . . . . . . . . 336

Contents v

8.2 Cache content management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3518.2.1 Controlling which documents are cached . . . . . . . . . . . . . . . . . . . . 3528.2.2 Cache freshness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3548.2.3 Automatic cache refreshing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3558.2.4 Caching of dynamic content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359

8.3 SOCKS client support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3678.3.1 How to set up flexible-client SOCKS . . . . . . . . . . . . . . . . . . . . . . . . 3688.3.2 Testing the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372

8.4 Proxy auto-configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3738.4.1 How to write a PAC file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3748.4.2 How to set up an automatic proxy. . . . . . . . . . . . . . . . . . . . . . . . . . 3768.4.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

8.5 Proxy chaining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3848.5.1 How to set up proxy chaining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3868.5.2 Testing the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3888.5.3 Configuration of the ibmproxy.conf file . . . . . . . . . . . . . . . . . . . . . . 390

8.6 Logging and performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3908.6.1 Activity Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3918.6.2 Configuration for performance improvements. . . . . . . . . . . . . . . . . 3978.6.3 WTE logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404

Part 4. Combining Network Dispatcher and Web Traffic Express . . . . . . . . . . . . . . . 413

Chapter 9. Multiple Web Traffic Express proxy servers. . . . . . . . . . . . . . 4159.1 Using Autoproxy or Network Dispatcher . . . . . . . . . . . . . . . . . . . . . . 4169.2 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4179.3 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418

9.3.1 Configuration file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4229.4 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423

Chapter 10. Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy . . . . . . . . . . . . . . . . . . . . . . . . . 425

10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42610.2 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42610.3 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42810.4 Network Dispatcher configuration file . . . . . . . . . . . . . . . . . . . . . . . 43510.5 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436

Chapter 11. Content Based Routing (CBR). . . . . . . . . . . . . . . . . . . . . . . . 44311.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444

11.1.1 Installation of the CBR function . . . . . . . . . . . . . . . . . . . . . . . . . . . 44411.2 CBR for HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445

11.2.1 CBR scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44511.2.2 WTE CacheByIncomingUrl directive . . . . . . . . . . . . . . . . . . . . . . . 461

vi WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

11.3 CBR proxy for IMAP and POP3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 46211.3.1 Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46211.3.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46311.3.3 Configuration file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46911.3.4 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469

Part 5. Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473

Appendix A. Java source code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475TimeServlet Java source code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476CalcServlet Java source code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479

Other resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479Referenced Web sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480

IBM Redbooks collections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481

Special notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487

Contents vii

viii WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Preface

As the Internet economy evolves, it is no longer enough to simply establish an online presence to sell your products and services. To compete in this fast-paced, global marketplace, you need to respond rapidly and intelligently to changing application and network demands while simplifying the implementation of new e-business solutions. Availability, scalability and performance are critical to ensuring that your e-business runs smoothly and cost-effectively.

WebSphere Edge Server for Multiplatforms allows Internet service providers (ISPs) and enterprise customers to confidently host mission-critical Web sites for intranet and Internet e-business applications. WebSphere Edge Server brings together innovative caching, local- and wide-area load balancing and application-aware quality-of-service routing capabilities. The integration of these functions and their wide array of features can help reduce cost for the site owner while improving customer satisfaction.

This redbook is intended for networking personnel and information technology administrators who plan to use WebSphere Edge Server to provide Internet access to end users and distribute Web-accessible content more efficiently.

This redbook will help you install, tailor and configure the features and enhancements included in WebSphere Edge Server for Multiplatforms Version 1.0.3. WebSphere Edge Server has two main components:

1. Caching proxy component (also known as Web Traffic Express 3.6)

2. Load balancing component (also known as Network Dispatcher 3.6)

Throughout this redbook, we will refer to the WebSphere Edge Server components as Web Traffic Express (WTE) and Network Dispatcher (ND).

The team that wrote this redbookThis redbook was produced by a team of specialists from around the world working at the International Technical Support Organization, Raleigh Center.

Cristiane Ferreira is a Network Specialist in the RS/6000 Support and Services team at IBM Brazil. She has nine years of experience in UNIX platforms and has been working with AIX support for the past five years. Her areas of expertise include AIX and Linux systems, TCP/IP networking, firewalls, and networking security.

© Copyright IBM Corp. 2001 ix

Ana Mostardinha is a Solution Architect in Brazil. She has four years of experience in AIX and Windows. She holds a degree in Meteorology from the Federal University of Rio de Janeiro and a System Analyst degree from the State University of Rio de Janeiro. Her areas of expertise include AIX, MQSeries and Tivoli Storage Manager.

Byron Braswell is a networking professional at the International Technical Support Organization, Raleigh Center. He received his B.S. degree in Physics and M.S. degree in Computer Sciences from Texas A&M University. He writes extensively and teaches IBM classes worldwide on all areas of networking. Before joining the ITSO, Byron worked in IBM Learning Services Development in networking education development.

Thanks to the following people for their contributions to this project:

Margaret TicknorBill MooreCarla SadtlerGail ChristensenInternational Technical Support Organization, Raleigh Center

Craig LanzenBob DelimaIBM Raleigh

Manu GugnaniWangming YeShih-pai LeeJeff KaminskiJim LiJeff CarlsonDoug SladeFalguni ShahDave HolderChris NolinMaria WadlowIBM Pittsburgh

Shendong ChenIBM Austin

x WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Special noticeThis publication is intended to help networking personnel and information technology administrators to install and customize WebSphere Edge Server for Multiplatforms. The information in this publication is not intended as the specification of any programming interfaces that are provided by WebSphere Edge Server. See the PUBLICATIONS section of the IBM Programming Announcement for WebSphere Edge Server for more information about what publications are considered to be product documentation.

IBM trademarksThe following terms are trademarks of the International Business Machines Corporation in the United States and/or other countries:

Comments welcomeYour comments are important to us!

We want our IBM Redbooks to be as helpful as possible. Please send us your comments about this or other Redbooks in one of the following ways:

� Use the online Contact us review redbook form found at:

ibm.com/redbooks

� Send your comments in an Internet note to:

[email protected]

� Mail your comments to the address on page ii.

AIXAPPNe (logo)® IBM ®MQSeriesOperating System/2

OS/2RedbooksRedbooks Logo RS/6000SecureWayWebSphere

Preface xi



http://www.redbooks.ibm.com/contacts.html

xii WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Part 1 Introduction

In this part, we introduce WebSphere Edge Server for Multiplatforms and its component load balancing and caching proxy functions.

Part 1

© Copyright IBM Corp. 2001 1

2 WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Chapter 1. WebSphere Edge Server for Multiplatforms concepts

This chapter provides an overview of WebSphere Edge Server and a description of its main components.

1


1.1 WebSphere Edge Server for MultiplatformsWebSphere Edge Server for Multiplatforms is Web infrastructure software that addresses the scalability, reliability and performance needs of e-business applications in both local and geographically distributed environments. Its functions incorporate robust, leading-edge caching and load balancing that together compensate for the inherent weakness of the Internet to support critical business applications and expectations. WebSphere Edge Server has been developed using IBM’s extensive experience with very demanding Web sites.

WebSphere Edge Server for Multiplatforms V1 replaces IBM WebSphere Performance Pack V3.0. The new WebSphere Edge Server is the latest IBM integrated solution for local and wide-area load balancing, content-based quality of service routing, and Web content filtering and caching for multi-vendor Web server environments.

WebSphere Edge Server is made up of two main components which allow you to reduce Web server congestion, increase content availability and improve Web server performance:

1. Caching and filtering

The caching and filtering component, known as Web Traffic Express (WTE), is a caching proxy server that provides highly scalable caching and filtering functions used in receiving requests and serving URLs. Since tunable caching is capable of supporting high cache hit rates, this component can reduce bandwidth costs and provide more consistent rapid customer response times.

2. Load balancing

The load balancing component, known as Network Dispatcher (ND), is a server that is able to dynamically monitor and balance TCP servers and applications in real time. The main advantage of the load balancing component is that it allows heavily accessed Web sites to increase capacity, since multiple TCP servers can be dynamically linked in a single entity that appears in the network as a single logical server.

These components can increase the scalability, availability and reliability of your Web site while reducing infrastructure costs.

Installation procedures allow the user to select which components to install, and on which machine(s) the selected component(s) should be located. Subject to installation needs and operating platforms, components can coexist on a single machine or can be distributed among multiple machines.


Refer to Chapter 3, “Network Dispatcher installation” on page 27 and Chapter 6, “Web Traffic Express installation” on page 247 for specific hardware and software requirements for WebSphere Edge Server for Multiplatforms components.

1.2 Caching and filteringWeb Traffic Express, the caching and filtering component of WebSphere Edge Server, is both a caching proxy server and a content filter. The advanced caching of this component minimizes network bandwidth utilization and ensures that end users spend less time when retrieving the same content multiple times.

This component acts as a gateway for multiple clients and performs basic Web server duties, such as receiving requests and serving URLs.

A traditional proxy server receives a request for a URL from a client and forwards it to the destination content server. Web Traffic Express does something more; it can save or cache the Web documents it retrieves, and serve subsequent requests for those documents from its local cache. The client gets the requested information faster and network bandwidth utilization is reduced.

This component of WebSphere Edge Server also offers other key features of advanced caching, such as:

� The ability to handle very large caches

� An option to automatically refresh the cache of the most frequently accessed pages

� The possibility to cache even those pages where the header information says to fetch them every time

� A configurable daily garbage collection, to improve server performance and ensure cache maintenance

� Remote Cache Access (RCA), a function that allows multiple Web Traffic Express machines to share the same cache, thereby reducing the redundancy of cached content

Chapter 1. WebSphere Edge Server for Multiplatforms concepts 5

Moreover, Web Traffic Express allows you to set content filtering at the proxy server level, rather than or in addition to the browser level, where content filtering could be easily compromised or overridden. This way, depending on the parameters used in the configuration, offensive contents will not be displayed on the client’s browser. Content filtering in Web Traffic Express can use:

� Platform for Internet Content Selection (PICS) rules guiding use of rating labels (such as the Recreational Software Advisory Council on the Internet (RSACI) criteria for inappropriate language, nudity or violence) placed in HTML or HTTP headers, or third-party content rating label distributions.

� A CyberPatrol filtering plug-in which allows Web content filtering based on the SurfControl database of acceptable and unacceptable Web sites.

� A lists of URLs and/or sites for which access is to be blocked.

1.3 Load balancing and server monitoring capabilitiesThe Internet has grown so rapidly over the last few years that you are probably looking for a way to handle your company's share of that traffic. If this growth is not properly handled, users get slow responses or refused connections; this may result in an unsatisfactory user experience, which may in turn cause the user never to visit your site again. Internet sites can become unstable or even fail under critical load conditions. What is needed is a solution that balances the load effectively and protects the user from these bad experiences.

Network Dispatcher, the load balancing component of WebSphere Edge Server, has been developed to address these limitations and provide customers with advanced functions to meet their site’s scalability and availability needs. It consists of three functions:

� the Dispatcher, which provides an advanced IP-level load-balancing mechanism for any TCP or UDP protocol.

� Interactive Session Support (ISS), an intelligent DNS server that can address the limitations of Round Robin DNS while still providing the same DNS interface as before for your clients.

� the Content Based Routing (CBR) function, which provides full-function load balancing based on information in the HTTP data stream (such as URLs, paths, cookies and so on) or in the POP3 and IMAP data streams.

These three functions can be deployed separately or together in various configurations to suit a wide variety of customer application requirements:


You can use the Dispatcher function to balance the load on servers within a local area network or a wide area network using a number of weights and measurements that are set dynamically.

ISS is a DNS-based load monitoring component (daemon) that can be installed on each of your servers. A group of daemons is called an ISS cell. One of the members of the cell becomes a spokesman for the load monitoring service.

You can use the ISS function to balance the load on servers within a local area network or a wide area network using a domain name server round-robin approach or a more advanced user-specified approach. ISS periodically monitors the level of activity on a group of servers and detects which server is the least heavily loaded.

ISS provides an observer interface to enable other applications to use the load monitoring service. Observers watch the cell and initiate actions based on the load. Application servers with the ISS load monitor daemon installed can pass periodic load reports to the Dispatcher using the Dispatcher observer. The results of these reports can be factored into the load balancing performed by the Dispatcher.

You can use the Content Based Routing function (which must be installed and configured to work together with Web Traffic Express) to load balance traffic based on the content of a client’s URL request. CBR also offers a Cookie Affinity feature that allows requests from a particular HTTP client session to be load balanced to the same server for a specified time period. The client session will maintain affinity for a particular server without relying on the IP address of the client.

These three components each offer a high availability feature:

� The Dispatcher’s high availability feature involves the use of a secondary machine that monitors the main, or primary, machine and stands by to take over the task of load balancing should the primary machine fail at any time. This feature is available on all the platforms where WebSphere Edge Server is supported, without using High Availability Cluster Multi-Processing (HACMP).

� In ISS high availability, all the nodes in a site work together to eliminate any single point of failure.

� Multiple CBR machines can be load balanced in turn using Network Dispatcher, therefore granting CBR high availability.

The high availability feature provided by the Dispatcher function can be successfully used even in other configurations, for example to guarantee firewall high availability.


1.4 What is new in WebSphere Edge ServerIn addition to the functions already available in WebSphere Edge Server V1, WebSphere Edge Server for Multiplatforms V1.0.3 enhancements include:

Network Dispatcher V3.6

� Gigabit Ethernet support

Network Dispatcher now supports 1 Gb (gigabit) Ethernet NICs (network interface cards) on all supported platforms: AIX, Red Hat Linux, Solaris, Windows NT and Windows 2000. Gigabit Ethernet is defined by the IEEE 802.3z standard. 802.3z is an enhancement of the IEEE 802.3 standard. 802.3z extends the 802.3 protocol and Media Access Control (MAC) specifications to allow for an operating speed of 1,000 Mb per second. Gigabit Ethernet uses the existing 802.3 frame format and the CSMA/CD access method.

On Solaris systems, only Ultra 60 servers support 1 Gb Ethernet NICs. SPARC does not support 1 Gb Ethernet NICs.

� Quad-port Ethernet support

Quad-port Ethernet NICs are supported on all platforms: AIX, Red Hat Linux, Solaris, Windows NT and Windows 2000. Quad-port NICs can operate in one of three modes. These modes are dependent upon the network switch to which the card will connect, the underlying operating system on which the NIC is installed, and the device driver provided by the vendor.

– Mode 1

1 Port/1 MAC mode. Each port is assigned a unique MAC address. The port, when plugged into the switch, becomes the endpoint of one physical link.

– Mode 2

Fault tolerant mode. Two or more ports are grouped into a team. One port is assigned a primary role, while remaining ports are given a backup role. The entire team has access to one MAC address, but only the primary port will route traffic over an active link with the switch. If the NIC fault tolerant device driver determines that the primary port is inoperable, the secondary port, which has been inactive, takes over for the primary port, and begins routing traffic to the switch. In this way, protocols that use the interface exposed by a fault tolerant device driver experience no interruption in service if a link fails between a port on the NIC and a port on the switch.


– Mode 3

Port aggregation mode, or trunking mode. Two or more ports are grouped into a team. Each team is assigned a MAC address. Protocols then access the team via a logical device driver which controls how the data is multiplexed between the ports in the team. The theoretical bandwidth of the team then becomes the number of ports multiplied by the bandwidth of a single port. This mode of operation requires special switch hardware and software to operate correctly.

Network Dispatcher supports quad-port NICs in Mode 1 only. Neither Mode 2 (fault tolerant mode) nor Mode 3 (port aggregation mode, or trunking mode) are supported.

� Solaris Version 8 support

In addition to supporting Solaris V2.6 and Solaris V7 (32-bit mode), Network Dispatcher now also supports Solaris V8 in 32-bit mode. Solaris V8 is 64-bit enabled; however, Network Dispatcher is supported in the 32-bit operating environment only.

� Red Hat Linux V6.2 support

The previous release of Network Dispatcher supported Red Hat Linux V6.1 (Linux kernel version 2.2.12-20, as well as any additional fixes to 2.2.12). This new release of Network Dispatcher adds support for Red Hat Linux V6.2 (Linux kernel version 2.2.16-3, as well as any additional fixes to 2.2.16).

� Collocated server support for Red Hat Linux

Collocation describes the environment where Network Dispatcher does not run on a dedicated machine. Instead, it is installed and runs on one of the server machines that is being load balanced. With this new release of Network Dispatcher, collocation is supported along with high availability at the same time on Red Hat Linux V6.1 and V6.2.

� Capacity utilization and bandwidth rules

The new capacity utilization feature allows Network Dispatcher to measure the amount of data delivered by each of its load balanced servers. It tracks capacity at the server, rule, port, cluster and executor levels by maintaining a new byte counter value for each level: kilobytes transferred per second.

New bandwidth rules allow you to balance loads based on the number of kilobytes per second being delivered by a set of servers. The reserved bandwidth rule allows you to allocate a specified bandwidth to sets of servers within your configuration. The shared bandwidth rule allows you to share a specified amount of bandwidth at the cluster or executor level.


� Server evaluation option

The server evaluation option is a new rule option of the ndcontrol rule command. It is used to evaluate the rule’s condition across only the servers within the rule (new support) or across all the servers within the port (previous support).

� Generic Routing Encapsulation (GRE) support

Generic Routing Encapsulation is an Internet protocol specified in RFCs 1701 and 1702. GRE provides a lightweight encapsulation method for IP and other protocols. Using GRE support, Network Dispatcher encapsulates client IP packets inside IP/GRE packets and forwards them to a server platform that supports GRE.

For example, many OS/390 and IBM ^zSeries customers use an OSA Express Network card to provide Internet-intranet connectivity to their mainframe hosts. The OSA Express configuration allows multiple LPARs to share one MAC address, but each has a unique TCP/IP address.

GRE support allows Network Dispatcher to load balance packets to multiple server addresses associated with one MAC address. Earlier versions of ND required a one-to-one correspondence between the MAC address and server address.

Network Dispatcher implements GRE as part of its Wide Area Network Dispatcher (WAND) feature, thus providing wide area load balancing directly to any server system that can unwrap GRE packets. ND does not need to be installed at the remote site if the remote servers support encapsulated GRE packets.

See 5.5, “Wide area Network Dispatcher support” on page 191 for an example of configuring Wide Area Network Dispatcher support.

� Advisor fast-failure detection

The Advisor is a lightweight client that runs as part of Network Dispatcher. It periodically executes a command to determine the status of the servers. If the command is successful, a server is considered “up”. If it fails, a server is considered “down.” In either case, the Advisor informs the Network Dispatcher Manager function. The Manager sets weights for the servers to determine which server should receive new session requests. A down server is given a weight of zero and packets will not be forwarded to it until the server responds to the Advisor.

In previous releases of Network Dispatcher, an advisor looped through all the servers configured on a cluster or port. It gathered information on each server and then informed the Manager with a complete update of all servers. With this enhancement, the Manager is updated incrementally after each server is evaluated.


In addition, the timeout values used to determine how long an advisor will wait before declaring a server down are no longer hard coded. The user is now able to configure two timeout values: connecttimeout and receivetimeout. These timers allow the user to configure how generously or aggressively the Advisor marks a server as down.

� Quiesce enhancement for sticky connections

The quiesce command is used to remove a server from the Network Dispatcher configuration for any reason. The quiesce command allows all existing connections to be completed, but prevents any new ones from being sent to the quiesced server.

The quiesce command in previous versions of Network Dispatcher did not recognize existing connections that used the affinity/sticky feature. When a server was quiesced, all connections within the stickytime would go to a new server.

The quiesce enhancement extends the server quiesce function to recognize existing connections that have the affinity/sticky feature. For example, if a server is quiesced and an existing connection has affinity to the server, Network Dispatcher forwards subsequent new connections from that client to the quiesced server as long as the subsequent new connections arrive before the stickytime expires.

This enhancement provides a graceful, less abrupt handling of sticky connections when quiescing servers. You can quiesce a server and wait for the time when there is the least amount of traffic to completely remove the server from the configuration.

� Enhanced Interactive Session Support (ISS) load balancing of Network Dispatchers

In a two-tiered ISS configuration, an ISS DNS monitor determines how to load balance across a lower tier of Network Dispatcher machines. With the previous version of Network Dispatcher, load information provided by scripts running on the ND machines furnished load and CPU information on the ND machine, not the pool of servers behind it.

This new version provides a Network Dispatcher self advisor that collects load status information on the load balanced back end servers. This self advisor specifically measures the rate of connections per second on the back end servers and writes the results to the ndloadstat file.

Network Dispatcher also provides an external metric called ndload. The ISS agent on each Network Dispatcher machine calls the ndload script which extracts a string from the ndloadstat file and returns it to the ISS agent. Each ISS agent on each of the lower tier Network Dispatcher machines returns the load status value to the ISS monitor for use in determining which Network Dispatcher to forward client requests to.


� SSL proxy-to-server support

In previous releases of Network Dispatcher when implementing Content Based Routing (CBR), Secure Sockets Layer (SSL) connections were supported only between the client and the proxy host (WTE). Only HTTP traffic was supported between the proxy and the server.

This release of Network Dispatcher extends CBR to support SSL from the proxy to the server, which allows for complete SSL connections from the client to the server. CBR continues to support client-to-proxy SSL and proxy-to-server HTTP along with proxy-to-server SSL.

� Java 1.3 support

The previous release of Network Dispatcher ran on Java 1.1.8. This release of Network Dispatcher requires Java 1.3 on all supported platforms. The requirements for each platform are documented in Chapter 3, “Network Dispatcher installation” on page 27. Java 1.1.x is no longer supported.

Web Traffic Express V3.6

� Additional supported system types

This release of WTE includes support for Red Hat Linux 6.2 (kernel 2.2.16) and SuSE Linux 6.4 (kernel 2.2.14).

In addition, this release of WTE adds support for the Solaris 8 operating system running in 32-bit mode. Only SPARC and Ultra-SPARC chipsets are supported. Solaris 8 on an x86 chipset is not supported.

� Client-side certificate authentication

Previous versions of WTE support using SSL to authenticate a server and use encryption for data passed between the client and the server.

This release of WTE includes support for client authentication. WTE will retrieve a client certificate through an SSL session and use it to authenticate the client.

� Disabling listener threads when SSL is enabled

When SSL is enabled in WTE, a listener thread is enabled for the SSL port (typically port 443). However, in previous releases of WTE, enabling SSL did not disable the listener threads for standard HTTP requests (typically ports 80 and 8080); running active HTTP listener threads can possibly make a site insecure on a server performing secure transactions.

This release of WTE includes a new directive to disable listener threads for standard HTTP requests. The ability to turn off these listener threads enhances server security.


� Use of IP ranges in protection masks

This enhancement allows administrators to more easily configure WTE to authenticate requests from specified IP addresses using IP ranges. This avoids the need to create very large protection rule files that could adversely impact proxy performance.

� Requests over persistent connections

WTE can be configured to support persistent client-server connections. With a persistent connection, the WTE server accepts multiple requests from the client and sends responses over the same TCP/IP connection. Use of persistent connections reduces latency for clients and lowers CPU load on the WTE proxy server, but it requires more resources in terms of server memory and process threads.

Previous releases of WTE support persistent connections for GET requests only. This release of WTE adds support for POST and PUT requests to be handled over persistent connections.

� JavaServer Pages (JSP) and servlet response caching

This new function enables WTE to cache JavaServer Pages (JSP) and servlet-generated HTML from a WebSphere Application Server (WAS) that is configured with a WTE plug-in module. This function makes use of the WAS dynamic caching (dynacache) capabilities introduced in WebSphere Application Server V3.5. WTE retrieves JSP-created HTML from the WAS dynamic cache. Dynamic content can then be cached on the edge of the network, freeing the content host from repeatedly retrieving files from the application server to satisfy requests for those files from different clients.

� Lightweight Directory Access Protocol (LDAP) authorization plug-in

This release of WTE includes an LDAP plug-in which enables WTE to authorize clients by using an LDAP server and user-defined policy information. This is done by directing the authorization routine within WTE to retrieve data from an LDAP server rather than from the WTE registry.

The authorization routine, which is called first, verifies access to WTE objects granted by user-defined security policy information.

The authentication routine, which is called second, uses the LDAP server database as a WTE registry instead of the original WTE registry.

A consistent pop-up window prompting for user name and password is provided using either the LDAP server or the WTE registry. Access through WTE can be further refined by using a security policy, which allows system administrators to grant or deny access according to defined IP addresses, host names, or group or organizational units within an LDAP database.


� Internet Caching Protocol (ICP) plug-in

The Internet Caching Protocol allows for the sharing of caches in a multiple proxy configuration. The ICP plug-in enables WTE to query ICP compliant caches in search of HTML pages and other cacheable resources.

When the WTE server receives an HTTP request, it searches its own cache for the resource. If the resource is not found in the local cache and the ICP plug-in is enabled, WTE wraps the URL within an ICP query packet and delivers this packet to all identified ICP peer caches (including other WTE servers with the enabled ICP plug-in).

If no peers respond with hits, the WTE server that received the original request continues to process the request by using other search methods, or simply retrieves the requested URL itself.

� CyberPatrol filtering plug-in

The CyberPatrol filtering plug-in allows Web content filtering based on the SurfControl database of acceptable and unacceptable Web sites. SurfControl maintains a database of Web sites, the content of which can be subject to filtering. The database is assembled by SurfControl staff who decides whether a Web site should be included in any of their filtering categories.

Filters can be selected to block access to Web sites that are listed in negative categories, or they can be set to allow access only to Web sites included in the positive categories. WTE allows you to specify additional Web sites to block or to allow.


Chapter 2. What WebSphere Edge Server can do for you

In this chapter, we review several configurations demonstrating how WebSphere Edge Server components can fit into a network configuration to provide the benefits of load balancing and traffic reduction.

2


2.1 Building high performance Web sitesWebSphere Edge Server for Multiplatforms allows you to design several architectures to enhance the performance of your Web site. Figure 2-1 offers an idea of the multiple configurations that can be obtained combining the WebSphere Edge Server components:

Figure 2-1 A sample WebSphere Edge Server environment

� The Web Traffic Express (WTE) component minimizes response time and network bandwidth utilization by providing Web content caching. It also ensures reliable content filtering at the proxy server level. Multiple Web Traffic Express servers can be load balanced.

� The Network Dispatcher (ND) component distributes the load between multiple clustered Web servers and Web Traffic Express servers. As soon as a client request arrives, the load balancing machine uses sophisticated monitoring tools and forwards the request to the least loaded server. High availability is provided by a backup Dispatcher machine, which monitors the state of the primary machine and takes over if it fails.

� The Web server selected by the Network Dispatcher machine can respond directly to the client without any further involvement of the Network Dispatcher machine. Since there is no need for the server response to go back through the same physical path, a separate high-bandwidth connection can be used.

Client

ND Primary

Web Servers

Internet

ND BackupND Primary

ND Backup WTE

WTE


IBM used the WebSphere Edge Server technology to create a scalable and reliable system that efficiently handled unprecedented traffic volumes. On February 17th, 1998, at 12:41 (Japan Standard Time), the official Web site of the Olympic Winter Games in Nagano logged 98,226 hits per minute. Less than a week later, a peak load of more than 103,400 hits per minute was supported while still providing normal response time.

By using WebSphere Edge Server, you will have an efficient Web site, capable of providing fast responses and able to handle very large amounts of simultaneous requests without any major delay. Furthermore, the high availability features built into WebSphere Edge Server will make the Web services available even if one or more of your server machines should unexpectedly fail.

The following figure shows an example of what your Web customers do not see if your Web site uses WebSphere Edge Server:

Figure 2-2 A window that can be prevented

2.2 Who can benefitWebSphere Edge Server allows you to design and use multiple architectures. The scenarios described in this section provide specific examples of how various ISP implementations can benefit from the use of WebSphere Edge Server.

Chapter 2. What WebSphere Edge Server can do for you 17

2.2.1 Content hosting Internet Service ProvidersContent-hosting ISPs can use WebSphere Edge Server to support and distribute the content from their own Web sites more efficiently, and to provide more response- and cost-effective access to other sites (see figure below).

Figure 2-3 Content-hosting Internet Service Providers

Within a content-hosting server farm, the WebSphere Edge Server components can be configured to provide high availability and accessibility as follows:

� The scalability of Web sites can be achieved by adding multiple HTTP servers and by using efficient load balancing to dispatch requests to the HTTP server with the best capacity to handle the requests. Network Dispatcher, the load balancing component of WebSphere Edge Server, guarantees improved availability and scalability by allowing a farm of Web servers to provide a single Web site image to clients.

� High availability can be maintained by configuring a hot standby Network Dispatcher component.

� Proxy caching can be used to provide quicker access to content from other sites, as well as to optimize the backbone network traffic capacity.

� Both availability and performance may be further enhanced by geographically distributing HTTP servers, together with proxy caching for other Internet content closer to user access points, such as points of presence (POPs) and network access points (NAPs).

WebClient

Gatewayor

Firewall

Network DispatcherLoad-Balancing

Primary

HTTP Server A

HTTP Server B

HTTP Server C


Backup

Internet

HTTP Server E

HTTP Server D


Content-hosting service providers or corporate webmasters can therefore benefit from local and wide area load balancing, and proxy caching. The flexible configuration of these components can ensure that requests are directed to the most appropriate local or remote location, and can allow location outages or routine maintenance schedules to be handled without disrupting the customers’ activity.

The WebSphere Edge Server components can be used in conjunction with firewalls and authentication gateways to provide secure access where desired, and the load balancing function of WebSphere Edge Server can also be used to scale these capabilities.

2.2.2 Corporate Web sites and content aggregatorsUse of WebSphere Edge Server by corporate Web sites and content aggregators is similar to that of content hosting ISPs.

Figure 2-4 Corporate Web sites and content aggregators

In many cases, corporate Web sites and content aggregators need to maintain a demilitarized zone (DMZ) to ensure that access to Web content by employees, business partners, or customers does not expose internal computer resources to unauthorized users or hackers. Here, the load balancing Network Dispatcher function can be used to provide the degree of scalability necessary to satisfy users. In addition, Web Traffic Express caching and filtering proxy servers may be deployed on the same machines or on separate systems to filter or optimize access from within the corporation to external Web sites.


Primary

HTTPServer A

HTTPServer D

WebClient

Gatewayor

Firewall

HTTPServer B

HTTPServer C

Web Traffic ExpressCaching

Reverse Proxy


Backup

Web Traffic ExpressCaching

Reverse Proxy

Internet


Many corporate Web sites are located at the head office or in regional locations, but branch offices or business partners may need frequent access to the content. Most offices of this type have relatively low line speed (somewhat less than 1.5 Mbps) network access to the regional or corporate sites. Relatively few users can concurrently use all the available bandwidth, and the response times become erratic. At these locations, HTTP servers with general purpose caching can provide more consistent user response time.

Figure 2-5 Home office-branch office configuration

Some industries also have small branch offices in addition to larger branch or regional offices. A simple deployment of Web Traffic Express caching and filtering on a single platform, eventually combined with a firewall, is probably sufficient to provide more consistent response time for employees in the small branch offices. In the larger branches or regional offices, it may be desirable to have more than one Web Traffic Express proxy server and to use the load balancing Network Dispatcher functionality to provide better resource management.

Two approaches can be used to reduce redundant page storage and to address the effectiveness of caching in these locations:

1. Hierarchical caching


Backup


Primary

WebClient

Gatewayor

Firewall

HTTP Server A

Internetor

IntranetGatewayor

Firewall

HTTP Server

Web Traffic ExpressCaching & Filtering

Proxy

Gatewayor

Firewall

Corporate Head OfficeBranch Office

HTTP Server BHTTP Server C


Primary


Backup


ProxyWebClient


Optimize a primary (initial) cache for higher page hit rate by favoring more files of a smaller size, and a hierarchical cache for higher hit byte rate by favoring fewer files of a larger size.

2. Remote Cache Access (RCA)

Configure RCA together with the load balancing function of Network Dispatcher to reduce the index size of each proxy and to increase the scalability of the caching storage.

2.2.3 Backbone Internet Service ProvidersBackbone ISPs typically provide co-location and/or peering services for other ISPs in addition to content hosting for large national or international corporations. In many cases, they may provide virtual ISP services for other service providers such as content-hosting ISPs. Backbone ISP customers are increasingly demanding both high availability and differentiated service levels, and backbone ISPs are responding by enhancing their Internet infrastructures. Features demanded from backbone ISPs include:

� Load balancing for a variety of traffic (for example, mail and FTP in addition to Web traffic)

In addition, requests are made for traffic balancing management and high availability for authentication servers and management systems.

� Proxy caching

To minimize the effects of hot potato routing and Web traffic surges, backbone ISPs are typically installing highly scalable caches at major peering points and network interconnections points, such as network access points (NAPs).


Figure 2-6 Backbone ISPs

International ISPs in particular need caching to reduce the costs associated with transoceanic links. Such installations can make very effective use of the RCA feature of Web Traffic Express, a variation of the traditional caching function that, when used together with load balancing and shared file storage, can dramatically reduce page storage redundancy.

Typical configurations for backbone ISPs include load balancing for authentication gateways (such as authentication servers and subscriber management application servers) as well as for WTE caching proxy servers. Because of the amount of traffic and the desire for high availability, such caching servers would likely use the remote cache access (RCA) feature of Web Traffic Express. This feature enables all of the WTE servers in a defined group to share the contents of their caches with one another.

2.2.4 Access Internet Service ProvidersAccess Internet Service Providers (Access ISPs) need to provide more consistent response time to their customers and to conserve their backbone network link and access charges. Caching at POPs addresses these needs. These configurations are similar to those for backbone ISP solutions (see figure below).

Gatewayor

Firewall


Proxy


Proxy

WebClient

InternetWeb

Servers

Gatewayor

Firewall


Proxy


Primary


Backup


Because access service providers target many of the small- to medium-size business customers, there is also an opportunity for them to create revenue producing services by deploying smaller caching devices on customer premises, but configuring and managing them as an ISP service. For this scenario, the caching would look much as it does for corporate branch offices.

Figure 2-7 Access Internet Service Providers

Gatewayor

Firewall


Proxy


Proxy

WebClient

ISP'sIntranet

Gatewayor

Firewall


Proxy


Primary


Backup

Gatewayor

Firewall

Gatewayor

FirewallInternetWeb

Servers

Gatewayor

Firewall


Proxy


Proxy

WebClient


Proxy


Backup


Primary


Part 2 Network Dispatcher

In this part, we describe the Network Dispatcher component of WebSphere Edge Server for Multiplatforms, which is also referred to in many publications as the load balancer.

Part 2


Chapter 3. Network Dispatcher installation

This chapter covers the installation steps for Network Dispatcher in the following environments:

� AIX: see 3.1, “Installing Network Dispatcher on AIX” on page 28

� Windows NT: see 3.2, “Installing Network Dispatcher on Windows NT” on page 40

� Linux: see 3.3, “Installing Network Dispatcher on Linux” on page 55

In addition, procedures for uninstallation and starting and stopping services are also covered.

3


3.1 Installing Network Dispatcher on AIXIn this section, we describe the requirements for installing Network Dispatcher on a machine running AIX. We explain the procedure for installation and uninstallation, and how to start using Network Dispatcher.

3.1.1 RequirementsThese are the required components for installing and running Network Dispatcher on AIX:

� Any RS/6000-based machine

� IBM AIX Version 4.3.3.10 or higher, plus the Java required PTFs

� IBM AIX Developer Kit, Java 2 Technology Edition, Version 1.3.0 for the Java Runtime Environment. You must download both the Developer Kit installable package and the Runtime Environment installable package prior to running the InstallShield program.1

� 50 MB of available disk space for installation

Note: Additional disk space will be needed for logs

� Any of the following Network Interface Cards (NICs) supported by the TCP/IP protocol stack to make network connections:

– 16 Mb Token ring– 10 Mb Ethernet– 100 Mb Ethernet– 1 Gb Ethernet– Quad port Ethernet– Fiber distributed data interface (FDDI)

� Web Traffic Express Version 2.0 or higher if you are using the CBR component for load balancing HTTP traffic

� Netscape Navigator Version 4.07 (or higher) or Netscape Communicator Version 4.61 (or higher) for viewing online help

1 Go to http://www-105.ibm.com/developerworks/tools.nsf/dw/java-devkits-byname?OpenDocument&Count=100 (the IBM developerWorks tools and products page) to download the appropriate Java technology runtime environment.

Note: The installation process for AIX does not prompt you for the path to install the software. It is automatically installed in the directory /usr/lpp/nd, so make sure that there is enough free space on the proper file system before installing the product.


http://www-105.ibm.com/developerworks/tools.nsf/dw/java-devkits-byname?OpenDocument&Count=100

3.1.2 Choosing the componentsBefore beginning the installation, we need to discuss the Network Dispatcher components that you can choose to install.

During installation, you must choose one of the three installation methods listed below (refer to Figure 3-3 on page 34). Depending on the installation method you choose, you will have a different set of components to select for installation.

The installation methods are:

1. Express: installs only the Dispatcher component, and does not configure it.

2. Guided: installs one of the following components:

– Remote administration client

This is the complete Network Dispatcher graphical user interface (GUI), which, when installed, allows you to remotely manage a Network Dispatcher server.

– Dispatcher component

This is the main component for load balancing and high availability.

– CBR for IMAP/POP3

This is the Content Based Routing component for IMAP and POP3.

– CBR for HTTP

This is the Content Based Routing component for HTTP (this component needs Web Traffic Express).

You can also install those same components using the custom install method (see below), but the Guided install uses a wizard that prompts you for information to create a basic configuration during installation.

3. Custom: when selecting the custom method, you must select which of the following components you want to install:

– Dispatcher runtime

This is the base component for load balancing and high availability. By selecting this option, the next two options are automatically selected too.

– Dispatcher administration

This is the graphical user interface (GUI) which allows you to configure a server running the Dispatcher runtime (locally or remotely).

– Dispatcher license

This option allows you to copy the Dispatcher component license into the server.

Chapter 3. Network Dispatcher installation 29

– Interactive Session Support (ISS) runtime

This is the base component for Interactive Session Support. If this option is selected, the next two options are automatically selected also.

– Interactive Session Support administration

This is the graphical user interface (GUI) which allows you to configure a server running the ISS runtime (locally or remotely).

– Interactive Session Support license

This option allows you to copy the ISS component license into the server.

– Content Based Routing (CBR) runtime

This is the base component for Content Based Routing. If this option is selected, the next two options are automatically selected also.

– Content Based Routing administration

This is the graphical user interface (GUI) which allows you to configure a server running the CBR runtime (locally or remotely).

– Content Based Routing license

This option allows you to copy the CBR component license into the server.

– Documentation

This option is used to install the product manuals.

3.1.3 InstallationAIX 4.3.3 is shipped with Java Development Kit and Java runtime V1.1.8. As noted previously (refer to “Java 1.3 support” on page 12), this version is no longer supported by Network Dispatcher. You must install IBM Java Development Kit (JDK) V1.3.0, which is available for download on the developerWorks site, at the following URL:

http://www.ibm.com/developerworks/java/jdk/index.html

JDK 1.3.0 requires the following filesets to be installed on the machine at the levels listed below:

� X11.fnt.fontServer.4.3.3.12.bff

� X11.base.lib.4.3.3.15.bff

� X11.base.rte.4.3.3.14.bff

� X11.base.rte.4.3.3.10.bff

� X11.motif.lib.4.3.3.15.bff



� bos.rte.libc.4.3.3.15.bff

� devices.isa_sio.baud.rte.4.3.2.1.bff

These fixes are also available from the Web site mentioned above.

After you uninstall JDK 1.1.8 and install the AIX fixes, you can install JDK 1.3.0. The file we downloaded from the developerWorks site is Java130.rte.

You can use SMIT to install JDK. Run smit install_latest, then type in the path where you downloaded the Java130.rte file, click Enter, and on the next screen click Enter again.

When software installation is complete, update the PATH variable to include the JDK path. You can edit the /etc/environment file, locate the definition of the PATH variable, and add /usr/java130/bin to it, as shown in Example 3-1.

Example 3-1 Updating the PATH variablePATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java130/binexport PATH

Log out and log in again so that your user has the updated PATH. Use the following command to test JDK.

Example 3-2 Testing the Java runtime# java -versionjava version "1.3.0"Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0)Classic VM (build 1.3.0, J2RE 1.3.0 IBM build ca130-20000622 (JIT enabled: jitc))

Once you have JDK running, you can start the Network Dispatcher installation. Mount the WebSphere Edge Server CD-ROM in the CD-ROM drive (in case it is currently unmounted) and run the setup script to start the installation wizard as follows:

Example 3-3 Starting the installation wizardmkdir /cdrommount -v cdrfs -p -r /dev/cd0 /cdromcd /cdrom./setup

Important: The installation wizard requires an X-Windows server. If you do not have a graphics adapter on your machine, make sure you can use Telnet to access it and export the display to another machine which is running an X-Windows server.


Once the installation wizard is started, the first thing you need to do is to select the language of the software. We selected English, as shown in Figure 3-1.

Figure 3-1 Selecting the language


Next, you need to select which component of WebSphere Edge Server you are installing. Select Network Dispatcher, and click Next (see Figure 3-2).

Figure 3-2 Selecting the component to be installed


On the next window, you need to select the installation method; we selected the Custom method, as shown in Figure 3-3. For more details about the installation methods refer to 3.1.2, “Choosing the components” on page 29.

Figure 3-3 Selecting the installation method


In the next window (shown in Figure 3-4), select all the components to be installed.

Figure 3-4 Selecting the software components to install


The next window shows a summary of the selected options to install. If the listing is correct, click Next to proceed and start the installation.

Figure 3-5 Summary of the selected options


After the software is copied to the machine, you are presented with a summary of the installation, as shown in Figure 3-6.

Figure 3-6 Installation summary


The last window will display a reference to the readme file, which is shipped with the installation media. We recommend that you read this file carefully, since it contains important, last-minute changes and information about the product.

Figure 3-7 Reference to the readme file

3.1.4 Starting the softwareBefore using the Network Dispatcher GUI, you need to start the Network Dispatcher server. You can do it by running the following command:

# ndserver

If you want to stop the server, run the following command:

# ndserver stop

Use the ndadmin command to start the Network Dispatcher GUI:

# ndadmin


The Network Dispatcher GUI is shown in Figure 3-8.

Figure 3-8 Network Dispatcher GUI

If you want to begin configuring Network Dispatcher, refer to Chapter 4, “Network Dispatcher basic configuration” on page 69 for a basic scenario configuration.

3.1.5 UninstallationThe recommended procedure to uninstall the software is to use the same setup script you used to install it. When uninstalling, you need to use the -nduninstall option, as shown below:


Example 3-4 Starting the uninstallation# mkdir /cdrom# mount -v cdrfs -p -r /dev/cd0 /cdrom# cd /cdrom# ./setup -nduninstallRunning ND Uninstall...Details on the uninstall are located in the log file: /usr/lpp/nd/uninstall.logND Uninstall complete.

All the steps performed in the uninstallation processes are recorded on the log file /usr/lpp/nd/uninstall.log.

The uninstall process does not completely remove the software. If you want to completely remove it, you need to delete the installation directory (in AIX it is /usr/lpp/nd). Do not remove this directory if you plan to install Network Dispatcher on this machine again, since it keeps the configuration files even after you uninstall the software.

3.2 Installing Network Dispatcher on Windows NTIn this section, we cover the steps to install Network Dispatcher on Windows NT, we discuss some post-installation checks you need to perform and the steps to uninstall the software.

3.2.1 RequirementsThese are the components required to install and run Network Dispatcher on Windows NT or Windows 2000:

� Any Intel x86 PC supported by Microsoft Windows 2000 or Microsoft Windows NT

� Windows 2000 Professional, Server or Advanced Server; or Windows NT Workstation or Server, Version 4.0 with Service Pack 5 or higher

Note: Windows Server is required for machines running the NameServer Observer mode of ISS.

� IBM Cross Platform Technologies for Windows V2.0 (Java SDK 1.3). You must download both the Developer Kit installable package and the Java Runtime Environment installable package prior to running the InstallShield program.2


2 Go to http://www.ibm.com/developerworks/java/jdk/index.html to download the appropriate Java SDK.



Note: Additional disk space will be needed for logs.

� Any of the following Network Interface Cards (NICs) supported by the TCP/IP protocol stack to make network connections:

– 16 Mb Token ring– 10 Mb Ethernet– 100 Mb Ethernet– 1 Gb Ethernet– Quad port Ethernet

� Web Traffic Express Version 2.0 or higher if you are using the CBR component for load balancing HTTP traffic

� Netscape Navigator Version 4.07 (or higher), Netscape Communicator Version 4.61 (or higher) or Internet Explorer 4.0 (or higher) for viewing online help

3.2.2 InstallationBefore installing Network Dispatcher, you must install the Java Development Kit (JDK) V1.3.0, which is available for download on the developerWorks site, at the following URL:


The downloaded package contains an install.exe executable file. Run this file to start the installation, as shown in Figure 3-9. Make sure to use the path in which you downloaded the files from the Internet.

Figure 3-9 Installing JDK

After installing JDK, make sure that the Path environment variable was updated to include the JDK path. To do so, click Start->Settings->Control Panel->System, select the folder Environment and locate the variable path. Make sure that the path to the bin directory of JDK was added to the value of the variable path (in our example, it is C:\Program Files\Ibm\Java13\jre\bin), as shown in Figure 3-10.



Figure 3-10 Path after installing JDK

Once JDK is installed, you can begin the installation of Network Dispatcher. Click Start->Run... and select the setup.exe file from the WebSphere Edge Server installation media, as seen below.

Figure 3-11 Starting the installation


Once the installation wizard is started, the first thing you must do is to select the language. We selected English, as shown in Figure 3-12.



Next, you must select which component of WebSphere Edge Server you are installing. Select Network Dispatcher, and click Next.



In the next step, you must select a path where Network Dispatcher should be installed. We recommend that you use the default path.

Figure 3-14 Selecting the path where the product will be installed


In the next window, you select the installation method; we selected the custom method, as shown in Figure 3-15. For more details on the installation methods, refer to 3.1.2, “Choosing the components” on page 29.

Figure 3-15 Choosing the installation method


After installing the software, you are presented with a summary of the installation, as shown in Figure 3-18.

Figure 3-18 Summary of the installation


The last window displays a reference to the readme file, which is shipped with the installation media. We recommend that you read this file carefully since it contains important, last-minute changes and information about the product.


Note that you need to reboot the machine after the installation process is completed.


3.2.3 Starting the softwareOnce the installation is finished, we recommend that you check a few items in the environment before beginning to use the graphical user interface (GUI) to configure Network Dispatcher.

After rebooting the machine, make sure that the Network Dispatcher service is started. Click Start->Settings->Control Panel->Services, and locate the IBM Network Dispatcher service. The status of this service must be Started, as shown in Figure 3-20.

Figure 3-20 IBM Network Dispatcher service


The second thing you need to check is the IBMNDPATH environment variable. Click Start->Settings->Control Panel->System, choose the Environment tab and locate the variable IBMNDPATH. Make sure that its value corresponds to the path where the product was installed (see Figure 3-21).

Figure 3-21 The IBMNDPATH variable


Next, you can use the GUI to configure the software. Click Start->Programs->IBM WebSphere->Edge Server->IBM Network Dispatcher to open the GUI. The Network Dispatcher GUI main window is shown in Figure 3-22.

Figure 3-22 Network Dispatcher GUI main window

If you are ready to begin configuring Network Dispatcher, refer to Chapter 4, “Network Dispatcher basic configuration” on page 69 for a basic scenario configuration.


3.2.4 UninstallationIf you want to uninstall Network Dispatcher, click Start->Settings->Control Panel->Add/Remove Programs and select IBM Network Dispatcher from the Install/Uninstall tab, as shown in Figure 3-23. Click the Add/Remove button, and the software uninstallation process will be started.

Figure 3-23 Uninstalling the software

The system shows the progress of the uninstallation, and when it finishes, it prompts you for a reboot.

The uninstall process does not completely remove the software. If you want to completely remove the software, you must delete the installation directory (in our example, C:\Program Files\Ibm\nd) and the following registry entries:

� HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Enum\Root\LEGACY_IBMNETDISP

� HKEY_LOCAL_MACHINE\SYSTEM\ControlSet002\Enum\Root\LEGACY_IBMNETDISP

� HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\Root\LEGACY_IBMNETDISP


The files kept in the installation directory are the log files and the configuration files for each component.

Do not remove either the directory or the registry entries if you plan to reinstall Network Dispatcher again on this machine.

3.3 Installing Network Dispatcher on LinuxIn this section, we describe the requirements for Network Dispatcher to be installed on a machine running Linux. We explain the steps involved in installing and uninstalling Network Dispatcher and how to start the software.

3.3.1 RequirementsThese are the required components to install and run Network Dispatcher on Linux:

� Any IBM-compatible PC or PS/2 (including SMP models) supported by one of the following Linux distributions:

– Red Hat Linux Version 6.1 (Linux kernel Version 2.2.12-20)– Red Hat Linux Version 6.2 (Linux kernel Version 2.2.16-3)


Note: Additional disk space will be needed for logs

� The following Network Interface Cards (NICs) supported by the TCP/IP protocol stack to make network connections:

– 10 Mb Ethernet– 100 Mb Ethernet– 1 Gb Ethernet– Quad port Ethernet

� A version of the Korn Shell (ksh), which must be installed

� IBM Runtime Environment for Linux, Java 2 Technology Edition, V1.3.3

� Web Traffic Express Version 2.0 or higher if you are using the Content Based Routing (CBR) component for load balancing HTTP traffic

Note: When using the CBR component, you must use Network Dispatcher V3.6.0.1 along with IBMRuntime Environment for Linux, Java 2 Technology Edition V1.3-1.0.

3 Go to http://www.ibm.com/developerworks/java/jdk/index.html to download the appropriate Linux Java technologyruntime environment.



� Netscape Navigator Version 4.07 (or higher) or Netscape Communicator Version 4.61 (or higher) for viewing online help

3.3.2 Choosing the componentsBefore beginning the installation, we need to discuss the Network Dispatcher components which you can choose to install.

During the installation, you must choose one of the three installation methods listed below (refer to Figure 3-26 on page 61). Depending on the method you choose, you will have a different set of components to select for installation.

The installation methods are:

1. Express: installs only the Dispatcher component, and does not configure it.

2. Guided: installs one of the following components:

– Remote administration client

This is the complete Network Dispatcher graphical user interface (GUI), which, once installed, allows you to remotely manage a Network Dispatcher server.

– Dispatcher component

This is the main component for load balancing and high availability.

– CBR for IMAP/POP3

This is the Content Based Routing component for IMAP and POP3.

– CBR for HTTP

This is the Content Based Routing component for HTTP (this component needs Web Traffic Express).

You can also install those same components using the custom install, but the guided install uses a wizard that prompts you for information to create a basic configuration.

Note: The installation process for Linux does not prompt you for the path to install the software. It is automatically installed in the directory /opt/nd, so make sure that there is enough free space on the proper file system before installing the product.


3. Custom: if you select the custom method, you must select which of the following components you want to install:

– Dispatcher runtime

This is the base component for load balancing and high availability. If this option is selected, the next two options are automatically selected also.

– Dispatcher administration

This is the graphical user interface (GUI) which allows you to configure a server running the Dispatcher runtime (locally or remotely).

– Dispatcher license

This option allows you to copy the Dispatcher component license into the server.

– Content Based Routing runtime

This is the base component for Content Based Routing (CBR). If this option is selected, the next two options are automatically selected also.

– Content Based Routing administration

This is the graphical user interface (GUI) which allows you to configure a server running the CBR runtime (locally or remotely).

– Content Based Routing license

This option allows you to copy the CBR component license into the server.

– System Monitor Agent

This is the component for System Monitor Agent (SMA). For more information about SMA, refer to IBM Network Dispatcher User’s Guide Version 3.0 for Multiplatforms, GC31-8496-05.

– Documentation

This option is used to install the product manuals.

3.3.3 InstallationBefore installing Network Dispatcher, you must install the Java Development Kit (JDK) or Java Runtime Environment V1.3.0, which are both available for download on the developerWorks site, at the following URL:


From the site above, we downloaded the following files:

� Java Development Kit: IBMJava2-SDK-1.3-6.0.i386.rpm

� Java Runtime: IBMJava2-JRE-1.3-6.0.i386.rpm



You can choose which one you prefer to install (do not install both!). We installed JDK, using the following command:

# rpm -ivh IBMJava2-SDK-1.3-6.0.i386.rpm

This package is installed in the directory /opt/IBMJava2-13. If you install Java Runtime, the install path is the same, but the binary path is /opt/IBMJava2-13/jre/bin.

You can test if the Java installation was successful by running the following command (see example below):

Example 3-5 Testing Java Runtime# /opt/IBMJava2-13/bin/java -versionjava version "1.3.0"Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0)Classic VM (build 1.3.0, J2RE 1.3.0 IBM build cx130-20010207 (JIT enabled: jitc)

Before proceeding to the Network Dispatcher installation, you need to set the environment variable JAVA_HOME to the path where JDK was installed and update the path to include the JDK bin directory, as shown in Example 3-6.

Example 3-6 Updating the environment variablesexport JAVA_HOME=/opt/IBMJava2-13export PATH=$JAVA_HOME/bin:$PATH

We recommend that you add the commands shown in Figure 3-6 to the /etc/profile file to make sure that those environment variables will be set every time you log in.

The next step is to install Network Dispatcher. First, you must mount the CD-ROM drive (in case it is unmounted) and run the setup script, as shown in Example 3-7.

Example 3-7 Starting the installationmount /cdromcd /mnt/cdrom/./setup

Important: The installation wizard requires a X-Windows server. If you do not have a graphics adapter on your machine, make sure you can use Telnet to access it and export the display to another machine which is running a X-Windows server.


Once the installation wizard is started, the first thing you need to do is to select the language of the software. We selected English, as shown in Figure 3-24.



Next, you must select which component of WebSphere Edge Server you are installing. Select Network Dispatcher, and click Next (see Figure 3-25).



In the next window, you need to select the installation method; we selected the custom method, as shown in Figure 3-26. For more details on the installation methods, refer to 3.3.2, “Choosing the components” on page 56.

Figure 3-26 Selecting the installation method


After installing the software, you are presented with a summary of the installation, as shown in Figure 3-29.

Figure 3-29 Summary of the installation


The last window will display a reference to the readme file, which is shipped with the installation media. We recommend you read this file carefully since it contains important, last-minute changes and information about the product.


3.3.4 Starting the softwareBefore using the Network Dispatcher GUI, you must start the Network Dispatcher server by running the following command:

# ndserver

If you want to stop it, run the following command:

# ndserver stop

Use the ndadmin command to start the Network Dispatcher GUI:

# ndadmin


The Network Dispatcher GUI is shown in Figure 3-31.


3.3.5 UninstallationThe recommended procedure to uninstall the software is to use the same setup script you used to install it. When uninstalling, you need to use the -nduninstall option, as shown below:

Example 3-8 Starting the uninstallation# mount /cdrom# cd /mnt/cdrom/# ./setup -nduninstallRunning ND Uninstall...Details on the uninstall are located in the log file: /opt/nd/uninstall.logND Uninstall complete.

All the steps performed in the uninstallation processes are recorded in the log file /opt/nd/uninstall.log.


The uninstall process does not completely remove the software. If you want to completely remove the software, you must delete the installation directory (in Linux it is /opt/nd). Do not remove this directory if you plan to install Network Dispatcher on this machine again, since it keeps the configuration files even after you uninstall the software.


Chapter 4. Network Dispatcher basic configuration

In this chapter, we explain the implementation of basic scenarios using Network Dispatcher.

First, we describe a basic scenario using Network Dispatcher to load balance two WebSphere Application Servers.

Then, we improve this scenario by adding one more Network Dispatcher machine for high availability.

Finally, we add Interactive Session Support (ISS) to the scenario to improve the load balancing of the Web servers.

4


4.1 Load balancingIn this section, we will describe a simple load balancing scenario. Later on, we will add some extra functions to this same scenario.

4.1.1 Scenario descriptionWe created a simple scenario, involving one Network Dispatcher load balancing the requests to two Web servers. All requests sent to the cluster address go to the Network Dispatcher machine, which will forward these requests to the Web servers.

In this scenario, we installed Network Dispatcher on a Windows NT machine, and WebSphere Application Server on two other machines, one running Windows NT and the other running AIX, as shown in Figure 4-1.

Figure 4-1 Basic load balancing scenario

9.24.105.39m23kk904

9.24.105.59rs60008

Web servers

9.24.105.82m238p4yl

Network DispatcherCluster:9.24.105.87

wses1


The following table shows more information on the machines we used in this scenario.

Table 4-1 Machines used in the load balancing scenario

These machines were all connected to the same Ethernet network.

4.1.2 ConfigurationFirst, we explain how to use the GUI. Click Start->Programs->IBM Network Dispatcher.

If you are using Network Dispatcher on a UNIX machine (AIX, Linux or Solaris), run the ndadmin command to start it:

# ndadmin

During the installation, we chose to install all the administration components (Dispatcher, Interactive Session Support and Content Based Routing), so the GUI will show all of them under the group Network Dispatcher (for more information on installing those components, refer to Chapter 3, “Network Dispatcher installation” on page 27).

If you click any item under Network Dispatcher, it will insert a menu option on the menu bar. This new option will be placed beside the File menu option (see Figure 4-2). You can also access this context menu by selecting the item (in this case, Dispatcher) and right-clicking it.

Host Name IP Address Operating System and Software

Service

m238p4yl 9.24.105.82 Windows NT 4.0Service Pack 5WebSphere Edge Server 1.0.3

Network Dispatcher

rs60008 9.24.105.59 AIX 4.3.3WebSphere Application Server V3.5 with PTF3

Web server

m23kk904 9.24.105.39 Windows NT 4.0WebSphere Application Server V3.5 with PTF3

Web server

Chapter 4. Network Dispatcher basic configuration 71


Now we can make a connection to our local Dispatcher server. If you are running the GUI in a separate machine, refer to 5.1, “Remote configuration” on page 156.

We will describe the steps to perform this basic configuration using both the wizard and the GUI. At the end of this section, we will show the configuration file that was produced. If you prefer to use the command line interface, you can use the commands generated by this configuration, shown in “Basic configuration using the command line interface” on page 89.

Basic configuration using the configuration wizardOpen the GUI as explained above. Click Dispatcher (under Network Dispatcher), and the context menu will appear on the menu bar. Click Dispatcher on the menu bar and it will show you two menu items: Connect to Host... and Start Wizard, as shown in Figure 4-3.


Figure 4-3 Dispatcher menu

Click Start Configuration Wizard, and the Welcome window will be displayed. Click Next, and a window explaining the purpose of the configuration wizard will be displayed. Click Next again.


The next window will begin prompting for information on your environment. First, you must type in the host name (or IP address) of the machine where the Dispatcher is running (see Figure 4-4).

Figure 4-4 Entering the host name of the Dispatcher

In this scenario, the Dispatcher machine we are using is m238p4yl.itso.ral.ibm.com (IP address 9.24.105.82). Click Update Configuration & Continue to proceed.

Note: We strongly recommend that you use a DNS server. Make sure it is resolving forward and reverse lookups correctly. In this chapter’s examples, we often use the host name and domain, which were previously added to the DNS data files.


Next, you must add the host name which resolves to the IP address of your cluster (see Figure 4-5).

Figure 4-5 Adding the host name of the cluster

We added the host name and domain of our cluster, which is wses1.itso.ral.ibm.com (IP address 9.24.105.87). Click Update Configuration & Continue; this will display a window confirming the creation of the new cluster.


Next, you need to add a port to the cluster. Since we are creating a cluster to load balance HTTP traffic, we selected port 80, as shown in Figure 4-6.

Figure 4-6 Selecting a port

Again, we received a confirmation that the port was successfully added to the cluster.


In the next window, add all server host names that are part of this cluster, that is, all Web servers that would respond to HTTP requests sent to the host name of the cluster (see Figure 4-7).

Figure 4-7 Creating a list of servers for a cluster


To add the servers, click the Add a server button, and type in the host name and domain of the first server. We added the host name m23kk904.itso.ral.ibm.com.

Figure 4-8 Adding a server to the cluster

Click Next, and the window shown in Figure 4-7 will be updated with this new server. Click Add a server again to add the next server, and continue until you have finished adding all your servers to this list.


When we finished adding our two servers, both were listed as shown in Figure 4-9.

Figure 4-9 Complete list of servers for the new cluster

Once the list is complete, click Update Configuration & Continue to proceed with the configuration.


Next, we chose to start an advisor for the port we added: port 80 (see Figure 4-10).

Figure 4-10 Starting an advisor

A window is displayed to confirm that the advisor was started.

The next window has information about setting up the servers that are part of the cluster to listen to requests sent to them using the cluster IP address. The servers need to recognize this cluster IP address as one of their valid local IP addresses in order to process the packet; otherwise, they will just forward the packet back to the Dispatcher machine (we cover this configuration in “Configuration of the back-end servers” on page 95).

Once you have read the information about the operating system you are using, click Exit to finish the configuration.


A window will be displayed confirming the end of the configuration for the cluster. If you want to add another cluster, click New Cluster (see Figure 4-11). If you do not have any more clusters to add, click Exit.

Figure 4-11 Finishing the cluster configuration

Go to “Configuring the Dispatcher and the back-end servers” on page 90 for the next steps in the configuration of our basic scenario.

Basic configuration using the GUIOpen the window as described in 4.1.2, “Configuration” on page 71, expand the group Network Dispatcher and click Dispatcher.

On the menu bar, click Dispatcher->Connect to Host..., as shown in Figure 4-3 on page 73. On the pop-up window, type the host name of the Dispatcher server as shown in Figure 4-12 (if you are using the GUI locally, just confirm the local address by clicking OK).


Figure 4-12 Connecting to the Dispatcher server

After connecting to the Network Dispatcher server, one item will be added in the left pane of the window, which will be called Host:<hostname.<domain> as shown on Figure 4-13.

Figure 4-13 Dispatcher host name

Note: If you receive an error while trying to connect to the Network Dispatcher server, make sure that ndserver is started.

Click the newly added item Host in the window, and a new Host item will be added to the menu bar. Select Host on the menu bar, then choose Start Executor, as shown in Figure 4-14.

Figure 4-14 Context menu for the Host item


One more item will be added to your GUI, named Executor <Dispatcher’s IP address>, as shown in Figure 4-15.

Figure 4-15 GUI showing the Executor status

Once the Executor is started, the next step is to add the cluster. In our scenario, our cluster address is 9.24.105.87 (see Figure 4-1 on page 70). To add this cluster, click Executor on the left pane of the window, which will add Executor to the menu bar. Click Executor on the menu bar, and click Add Cluster.... A pop-up window will be displayed, prompting you for information about this new cluster, as shown in Figure 4-16.

Figure 4-16 Adding a cluster

Fill in the IP address of your cluster in the Cluster address field, and the IP address of your Network Dispatcher server in the Primary host for the cluster field (we will discuss primary and backup hosts in 4.2, “High availability” on page 104).

Check the option Configure this cluster?, which will enable the Dispatcher to receive packets sent to the cluster address (this works as an IP alias of the local interface). When you select this checkbox, another pop-up window is displayed, as shown in Figure 4-17.

Figure 4-17 Configuring the cluster address

You must add the network interface that is configured on the same network as the cluster address. Also add the network mask.


If you are using Windows NT, you must specify the interface by using en for Ethernet and tr for Token Ring, plus the sequence number of the adapter, beginning with 0. For example, click Start->Settings->Control Panel->Network, and select the Adapters tab. It will show you the list of network adapters on your machine, and their sequence number. In Figure 4-18 there is only one adapter, and the sequence number is 1. This will be specified as en0. If there were a second adapter, it would be en1, and so forth. The same logic applies to Token Ring.

Figure 4-18 List of network adapters


If you are using AIX or Linux, use the ifconfig command to list all interfaces on the machine, as shown below:

Example 4-1 Output of ifconfig command# ifconfig -alo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0en0: flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,PSEG> inet 9.24.105.62 netmask 0xffffff00 broadcast 9.24.105.255

Use the interface identification that the ifconfig command output shows (for example en0, en1).

Once the cluster is added, you need to add the port that our back-end servers are listening to. Click Cluster on the left pane of the window, and on the context menu click Cluster->Add port. In our scenario, we used port 80, as shown in Figure 4-19.

Figure 4-19 Adding the port

The next step is to add the servers that are listening to port 80 and will receive the HTTP requests for that cluster. Click Port:80 on the left pane of the window, and on the context menu click Port:80->Add server. A pop-up window will be displayed prompting for the IP address of the first server. We typed in the address of our first server, which was 9.24.105.39, as shown in Figure 4-20.

Figure 4-20 Adding a new server

Make sure that the Network router address checkbox is not checked (we will discuss this option in more detail in 5.5, “Wide area Network Dispatcher support” on page 191). Click OK to add this server.

For our scenario, we repeated the same process to add the second server, the IP address of which was 9.24.105.59.

Once you have finished including the servers, you need to start the Manager. Click Host on the left pane of the window (it is right below Dispatcher, at the top of the tree). In the context menu, click Host->Start Manager. A pop-up window will be displayed, as shown in Figure 4-21. Click OK to confirm the default values.

Figure 4-21 Starting the manager

Right after the Manager is started, another pop-up window will automatically be displayed, allowing you to start an advisor. Since we are working with HTTP on port 80 in our back-end servers, we selected the advisor Http (in the Advisor name field) and port 80 (in the Port number field), as shown in Figure 4-22.


Figure 4-22 Adding an advisor

You do not need to add the file name of the log file. The default file name is Http_80.log.

Go to “Configuring the Dispatcher and the back-end servers” on page 90 for the next steps in the configuration of our basic scenario.

Basic configuration using the command line interfaceYou can perform the same configuration we did in the previous sections using the command line interface. This works for all platforms. Example 4-2 shows the sequence of commands that will create a cluster, add the port and the servers, and start the advisor for port 80.

Example 4-2 Configuration using the command line interfacendcontrol executor start

ndcontrol executor set nfa 9.24.105.82

ndcontrol cluster add 9.24.105.87 primaryhost 9.24.105.82

ndcontrol port add 9.24.105.87:80

ndcontrol server add 9.24.105.87:80:9.24.104.237ndcontrol server add 9.24.105.87:80:9.24.105.59

ndcontrol cluster configure 9.24.105.87 en0 255.255.255.0

ndcontrol manager start manager.log 10004ndcontrol manager proportions 49 49 2 0ndcontrol advisor start Http 80 Http_80.log


In the following section, we explain how to save your configuration so it will be run automatically after a system reboot.

Configuring the Dispatcher and the back-end serversOnce the cluster is configured (whether using the configuration wizard, the GUI, or the command line interface), you still need to go through a few more steps:

Checking the IP aliases on the DispatcherDuring the configuration, you are supposed to add an IP alias for the cluster IP address to the network interface of the Network Dispatcher machine. During the configuration, we showed you how to add the IP alias through the Dispatcher (see Figure 4-16 and Figure 4-17 on page 85).

You may want to check the status of the IP alias in your configuration. You can do that using the GUI or command line.

If you want to use the GUI, click Cluster:9.24.105.87, then click the Current Statistics tab in the right pane of the window. Locate the Current alias on NIC field. If it shows “configured”, then the IP alias was added; if it shows “unconfigured”, then the IP alias was not added or it was removed. See Figure 4-23.

Important: The non-forwarding address (NFA) used in the second command in Example 4-2 should be the address that the machine is using to listen to network requests (the local address of the machine, not the cluster IP address).


Figure 4-23 Checking the status of the cluster alias

You can also use the command line to check the IP aliases. If you are using a UNIX system, use the ifconfig command, as shown in Example 4-3. When using the ifconfig command, check the proper command syntax for your operating system.

Example 4-3 Checking for IP aliases in AIX# ifconfig -alo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0en0: flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,PSEG> inet 9.24.105.62 netmask 0xffffff00 broadcast 9.24.105.255 inet 9.24.105.87 netmask 0xffffff00 broadcast 9.24.105.255

If you are using a Windows system, use the ndconfig command, as shown in Example 4-4. Note that the ndconfig command does not show all IP addresses configured on the machine; it only shows the ones that were configured by the Dispatcher on a specified network interface.

Example 4-4 Checking for IP aliases in Windows NTC:\>ndconfig en0en0: flags=00000063<UP,BROADCAST,NOTRAILERS,RUNNING> inet 9.24.105.87 netmask 0xFFFFFF00 broadcast 9.24.105.255

Patching the Linux kernelA requirement of using Linux as a back-end server (and in collocation scenarios) is that the Linux kernel must not ARP on the loopback device.

There is a patch available at the developerWorks Web site mentioned later in this section, but you still must identify whether or not you need to install this patch on your Linux system.

Log in to your Linux system, then check if the following file exists:

/proc/sys/net/ipv4/conf/lo/hidden

� If it exists, you do not need to install the patch, but you do need to run the following commands:

echo 1 > /proc/sys/net/ipv4/conf/lo/hiddenecho 1 > /proc/sys/net/ipv4/conf/all/hidden

These commands will prevent the machine from ARPing on the loopback device. This command must be run after each reboot. You can add it to /etc/rc.d/rc.local so that it will be automatically run at boot time.

� If the file does not exist, you need to install the patch. You can get it from the following URL:

http://www.ibm.com/developerworks/linux/

Click the following links on the page: Projects and Patches->Patches->IBM Network Dispatcher->Network Dispatcher/patch arp 2.2.12-13->download.

For instructions on installing this patch, refer to IBM Network Dispatcher User’s Guide Version 3.0 for Multiplatforms, GC31-8496-05, and refer to the product readme file as well.

After you install the patch, you also need to run the following command:

echo 1 > /proc/sys/net/ipv4/conf/lo/arp_invisible

Note that the path is the same as the built-in version on later kernels, but the file name is different. You need to run this command after each reboot, as we mentioned before.

Setting the manager proportionsThe weight of each server is calculated by the Manager using four values:

1. number of active connections

2. number of new connections

3. port specific (provided by the advisors, which inform the load on each server and which servers are up)

4. system metrics (provided by ISS)

You can change the proportions used by the Manager to calculate the weight. The default values are 49 (active connections), 49 (new connections), 2 (port specific) and 0 (system metrics). If you configure the ISS (refer to 4.3, “Using Interactive Session Support” on page 127 for more information), the Manager automatically sets the proportions to 48, 48, 2, and 2.

To change the default proportions, you can use the GUI. Click Manager in the left pane of the window, and in the right pane click the Configuration Settings tab, as shown in Figure 4-24.


Figure 4-24 Setting the Manager proportions

On this tab, the first fields are the proportion fields. If you want to set 40% for active connections, 40% for new connections, 20% for the advisor and 0% for ISS (we are not using these values in this scenario), type in the values as shown in Figure 4-24.

Click Update Configuration at the bottom of the window to activate the changes.

If you are using the command line interface, you can run the following command to apply these proportions, instead of using the GUI:

# ndcontrol manager proportions 40 40 20 0

We recommend that you use the default values of 49, 49, 2, and 0 if you are using any advisors.


Saving the configurationThe first thing you need to do after the configuration is ready is to save it. Click Host:m238p4yl.itso.ral.ibm.com (locate the corresponding item on the left pane of the GUI window). In the context menu, click Save configuration file as... and add the file name. We used default.cfg.

Starting Network Dispatcher after a rebootIf you are running Network Dispatcher in Windows NT, the service IBM Network Dispatcher is configured to be automatically started after each reboot, so you can proceed to “Configuration of the back-end servers” on page 95.

If you are running AIX, Linux or Solaris, you must configure the system to run the ndserver command after each reboot.

In our AIX environment, we enabled the automatic startup of ndserver. We added it to the inittab by running the following command:

# mkitab “nd:2:wait:/usr/bin/ndserver > /dev/console 2>&1”

Configuration of the back-end serversAlthough the Network Dispatcher server is ready to begin load balancing the traffic, there is still one more configuration step to perform: preparing the back-end servers to accept packets sent to the cluster IP address.

We do this by adding an IP alias to the loopback interface of these servers. This alias is the IP address of the cluster.

Important: If you choose to save your configuration in a file called default.cfg, Network Dispatcher will automatically read it every time ndserver is started.

Important: The cluster address must be added to the back-end servers as a non-advertising address (the server must not respond to ARP requests for that address). That is why we use it as an alias to the loopback interface.


We describe below how we added this alias to our Windows NT and AIX machines:

� Windows NT

First, you must install the MS Loopback Adapter (if you have not already done so).

Click Start->Settings->Control Panel->Network. In the Network window, select the Adapters tab, as shown in Figure 4-25.

Figure 4-25 List of adapters


Click the Add... button, and select the MS Loopback Adapter, as shown in Figure 4-26.

Figure 4-26 Adding the MS Loopback Adapter

Click OK; another pop-up window will be displayed, prompting for the frame type you are using. The options are:

– 802.3 (Ethernet)

– 802.5 (Token Ring)

– FDDI

Figure 4-27 Frame type

Click OK; on the next pop-up window, you need to type in the path to the Windows NT CD-ROM.


Once the system finishes copying the files, the Network window will display the newly added MS Loopback Adapter. The process is not yet completed: click Close, and the system will install the drivers it needs. After that, it will prompt you to add the address you want to put in the loopback adapter. Type in the IP address of the cluster, as shown in Figure 4-28.

Figure 4-28 Applying an IP address to the loopback adapter

Click OK. You will need to reboot the machine.


After rebooting, check the routing table. Compare it to the one in Example 4-5.

Example 4-5 Routing table after adding the loopback adapterC:\>route print===========================================================================Interface List0x1 ........................... MS TCP Loopback interface0x2 ...00 06 29 b0 94 63 ...... Intel 8255x-based Integrated Fast Ethernet0x3 ...20 4c 4f 4f 50 20 ...... MS LoopBack Driver======================================================================================================================================================Active Routes:Network Destination Netmask Gateway Interface Metric 0.0.0.0 0.0.0.0 9.24.105.1 9.24.105.39 1 9.24.105.0 255.255.255.0 9.24.105.39 9.24.105.39 1 9.24.105.0 255.255.255.0 9.24.105.87 9.24.105.87 1 9.24.105.82 255.255.255.255 127.0.0.1 127.0.0.1 1 9.24.105.87 255.255.255.255 127.0.0.1 127.0.0.1 1 9.255.255.255 255.255.255.255 9.24.105.39 9.24.105.39 1 127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1 224.0.0.0 224.0.0.0 9.24.105.39 9.24.105.39 1 224.0.0.0 224.0.0.0 9.24.105.87 9.24.105.87 1 255.255.255.255 255.255.255.255 9.24.105.39 9.24.105.39 1===========================================================================

Note that after the loopback adapter was added, the system also added one more route to the routing table. Now, there are two routes to the same destination (the local network 9.24.105.0) using two different gateways: first, the Ethernet adapter IP address (9.24.105.39) and second, the cluster IP address that was added to the loopback (9.24.105.87).

The second route is incorrect. Since the first one is correct, the second one should not cause your system any trouble, but we recommend that you remove the second route anyway. You can use the following command in a command prompt window:

C:\> route delete 9.24.105.0 9.24.105.87

This command must also be run after a reboot. We created a batch file, C:\routedel.bat, and added the following lines to it:

@echo offroute delete 9.24.105.0 9.24.105.87exit


Then we added a shortcut between this batch file and the Startup program group, as shown in Figure 4-29.

Figure 4-29 Shortcut to the batch file

This batch file will be run after a reboot and it will delete that second route. If you need to add more aliases to the loopback, add the route delete for each alias to this same batch file.

� AIX

The only thing you need to do in AIX in order to add the IP alias is to run the ifconfig command, as follows:

# ifconfig lo0 alias 9.24.105.87 netmask 255.255.255.0

Add this command to the end of the /etc/rc.net file so it will be run every time the networking configuration is run (for example, after a system reboot).

Important: Make sure you use the netmask option with the correct netmask of your network. If you use a wrong netmask, or do not use the option at all, a new route will be added to your routing table using the loopback as gateway, causing you to experience several routing problems.


If you want to confirm that the alias was added to the loopback adapter, run the following command:

# ifconfig lo0

You will see output similar to that in Example 4-6.

Example 4-6 Loopback configurationlo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0 inet 9.24.105.87 netmask 0xffffff00 broadcast 9.24.105.255

You can test to see if your server is answering requests sent to the IP address of the cluster by using a browser on the local machine and requesting the IP address of the cluster.

The following table shows which command you need to use to add an alias on different platforms.

Table 4-2 Adding an IP alias

After adding the alias, check for the extra route that may have been created, and remove it according to the correct procedures for each operating system.

Some operating systems do not allow adding an alias to the loopback adapter. One solution is to change the loopback IP address, but you will not be able to use this server on more than one cluster.

Platform Command

AIX ifconfig lo0 alias <cluster IP address> netmask <netmask>

HP-UX ifconfig lo0 <cluster IP address>

Linux ifconfig lo:1 <cluster IP address> netmask 0.0.0.0 upFor the second alias, use lo:2, and so forth.

OS/2 ifconfig lo <cluster IP address>

Solaris ifconfig lo0:1 <cluster IP address> 127.0.0.1 up

Important: If you test the access with a browser and do not get a response, but other services are answering to the cluster IP address (FTP, for example), you may need to configure your HTTP server to listen to the IP address of cluster (refer to the documentation of your HTTP server for more information on how to do this).

4.1.3 TestsThe first thing we did for our scenario was to check if the advisor was getting responses from the back-end servers, and if these were active. We used the command ndcontrol manager report. The output is shown in Example 4-7.

Example 4-7 ndcontrol manager report outputC:\>ndcontrol manager report

----------------------------------| HOST TABLE LIST | STATUS |----------------------------------| 9.24.105.39| ACTIVE|| 9.24.105.59| ACTIVE|--------------------------------------------------------------------------------------------------------------------------------------| 9.24.105.87| WEIGHT | ACTIVE % 40 | NEW % 40 | PORT % 20 | SYSTEM % 0 |----------------------------------------------------------------------------------------------------| PORT: 80 | NOW | NEW | WT | CONNECT | WT | CONNECT | WT | LOAD | WT | LOAD |----------------------------------------------------------------------------------------------------| 9.24.105.39| 8 | 8 | 10 | 0 | 9 | 1 | 5| 531| 0| 0|| 9.24.105.59| 10 | 10 | 10 | 0 | 10 | 1 | 14| 110| 0| 0|----------------------------------------------------------------------------------------------------| PORT TOTALS: | 18 | 18 | | 0 | | 2 | | 641 | | 0 |----------------------------------------------------------------------------------------------------

--------------------------------| ADVISOR | PORT | TIMEOUT |--------------------------------| http | 80 | unlimited |--------------------------------

The first table (HOST TABLE LIST) shows the list of back-end servers and their status. In this scenario, we have two back-end servers (9.24.105.39 and 9.24.105.59) and they are both active.

The next table shows a list of servers per port. In this case, we configured port 80 which balances to two servers. This table also shows the weight, number of active and new connections, the load value informed by the advisor and by ISS. This information is shown for each server.

The last table shows information about the advisors: the name of the advisor that was started, the port and the timeout value attributed to it.


To test the load balancing with several connections, we created a script on a Linux machine to send an HTTP request to the cluster address, using the wget command. The script is shown in Example 4-8.

Example 4-8 Script to send HTTP requests#!/bin/bashwhile truedo

wget wses1.itso.ral.ibm.comsleep 1

done

You can use different values with the sleep command to increase or decrease the delay between requests sent, or even uncomment it if you want to produce heavier traffic. Since this script will get the index.html file from the Web server and store it locally, we recommend that you run it inside a temporary directory. To interrupt this script, use CTRL+C.

For our purposes, we started the script and the output of ndcontrol manager report showed us the traffic that was generated (see Example 4-9).

Example 4-9 ndcontrol manager report output after creating several HTTP requestsC:\>ndcontrol manager report




You can also see the traffic being balanced using the server monitor on the GUI. Click Port:80 in the left pane of the window, and in the context menu click Port:80->Monitor.... The monitor window showing the traffic produced by our tests is shown in Figure 4-30.

Figure 4-30 Server monitor window showing traffic

4.2 High availabilityNetwork Dispatcher provides a built-in high availability function. It allows you to configure a backup machine for your load balancer server; if case the primary load balancer fails, this backup machine will take over the load balancing for all clusters.


The high availability configuration detects and recovers from network and server failures. Network Dispatcher is smart enough to determine that a server or a network is down. In case of a server failure, clients lose only the current connections, but they can immediately establish a new connection to the remaining servers with no problem.

The high availability environment involves two Dispatcher machines with connectivity to the same clients and to the same cluster of servers, as well as connectivity between themselves. Both Network Dispatcher servers must be using the same operating systems, and they must be connected to the same network.

The two Dispatcher machines are usually referred to as the primary machine and the backup machine.

The primary machine normally works as a Dispatcher, and is in active state while it is balancing the load among the servers of its clusters.

The backup machine, configured in a very similar way to the primary machine, stays in standby mode unless the primary machine fails.

The two machines are synchronized (which means the active server sends the connection table to the standby server) and only the primary machine routes packets, while the backup machine is continually being updated.

The two machines establish communication to monitor each other’s status, referred to as a heartbeat, using a port that you can choose. The standby (backup) machine uses the heartbeat and multiple reachability criteria to decide if a failure has occurred on the active (primary) server. If the primary machine fails, the backup machine detects this failure, switches to active state, and begins taking over the routing of packets.

When the primary machine is operational again, but in standby state, you can either decide that it automatically becomes the active machine, or leave it in standby mode. In this last case, you will have to manually change its status if you later want it to become the active machine again. This is known as auto recovery or manual recovery.

Note: You can also configure the backup machine to respond to other clusters addresses while it works as a backup for the clusters on the primary machine. See Chapter 5.6, “Mutual high availability” on page 206.


4.2.1 Scenario descriptionFor this scenario, we used the same layout as we did for load balancing (see 4.1, “Load balancing” on page 70), and we added just one more machine to act as the backup machine in the high availability environment (see Figure 4-31).

Figure 4-31 High availability scenario

The table below describes the machines we used in this scenario.

Table 4-3 Machines used in the high availability scenario

Host name IP Address Operating System and Software

Service

rs600037 9.24.105.62 AIX 4.3.3WebSphere Edge Server 1.0.3

Primary Dispatcher


Backup Dispatcher


Web server


Web server

9.24.105.39m23kk904

9.24.105.59rs60008

Web servers

Network DispatchersPort 80 cluster:9.24.105.87

wses1

9.24.105.62rs600037Primary

High availability9.24.105.60rs600010Backup


4.2.2 ConfigurationWe performed the configuration of the high availability feature of Network Dispatcher in the following order:

1. Configuration of load balancing on the primary machine, and tests

2. Configuration of high availability on the primary machine

3. Configuration of load balancing on the backup machine, and tests

4. Configuration of high availability on the backup machine

5. Creation of high availability scripts on both machines, and tests

We will now explain each of these steps in more detail.

Configuration of load balancing on the primary machineWe used the same configuration shown in 4.1, “Load balancing” on page 70. However, this time we used Network Dispatcher on an AIX machine, as shown in Figure 4-31 on page 106. We used the same back-end servers and the same cluster IP address (see Figure 4-32).


Figure 4-32 Configuration of the primary Network Dispatcher server

Configuration of high availability on the primary machineAfter we have the load balancing environment working, we need to make one change before proceeding to the high availability configuration: we need to unconfigure the cluster IP address by removing the IP alias that was added to the network interface of the machine.

Open the GUI on the machine that you selected to be the primary machine, and connect to the local server. Click Cluster in the left pane of the window, and click Cluster->Unconfigure cluster address... in the context menu. A pop-up window will be displayed, as shown in Figure 4-33. Click Yes to confirm.


Figure 4-33 Unconfiguring the cluster address

To start adding the high availability information, click Executor in the left pane of the window, and in the context menu click Executor->Add High Availability Backup.... See Figure 4-34.

Figure 4-34 Configuring high availability on the primary machine

Choose a port number that will be used by both Network Dispatcher servers to exchange the information needed to keep them synchronized, and fill it in the Port number field. You can choose any available port, you just need to make sure that this port is available on both servers. If you want to check which ports are currently in use, run the following command (in both UNIX and Windows environments):

# netstat -an


In the Role field, select the role that this machine will have in the high availability scenario (primary, backup, or both). In our scenario, this machine is our primary machine, so we selected primary.

In the Recovery strategy field, choose how your primary machine is going to behave in case the backup machine has taken over. If you select Auto, it will take over as soon as it is up (and responding to the network). If you select Manual, you will need to run a command to force it to take over. As long as you do not run the command, the backup remains active. In our scenario, we selected Auto.

The last things you need to add in this window is the heartbeat source address and the heartbeat destination address. The heartbeat is a ping packet that is sent from the local machine to the other server in the same high availability cluster to make sure it is responding.

In the heartbeat source address field, fill in the local IP address of the machine. In the heartbeat destination address field, fill in the IP address of the backup machine.

When you are finished, click OK.

Back in the GUI, the next thing we need to add is a reach target IP address. This works in a similar way to the heartbeat, but this time we use another machine as the destination for the ping packet. We recommend that you use as the reachability target a machine that is also guaranteed to be up. For example, you can use the IP address of your router (use the IP of the interface that is directly connected to the local network).

We used the IP address of our local router, which is 9.24.105.1.

To add this IP address as a reach target IP address, click Executor in the left pane of the window, then go to the context menu and click Executor->Add Reach Target. A pop-up window will be displayed, as shown in Figure 4-35.

Note: If you select Auto in the Recovery strategy field, you may have problems when you want to run maintenance procedures on the primary machine, since it will try to take over after every reboot. In this case, we recommend that you select Manual as the Recovery strategy field, or that you not allow the Network Dispatcher to start after the reboot while you are doing the maintenance procedures on the machine.


Figure 4-35 Adding a reach target on the primary machine

Fill in the IP address of the machine you selected to be the reach target, and click OK.

Note: If the Network Dispatcher machine is connected to two or more networks, we recommend that you select a reach target in each network.


After we add this information, our primary machine is ready. See Figure 4-36 for the full configuration (look at the pane on the right).

Figure 4-36 High availability configuration for the primary machine


Although the configuration is not complete (we still have to configure the backup machine), you can see the information and status of your primary machine by running the command ndcontrol high status, as shown in Example 4-10.

Example 4-10 ndcontrol high status on the primary machine# ndcontrol high status

High Availability Status:-------------------------Role ................. PrimaryRecovery strategy .... AutoState ................ ActiveSub-state ............ Not SynchronizedPrimary host ......... 9.24.105.62Port ................. 12345Preferred target ..... n/a

Heartbeat Status:-----------------Count ................ 1Source/destination ... 9.24.105.62/9.24.105.60

Reachability Status:--------------------Count ................ 1Address .............. 9.24.105.1 reachable

Note that all the information you have entered is shown, as is the status of the communication between the two servers (the Sub-state) and the status of the heartbeat and reachability. Since we have not yet configured the backup machine, the Sub-state is shown as Not Synchronized.


Configuration of load balancing on the backup machineWe will now prepare the backup machine. Follow the same steps as described in 4.1, “Load balancing” on page 70 to complete the load balancing configuration. The only difference this time is that you must not configure the cluster when you are adding the cluster IP address. See Figure 4-37.

Figure 4-37 Adding a cluster on the backup machine

You must fill in the cluster IP address in the Cluster address field. In the Primary host for the cluster field, you must enter the IP address of the primary machine. Leave the Configure this cluster? checkbox unselected, because we are going to use the high availability scripts to add the IP alias to the proper network interface.


Configuration of high availability on the backup machineAfter you have finished the load balancing configuration, you can add the high availability configuration.

In the left pane of the window, click Executor, then go to the context menu and click Executor->Add High Availability Backup.... The panel shown in Figure 4-38 will be displayed.

Figure 4-38 Configuring high availability on the backup machine

This is the same window that you see when adding the primary machine (see Figure 4-34 on page 109). In the Port number field, make sure you use the same port number that you selected on the primary machine.

In the Role field, select Backup. In the Recovery strategy field, select the same strategy you selected for the primary machine. In our scenario, we selected Auto.

For the heartbeat information required, you must provide the source (the local IP address) and the destination (the IP address of the primary machine).

Click OK to add the high availability configuration.


Next, you need to add a reach target. We recommend that you use the same machine(s) you used on the primary machine. Click Executor in the left pane of the window, then go to the context menu and click Executor->Add Reach Target. A pop-up window will be displayed, as shown in Figure 4-39.

Figure 4-39 Adding a reach target on the backup machine

Fill in the IP address of the target, and click OK.


Look at Figure 4-40 to see the complete high availability configuration on the backup machine (look at the right pane).

Figure 4-40 High availability configuration for the backup machine

As before, we can now run the ndcontrol high status command on both machines, and compare the output.


See in Example 4-11 the output of this command on the primary machine after adding the backup machine.

Example 4-11 ndcontrol high status on the primary machine# ndcontrol high status

High Availability Status:-------------------------Role ................. PrimaryRecovery strategy .... AutoState ................ ActiveSub-state ............ SynchronizedPrimary host ......... 9.24.105.62Port ................. 12345Preferred target ..... 9.24.105.60



Note that all information remains unchanged, except for the Sub-state, which now shows Synchronized.


See the output of the same command on the backup machine in Example 4-12.

Example 4-12 ndcontrol high status on the backup machine# ndcontrol high status

High Availability Status:-------------------------Role ................. BackupRecovery strategy .... AutoState ................ StandbySub-state ............ SynchronizedPrimary host ......... 9.24.105.62Port ................. 12345Preferred target ..... 9.24.105.62



The output is similar to the output for the primary machine, except for the Role (this one is Backup), the State (it is Standby now, which means that the other server is active) and the remote IP addresses, which point to the IP address of the primary machine.

As soon as you finish configuring both servers, we recommend that you make a backup of the Network Dispatcher configuration, as described in “Saving the configuration” on page 95.


Creating the high availability scriptsThe following are the four scripts that Network Dispatcher uses:

goActive: This script is run every time the Network Dispatcher server goes into active state. This script deletes loopback aliases and adds devices aliases.

goStandby: This script is run every time the Network Dispatcher server goes into standby state. This script deletes devices aliases and adds loopback aliases.

goInOp: This script is run when the Network Dispatcher executor is stopped, or when it is started for the first time. This script deletes all devices aliases and loopback aliases.

goIdle: This script is run when the Network Dispatcher server goes into idle state (when the machine is a stand-alone dispatcher, or when the high availability features have not yet been added). Do not create this script if you are using high availability.

These scripts must be placed in the <product install path>/dispatcher/bin directory.

You will find a set of examples in the <product install path>/dispatcher/samples directory. You can start building your own scripts using these examples. If you want to use the examples, copy them to the bin subdirectory, and edit them there.

Our scenario is simple: we use one cluster for load balancing two back-end Web servers. When either of the servers (primary or backup) takes over, it needs to start responding to requests sent to the cluster IP address, so it has to add an IP alias of the cluster IP address to the local network interface.

This will be done in the goActive script, on both machines (but it will run only on the machine going into active state).

Important: If you are using UNIX, you need to add the execution permission to the scripts after you create them. Make sure to run the following command:

# chmod u+x <product install path>/bin/go*

A simple goActive script is shown in Example 4-13.

Example 4-13 The goActive script#!/bin/ksh# goActive scriptINTERFACE=en0CLUSTER=9.24.105.87NETMASK=255.255.255.0ifconfig lo0 delete $CLUSTERifconfig $INTERFACE alias $CLUSTER netmask $NETMASK

The next script is goStandby, which deletes the cluster IP alias from the device and adds it to the loopback, as shown in Example 4-14.

Example 4-14 The goStandby script#!/bin/ksh# goStandby scriptINTERFACE=en0CLUSTER=9.24.105.87NETMASK=255.255.255.0ifconfig $INTERFACE delete $CLUSTERifconfig lo0 alias $CLUSTER netmask $NETMASK

The last script in our customization is goInOp, which is supposed to delete all aliases, as shown in Example 4-15.

Example 4-15 The goInOp script#!/bin/ksh# goStandby scriptINTERFACE=en0CLUSTER=9.24.105.87NETMASK=255.255.255.0ifconfig $INTERFACE delete $CLUSTERifconfig lo0 delete $CLUSTER

Once these scripts are created on both machines, you must stop the executor (using the ndcontrol executor stop command) and restart Network Dispatcher before starting the tests, so that it can run the scripts upon starting. Do it first on the primary machine; after you start the primary machine, do it on the backup machine. For more information on starting and stopping the servers, refer to Chapter 3, “Network Dispatcher installation” on page 27.

Note: The scripts are usually identical for the primary machine and the backup machine, unless there is some particular command you need to run in each machine. In our scenario, we used the exact same script on both machines.


Configuration filesIf you want to use the command line interface, here are the commands that were generated by following the configuration steps above.

Example 4-16 lists the statements for the primary machine configuration file.

Example 4-16 primary machine configuration filendcontrol executor startndcontrol executor set nfa 9.24.105.62



ndcontrol server add 9.24.105.87:80:9.24.105.39


ndcontrol manager start manager.log 10004ndcontrol manager proportions 49 49 2 0

ndcontrol advisor start Http 80 Http_80.log

ndcontrol highavailability heartbeat add 9.24.105.62 9.24.105.60ndcontrol highavailability backup add primary auto 12345ndcontrol highavailability reach add 9.24.105.1


Example 4-17 lists the statements for the backup machine configuration file.

Example 4-17 backup machine configuration filendcontrol executor startndcontrol executor set nfa 9.24.105.60







ndcontrol highavailability heartbeat add 9.24.105.60 9.24.105.62ndcontrol highavailability backup add backup manual 12345ndcontrol highavailability reach add 9.24.105.1

4.2.3 TestsFirst, review what kind of recovery strategy you selected. If you selected Auto, the only way you can test the high availability configuration is by forcing the active machine to fail. If you selected Manual, you can also use a command to force the takeover.

To check if your configuration is up and running, run the command ndcontrol high status and locate the State field, as shown in Example 4-11 on page 118 and Example 4-12 on page 119.

Tests with auto recoveryThe difference between auto and manual recovery is the behavior of the environment after a failed server has been recovered.

If it is an auto recovery and the backup machine became active because the primary machine was not responding, then as soon as the primary machine is available, it will become active and the backup machine will be in standby mode again.


If it is a manual recovery, the automatic takeover is only performed in case of failure of the active machine, or by running a command (see also “Tests with manual recovery” on page 127). So, if the primary machine fails, the backup machine becomes active. When the primary machine is back online again, it does not become active until the administrator explicitly commands it to do so, or unless the backup machine fails.

In testing, when using auto recovery you have to simulate a problem with the active server to force the takeover, since you cannot force it using any command. A simple way to do that is to disable the network interface.

In our scenario, rs600037 is the primary machine, and rs600010 is the backup machine. We checked that both machines were working properly, using the ndcontrol high status command.

Below is the output for the primary machine:

Example 4-18 Status of the primary machineHigh Availability Status:-------------------------Role ................. PrimaryRecovery strategy .... AutoState ................ ActiveSub-state ............ SynchronizedPrimary host ......... 9.24.105.62Port ................. 12345Preferred target ..... 9.24.105.60



Note that the Sub-state is synchronized, which means that both Network Dispatcher servers are currently exchanging information through the network. Also, the Reachability status shows that the router is reachable.

Note: No matter what type of recovery strategy you selected, the backup server will automatically take over in case of failure of the primary server.


Now look at the same command output for the backup machine.

Example 4-19 Status of the backup machineHigh Availability Status:-------------------------Role ................. BackupRecovery strategy .... AutoState ................ StandbySub-state ............ SynchronizedPrimary host ......... 9.24.105.62Port ................. 12345Preferred target ..... 9.24.105.62



The same information is given about the backup machine as about the primary machine.

Once both servers were up and synchronized, we stopped the Ethernet interface on the primary machine, using the command below.

# ifconfig en0 down

That forces the backup machine to take over.


A few seconds after running this command, we can see that the backup machine has taken over the primary Dispatcher functions by running the ndcontrol high status command again:

Example 4-20 Status after takeoverHigh Availability Status:-------------------------Role ................. BackupRecovery strategy .... AutoState ................ ActiveSub-state ............ Not SynchronizedPrimary host ......... 9.24.105.62Port ................. 12345Preferred target ..... n/a



Note that this machine is active, but the Sub-state is not synchronized (because it cannot communicate with the primary machine).

If you look at the IP addresses on the network interfaces of the backup machine, you will notice that this machine now has an IP alias on the Ethernet interface for the cluster IP address.

Example 4-21 IP addresses of the backup machine after takeover# ifconfig -alo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT>

inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255inet6 ::1/0

en0: flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,PSEG>

inet 9.24.105.60 netmask 0xffffff00 broadcast 9.24.105.255inet 9.24.105.87 netmask 0xffffff00 broadcast 9.24.105.255

We ran the script shown in Example 4-8 on page 103 to send HTTP requests to the cluster address while we were doing tests with high availability, to make sure that there would be no interruption on the communication between the client and the back-end Web servers.

Tests with manual recoveryIf you configured the high availability recovery strategy as manual, you can start your tests by forcing the backup machine to take over. To do so, go to the backup machine, and run the following command:

# ndcontrol high takeover

In case you decided to use the manual recovery, you can also do tests using the ifconfig <interface> down command, as described in “Tests with auto recovery” on page 123.

4.3 Using Interactive Session SupportBy using the monitoring capabilities provided by Interactive Session Support (ISS) on the back-end server machines, you can provide the Dispatcher with server load information. In this case, ISS cooperates with the Dispatcher, but ISS does not make any load balancing decision. The ISS monitor collects specific server information such as CPU usage, memory usage and disk activity from the ISS agents running on the individual servers, and forwards it to the Dispatcher. The Dispatcher uses this load information, along with other sources of information, to determine which is the least loaded server of the cluster, then performs load balancing.

Restriction: You can only run the ndcontrol high takeover command if the following three conditions are met:

1. the recovery strategy is manual

2. the machine state is Standby

3. the Sub-state is Synchronized

You can confirm this information by running the ndcontrol high status command.

4.3.1 Scenario descriptionWe used the same scenario as described in 4.1, “Load balancing” on page 70, as shown in Figure 4-41.

Figure 4-41 ISS scenario




Service


Network Dispatcher


Web server


Web server

9.24.105.39m23kk904

9.24.105.59rs60008

Web servers

Network DispatcherCluster:

9.24.105.87wses1 9.24.105.82

m238p4yl

ISS

ISS ISS


4.3.2 ConfigurationIn this section, we discuss all the steps necessary to configure ISS on the back-end servers and to configure them to send the information to the ISS monitor.

InstallationAs already mentioned, Network Dispatcher has three subcomponents: Dispatcher, ISS and CBR. In this section, we discuss the installation of the ISS component.

ISS is supported on three operating systems: AIX, Windows NT and 2000, and Solaris. Refer to the appropriate platform section of Chapter 3, “Network Dispatcher installation” on page 27 for details on how to install ISS on AIX and Windows NT. When you reach the point where you must choose which ND component to install, select these three components to install ISS on your back-end server:

� Interactive Session Support Runtime

� Interactive Session Support Administration

� Interactive Session Support License

Important: ISS requires the installation of Java Development Kit (or Java Runtime) V1.3.0. Make sure that there will be no conflicts with any software you have installed on your back-end servers.


Figure 4-42 shows these components selected during the installation procedure:

Figure 4-42 Installing ISS components

ISS has a special system monitoring function. If you want ISS to be part of your environment, you should select one machine to run as an ISS monitor, while an ISS agent process runs on the back-end servers in the cluster, gathering information about the status of the systems and feeding it back to the ISS monitor. The three ISS components listed above should be installed on the ISS monitor machine as well as on the back-end server machines.

On Windows NT, at the end of the ISS component installation, you will be instructed to reboot. After rebooting, notice that the IBM Interactive Session Support service is not automatically started, as shown in the Services folder of the Control Panel (see Figure 4-43).


Figure 4-43 The ISS server is not automatically started after rebooting

You should click Start to activate the ISS daemon. Refer to “Starting the ISS daemon” on page 133 for further information on this.

Scenario configurationIn this section, we show you how to create a scenario where both the Dispatcher and ISS components of Network Dispatcher are used. Here, we worked in the same environment that we described in 4.1, “Load balancing” on page 70, but we enhanced that scenario by adding the ISS function. In the environment shown in Figure 4-41 on page 128, we decided to have the Dispatcher machine run as the ISS monitor, too. ISS is used here only in its capacity to gather server load information from the back-end servers, where the ISS agent process runs. This information is then passed back to the ISS monitor process, running in this case on the same Dispatcher machine. The monitor interacts with the Dispatcher and provides it with information about the loads on the servers. The Dispatcher uses this information along with other sources to perform the load balancing.

The steps required to configure ISS in this information-gathering capacity start out essentially the same as if ISS were to perform the load balancing. The key difference between the information gathering function and the load balancing function of ISS is the choice of Observer type, as explained below.


Network environmentFigure 4-41 on page 128 shows the environment we used.

Note that as soon as you make ISS part of the load balancing process, the Manager configuration must reflect the presence of ISS. To ensure this, change the Manager proportions (see “Setting the manager proportions” on page 93) to take into account the input coming from ISS. In this case, we set the proportions of importance for the Manager to 48, 48, 2, and 2.

The following example demonstrates how easy it is to implement ISS high availability.

ISS configurationIn 4.1, “Load balancing” on page 70, we described the network environment we had set up using the Dispatcher, but without using ISS. In this section, we show how to add ISS to the already existing environment.

ISS configuration methodsIn this section, we show how to make some changes to the sample configuration file shipped with the product, then how to complete the customization with the GUI. Each customization that we perform with the GUI can also be performed on the command line with the isscontrol command. We will supply the syntax of the corresponding isscontrol command that can be used to accomplish several of the tasks for which we used GUI. For further details on the isscontrol command, please refer to IBM Network Dispatcher User’s Guide Version 3.0 for Multiplatforms, GC31-8496-05.

The Dispatcher sample configuration file is <install path>/iss/iss.cfg, where <install path> varies by operating system.

The original Dispatcher sample configuration file for the Windows NT platform, as it is shipped with the product, is shown in Example 4-22.

Example 4-22 ISS shipped sample configuration file# This is a default configuration file which should be located in# the root ISS directory (nd\iss). Replace yourHostName with the# hostname of this machine and uncomment the line.# This is the minimal configuration needed for ISS to# run successfully.

Cell issCell local

# Node yourHostName 01

The sample file in Windows NT is C:\Program Files\Ibm\nd\iss\iss.cfg.

Example 4-23 shows our customized version of the Dispatcher sample file. You do not need to rename the file; you can simply edit the sample file. In AIX, you can edit the sample file etc/iss.cfg and you will not need to rename it either.

As the comments in the Dispatcher sample file indicated, you must create this minimal configuration file before you use the ISS GUI if you do not have any preexisting configuration files. We replaced issCell with Cary, we removed the command in the Node line and we replaced yourHostName with m238p4yl.

Example 4-23 Customized Dispatcher ISS configuration file# This is a default configuration file which should be located in# the root ISS directory (nd\iss). Replace yourHostName with the# hostname of this machine and uncomment the line.# This is the minimal configuration needed for ISS to# run successfully.

Cell Cary local

Node m238p4yl 01

In order to perform the configuration steps listed below, you must be the root user on AIX and Solaris or, if the Dispatcher is installed on Windows NT, you must be a system administrator.

Starting the ISS daemonISS operates as either the monitor or the agent, depending on the contents of the configuration file or parameters entered.

To start the ISS daemon on AIX and Solaris from a command line, enter the command:

iss -g

The -g flag instructs the ISS daemon to monitor for communication from the ND GUI on port 12099. If your iss.cfg file is not located in /etc or is not called iss.cfg, then you will need to use the -c flag followed by the full path to the configuration file to let ISS know where the file is. The -l flag can be used to specify a log file.

Important: When you start using ISS, all the machines in your cell must have the same ISS configuration file.


Note that on Windows NT, ISS is a Windows NT service and is not automatically started. To start ISS as the monitor, open the Services window of the Windows NT Control Panel (see Figure 4-43 on page 131), select IBM Interactive Session Support and then click Start. In the Startup Parameters text field, you can specify the arguments for the iss command. Notice, however, that if you want to code a backslash (\), you have to type a double backslash (\\). For example, if you want to specify the configuration file C:\Myconf\iss1.cfg and the log file C:\Myconf\iss1.log, you have to type:

-c C:\\Myconf\\iss1.cfg -l C:\\Myconf\\iss1.log

Since we are using Windows NT, and it is also the machine where we are starting the ISS daemon as a monitor, we start it with the parameter -g, as shown in Figure 4-44.

Figure 4-44 Starting the ISS daemon using the parameter -g


Starting the configuration GUINow you can start the GUI. To do so, click Start->Programs->IBM WebSphere->Edge Server->IBM Network Dispatcher. If you are using a UNIX operating system, you can start the GUI by running the command ndadmin. Figure 4-45 shows the GUI window.

Figure 4-45 Network Dispatcher GUI window

The left pane displays a tree structure, with Network Dispatcher at the top level, and Dispatcher, Interactive Session Support and Content Based Routing as components if they are installed. You can select elements in the tree structure by left-clicking, and you can display pop-up menus by right-clicking. The pop-up menus for the tree elements are also accessible from the menu bar located at the top of the window. Each item in the tree is marked with a plus sign (+) or a minus sign (-). Click the plus sign (+) to expand the items within your selection and the minus sign (-) to minimize those items.


Connecting to a hostThe next step in configuring ISS is to make a connection to a host where the ISS daemon is running. In this case, we are performing this configuration on the ISS manager machine itself. If you cannot run the GUI from the same machine, refer to 5.1, “Remote configuration” on page 156 for information on how to make a remote connection through the GUI.

To make the connection, right-click Interactive Session Support in the tree structure. In the pop-up window that appears, select Connect to Host....

We selected the host name of the machine where we are performing the initial ISS configuration, the same machine where we would be running the ISS monitor and the Dispatcher (see Figure 4-46).

Figure 4-46 Selecting a host for the ISS configuration

Important: Make sure that you have started the ISS daemon with the parameter -g before connecting to it; otherwise, you will receive the following error message: The are no more keys available for connecting to a host.


After clicking OK, the GUI was refreshed to show the ISS host connection and the single Cary cell that we had defined in our iss.cfg file. Selecting Cell: Cary and clicking the Configuration Settings tab shows us the default values (see Figure 4-47).

Figure 4-47 Default cell configuration settings


Adding nodesThe next step is to add nodes to your configuration. To do this, right-click Cell: Cary and then select Add Node in the Cell.

You will be prompted for the node name and the order number for the node, as shown in Figure 4-48.

Figure 4-48 Adding a node

We configured the Dispatcher machine m238p4yl to be the primary ISS monitor in the iss.cfg configuration file. We added m23kk904, and using the default settings, it is a backup to the monitor node. We added this node as node number 2.

This means that m23kk904 will take over and become the ISS monitor, should the primary ISS monitor m238p4yl fail. These simple steps are enough to implement ISS high availability.


At this point, the GUI is updated to reflect the new node that we added (see Figure 4-49).

Figure 4-49 New node added to the Cary cell

Using the command line, we added the last node, rs60008, and set the CanMonitor to false using the following commands:

C:\> isscontrol add node rs60008 3C:\> isscontrol set node rs60008 CanMonitor false

This new node will not be able to take over in case of failure of the previous nodes, because we set the CanMonitor to false. Figure 4-50 shows the settings of the rs60008 node.


Figure 4-50 Configuration settings for the last node

When values are changed directly in the GUI window, it is important to click the Update Configuration button at the bottom of the window. This procedure writes the changes to the ISS configuration file (iss.cfg).

If the nodes you are adding have multiple network interfaces on them, then you can define these individually by right-clicking the Node entry; this opens the Node menu, which can be used to select Add Interface to Node. In our scenario, our nodes had only one network interface, so we did not need this step.


Defining resource typesThe next step is to add resources to your configuration. To do this, right-click Cell: Cary and then select Add ResourceType in the Cell.

In this case, we would like to add a CPU ResourceType and so we type CPU in the Add ResourceType window, as shown in Figure 4-51:

Figure 4-51 Adding a CPU resource type


ResourceType entries are given default values (shown in Figure 4-52), which in this case are appropriate for the CPU ResourceType.

Figure 4-52 Default values for a CPU resource

We then modified the CPU ResourceType Recover limit (the level down to which CPU utilization should be taken before the server is considered for ranking) and Fail limit (the level of CPU utilization beyond which ISS removes the server from ranking) by entering the new values 80 and 95, respectively, directly into the GUI fields. After this, we clicked the Update Configuration button to update the ISS configuration file (iss.cfg).


Defining servicesThe next step is to add services to our configuration. In general, a service is a group of servers measured by the same ResourceTypes (refer to “Defining resource types” on page 141). A node can be a member of a single service or of many services.

For our configuration, we right-clicked Cell: Cary and then selected Add Service in the Cell. In the Add Service window, we gave the service a name, wwwservice, and also supplied a service DNS name. The Service DNS name is not needed when you are using ISS to update the Dispatcher as in this case. However, you must fill in the blank with a placeholder name; for this reason, we typed in PlaceHolder, as shown in Figure 4-53.

Figure 4-53 Adding a service

After adding the service name, we selected Add ResourceType to Service from the Service menu. In the Add ResourceType to Service window, we selected the resource type CPU that we had defined in “Defining resource types” on page 141 (see Figure 4-54).

Figure 4-54 Adding a resource type to a service


After this, we selected Add Node Interface to Service from the Service menu; we were then presented with the Add Node Interface to Service window, showing the list of nodes we had added (see “Adding nodes” on page 138): see Figure 4-55.

Figure 4-55 Adding a node to the service selection list

We were able to select only one node at a time from this window. We added rs60008 through the GUI and added the other node to our wwwservice from the command line as follows:

C:\> isscontrol add node m23kk904 to service wwwservice


The Interface List section of the Service information for the wwwservice service was updated to reflect the nodes we had added, as shown in Figure 4-56.

Figure 4-56 Service statistics showing resource type and nodes


Defining observersThe next step is to add observers to the configuration. An observer is a load balancer or any other network process that uses information about the status of nodes in the cell. ISS supports three types of observers that can subscribe to reports on a cell and/or perform load balancing for that cell.

For our configuration, we right-clicked Cell: Cary and then selected Add Observer in the cell menu. The Add Observer window contains three radio buttons at the top to select which type of Observer you are defining. We selected Dispatcher. ISS will work in conjunction with the Dispatcher to perform the load balancing function. ISS does not make any load balancing decisions itself. The ISS monitor collects server load information from the ISS agents running on the individual servers and forwards it to the Dispatcher.

When we selected Dispatcher, the default port number changed to 10004. This is the port that is used to send the metric load information to the Dispatcher. In the selection list at the bottom of the Add Observer window, we selected the host where the Dispatcher server was running (m238p4yl). This host is the machine where the Dispatcher is running and to which ISS provided load statistics (see Figure 4-57).

Figure 4-57 Adding an Observer

After we clicked OK, we went back to the GUI and selected the new Observer, Observer: m238p4yl. The settings for this Observer were immediately displayed. Next, we added a service to this Observer by right-clicking Observer: m238p4yl in the tree and selecting Add Service to Observer.


A list of the defined Services was displayed in the Add Service to Observer window, and in this case there was only one: wwwservice. We selected it and clicked OK (see Figure 4-58).

Figure 4-58 Adding a service to the Observer window


As a result of this, the Service List section for our m238p4yl Observer was updated to reflect the service name we had just added to our Observer (see Figure 4-59).

Figure 4-59 Updated Observer information showing the Service name


At this point, we examined our ISS configuration file, /etc/iss.cfg, and saw that it had been modified to include the ISS keywords required to create the ISS environment that we had configured through the GUI (see Example 4-24).

Example 4-24 Configuration file iss.cfg# ISS Configuration file# Automatically generated at 2001/04/02 17:33:39

Cell Cary LOCALPortNumber 7139LogLevel INFOHeartbeatInterval 10HeartbeatsPerUpdate 2HeartbeatsToNetFail 4HeartbeatsToNodeFail 6

Node m238p4yl 1Node m23kk904 2Node rs60008 3NotMonitor

ResourceType CPUMetric INTERNAL CPULoadPolicy MINMetricWeight 1.0MetricNormalization 0.0 100.0MetricLimits 80.0 95.0

Service wwwservice PlaceHolder 0SelectionMethod BESTResourceList CPUNodeList rs60008 m23kk904

Dispatcher m238p4yl 10004ServiceList wwwservice

Once the configuration of our environment was complete, we placed a copy of this configuration file on the two other nodes, then started the ISS daemon on them as well. On both Web server nodes, the ISS daemon started in its capacity as an agent as a result of the format of its own node entry in the configuration file.

Managing ISSIn this section, we explain how to stop and start both ISS and its configuration GUI. See “Starting the ISS daemon” on page 133 for details on how to start ISS, and “Starting the configuration GUI” on page 135 for instructions on how to start the configuration GUI.


In the following section, we see how ISS can be controlled and reconfigured during run time.

Dynamically reconfiguring ISSYou can control and reconfigure ISS while it is running by using the isscontrol command or the GUI.

In our scenario, Web server node m23kk904 was configured to be capable of taking over the function of ISS monitor if required. Web server node rs60008 was configured to not take over the function of ISS monitor. Changing the ISS configuration was done in two steps:

1. We changed the designation of Web server node rs60008 so that it could be selected as an ISS monitor.

2. We checked to see whether this change was reflected in the configuration file of one of our other nodes, for example m23kk904.

Prior to the change being made, we used FTP to propagate the iss.cfg file to each of the nodes in our cell. We then started ISS on each of these nodes.

Next, we verified (both in the ISS configuration file and via the GUI) that m23kk904 had been designated as a potential monitor. We did this verification on both m238p4yl and m23kk904. Figure 4-60 shows the m23kk904 node status as it appeared on m238p4yl.


Figure 4-60 m23kk904 node status


We then used the GUI on m238p4yl to change the status of rs60008, by selecting Can Monitor on the Monitor pull-down menu (see Figure 4-61).

Figure 4-61 Changing the Monitor value for server rs60008

We then clicked the Update Configuration button at the bottom of the GUI. We also confirmed that the ISS configuration files were updated on all of the machines by looking at the C:\Program Files\Ibm\nd\iss\iss.cfg file on both m238p4yl (where the change was made) and m23kk904, and looking at the /etc/iss.cfg file on rs60008 (one of the nodes participating in the cell).

We confirmed by the timestamp on the ISS configuration file on m23kk904 that the change had been automatically propagated from m238p4yl.


Saving the host connectionsThe GUI host connections can be saved by selecting File on the menu bar of the configuration window and then clicking Save Host Connections As.... In the pop-up window that appears, enter the name of the configuration file where you want to save the Host Connection information (see Figure 4-62). This file is saved in the directory <install path>/admin/configurations, where <install path> varies by operating system.

Figure 4-62 Saving the host connection

As a result of saving this information, on subsequent invocations of the GUI this host connection will be listed as one of the choices in the Connect to Host selection window.

Disconnecting from the hostTo disconnect from the host, right-click the Host item, either in the tree or on the menu bar, then select Disconnect from Host.

Terminating the GUITo exit the GUI, select File->Exit. Your ISS configuration file should automatically be updated at this time.

Stopping ISSOn Windows NT, as a system administrator, from the Services folder of the Control Panel, select IBM Interactive Session Support, then click Stop.

On AIX and Solaris, as root, enter:

# isscontrol shutdown <nodename>

Chapter 5. Advanced features

This chapter provides details and scenarios describing the advanced features of Network Dispatcher.

5


5.1 Remote configurationSituations may arise where you cannot run the Graphical User Interface (GUI) on the same machine where you installed the Dispatcher, ISS or CBR components. You can still use the GUI remotely by installing it on a client machine which will communicate to Network Dispatcher through Java Remote Method Invocation (RMI) calls.

In this case, you will need to create a set of keys (public and private keys) for authentication.

Using the scenario shown in Figure 5-1, we set up a remote configuration client on the m238p4yl machine, and we connected to the Dispatcher component on the rs600037 machine.

Figure 5-1 Remote configuration scenario

9.24.105.39m23kk904

9.24.105.59rs60008

Web servers

NetworkDispatcher

Port 80 cluster:9.24.105.87

wses1

9.24.105.62rs600037

9.24.105.82m238p4yl


The following table shows more information on the machines we used.

Table 5-1 Machines used in the remote configuration scenario


First, we installed the administration components (see 3.1.2, “Choosing the components” on page 29) on the Windows NT machine, which will work as a configuration client.

Since this communication is authenticated, we need to create the public and private key on the Network Dispatcher machine. You can use the following commands, according to which components you need to manage remotely:

� ndkeys: creates keys for the Dispatcher component

� cbrkeys: creates keys for the Content Based Routing component

� isskeys: creates keys for the Interactive Session Support component

These commands must be run on the machine on which you installed the components you want to configure (Dispatcher, CBR or ISS).

In our scenario, we run the following command on the machine rs600037:

Example 5-1 Creating keys# ndkeys createKey files have been created successfully

This command creates a pair of keys: the public key is created in the key directory, and the private key is created in the admin directory. Note that each component has its own key subdirectory, and inside the admin directory you will find separate subdirectories for each component.


Service

m238p4yl 9.24.105.82 Windows NT 4.0Service Pack 5

Configuration client

rs600037 9.24.105.62 AIX 4.3.3WebSphere Edge Server V1.0.3

Network Dispatcher


Web server


Web server

Chapter 5. Advanced features 157

The public key file is authorization.key. The private key file is named with the IP address of the server and the port number on which RMI is listening.

In our scenario, the following files were created. Note that our server is running on AIX.

public key: /usr/lpp/nd/dispatcher/key/authorization.key

private key: /usr/lpp/nd/admin/keys/dispatcher/9.24.105.62-10099.key

After you locate the files, you need to copy the private key file to the remote configuration client machine, and place it under the directory <install path>/admin/keys/dispatcher. Since we are using a Windows NT machine as our client, we copied the 9.24.105.62-10099.key file to the C:\Program Files\Ibm\nd\admin\keys\dispatcher directory.

After the file is copied, you just need to start the GUI and connect to the remote server, as described in 4.1.2, “Configuration” on page 71.

If you want to authorize other machines to be used as remote configuration clients, you only need to copy the same key file to each one of them as described above. If you create a new pair of keys, the machines will not be able to access this server anymore unless you copy the new private key to them.

You can also delete the keys on the server, using the delete parameter with the ndkeys, isskeys and cbrkeys commands. If you delete the keys, the remote clients will no longer be able to access this server.

5.2 CollocationWhen you do not have a dedicated machine to run Network Dispatcher, and you install the software on one of the back-end servers, this is called collocation. In this case, the machine where Network Dispatcher is installed is also one of the balanced servers.

This is useful for small environments, where you do not want to use one machine exclusively for Network Dispatcher.

Collocation is supported on AIX, Red Hat Linux and Solaris. It is not supported on Windows NT and Windows 2000. In Linux platform, it is only supported in the absence of high availability.

Important: Make sure you have the patch for the Linux loopback device as described in “Patching the Linux kernel” on page 92.

5.2.1 Scenario descriptionWe created a simple scenario to illustrate this feature. We installed Network Dispatcher on a Linux machine, which also has a HTTP server running on it; it balances the load to itself and two other HTTP servers (see Figure 5-2).

Figure 5-2 Collocation scenario

In this scenario, all requests sent to the wses3.itso.ral.ibm.com cluster address will be received by the Network Dispatcher server (rhlinux1) which will load balance between the three HTTP Servers: rhlinux1, m23kk904 and rs60008.

9.24.105.39m23kk904

9.24.105.59rs60008

Web servers

9.24.105.63rhlinux1

NetworkDispatcher

Cluster:9.24.105.89

wses3


Table 5-2 shows more information on the machines we used in this scenario.

Table 5-2 Machines used in the collocation scenario


5.2.2 ConfigurationYou can follow the steps in Chapter 4.1, “Load balancing” on page 70 to create a basic load balancing scenario without using collocation (we will add the collocation option after this). In this scenario, we are using the wses3 cluster (IP address 9.24.105.89). In this first step, we added two Web servers: m23kk904 (IP address 9.24.105.39) and rs60008 (IP address 9.24.105.59) to port 80 (see Figure 5-3).


Service

rhlinux1 9.24.105.63 Red Hat Linux 6.2WebSphere Edge Server V1.0.3

Dispatcher


Web server


Web server


Figure 5-3 Basic scenario without collocation

We then added a new server for the 9.24.105.89 cluster, and this new server is the local IP of the machine where we installed Network Dispatcher. We need to use the same IP address that we defined as NFA. If you are using the GUI, the system selects the NFA automatically. It is the IP address that appears associated with the Executor in the left pane of the GUI window (see Figure 5-3). In our scenario, the IP address is 9.24.105.63.


To add this server, click Port:80 in the left pane of the window, then click Port:80->Add Server... in the context menu. In the pop-up window, type the IP address of the new server, as shown in Figure 5-4.

Figure 5-4 Adding a server

After adding the server, click the IP address of the server in the left pane of the window (in our scenario, click 9.24.105.63), and click the tab Configuration Settings in the right pane, as shown in Figure 5-5.


Figure 5-5 Configuring the collocated server

Locate the Collocated field, and change it to yes. Click the Update Configuration button at the bottom of this pane.


After configuring the collocated server, make sure that an IP alias was added to the local adapter (for more information refer to “Checking the IP aliases on the Dispatcher” on page 90). Make sure that the IP alias was created with the cluster IP address, as shown in Example 5-2.

Example 5-2 IP addresses associated with the local server# ifconfig -aeth0 Link encap:Ethernet HWaddr 00:60:94:31:05:94 inet addr:9.24.105.63 Bcast:9.24.105.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1921246 errors:0 dropped:0 overruns:0 frame:0 TX packets:336399 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 Interrupt:14 Base address:0xc000

eth0:1 Link encap:Ethernet HWaddr 00:60:94:31:05:94 inet addr:9.24.105.89 Bcast:9.255.255.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Interrupt:14 Base address:0xc000

lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:3924 Metric:1 RX packets:6331 errors:0 dropped:0 overruns:0 frame:0 TX packets:6331 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0

If this IP alias does not exist, you need to configure the cluster address. Click Cluster in the left pane of the window, and click Cluster->Configure Cluster Address... in the context menu. A pop-up window will be displayed, as shown in Figure 5-6.

Figure 5-6 Configuring the cluster address


You need to type in the interface name that should be associated with the cluster (if you are not sure which name you should use, refer to “Basic configuration using the GUI” on page 81). Since we are using Linux, we used eth0 (the first Ethernet interface). We also recommend that you type in the correct netmask; do not leave the Netmask field blank. Typing in the wrong netmask or leaving the field blank will cause the system to create a route in the routing table, which may cause problems in your environment.

Click OK, and check the ifconfig -a command again to make sure that the IP alias was added successfully.

5.2.3 TestsFirst, we can take a look at the output of ndcontrol manager report, to make sure that all servers are active, and the advisor is receiving responses from all of them (see Example 5-3).

Example 5-3 ndcontrol manager report output

----------------------------------| HOST TABLE LIST | STATUS |----------------------------------| 9.24.105.39| ACTIVE|| 9.24.105.59| ACTIVE|| 9.24.105.63| ACTIVE|--------------------------------------------------------------------------------------------------------------------------------------| 9.24.105.89| WEIGHT | ACTIVE % 49 | NEW % 49 | PORT % 2 | SYSTEM % 0 |----------------------------------------------------------------------------------------------------| PORT: 80 | NOW | NEW | WT | CONNECT | WT | CONNECT | WT | LOAD | WT | LOAD |----------------------------------------------------------------------------------------------------| 9.24.105.39| 9 | 9 | 10 | 0 | 10 | 0 | 3| 304| 0| 0|| 9.24.105.63| 9 | 10 | 10 | 0 | 10 | 0 | 10| 156| 0| 0|| 9.24.105.59| 10 | 10 | 10 | 0 | 10 | 0 | 15| 29| 0| 0|----------------------------------------------------------------------------------------------------| PORT TOTALS: | 28 | 29 | | 0 | | 0 | | 489 | | 0 |----------------------------------------------------------------------------------------------------



We used the same script we mentioned in 4.1.3, “Tests” on page 102. This time, we used it to send HTTP requests to wses3 (see Example 5-4).

Example 5-4 New script to generate HTTP requests#!/bin/bashwhile truedo

wget wses3.itso.ral.ibm.comsleep 1

done

After starting this script, we can see through ndcontrol manager report that the load is being balanced among all three servers, as shown in Example 5-5.

Example 5-5 ndcontrol manager report----------------------------------| HOST TABLE LIST | STATUS |----------------------------------| 9.24.105.39| ACTIVE|| 9.24.105.59| ACTIVE|| 9.24.105.63| ACTIVE|--------------------------------------------------------------------------------------------------------------------------------------| 9.24.105.89| WEIGHT | ACTIVE % 49 | NEW % 49 | PORT % 2 | SYSTEM % 0 |----------------------------------------------------------------------------------------------------| PORT: 80 | NOW | NEW | WT | CONNECT | WT | CONNECT | WT | LOAD | WT | LOAD |----------------------------------------------------------------------------------------------------| 9.24.105.39| 7 | 7 | 5 | 65 | 10 | 17 | 2| 304| 0| 0|| 9.24.105.63| 15 | 15 | 20 | 0 | 11 | 42 | 14| 156| 0| 0|| 9.24.105.59| 5 | 5 | 3 | 83 | 7 | 34 | 13| 31| 0| 0|----------------------------------------------------------------------------------------------------| PORT TOTALS: | 27 | 27 | | 148 | | 93 | | 491 | | 0 |----------------------------------------------------------------------------------------------------


We also monitored the load being balanced through the Monitor tool. To start it, click Port:80 in the left pane of the window, and click Port:80->Monitor... in the context menu (see Figure 5-7).


Figure 5-7 Load balancing monitor

5.3 Load balancing with fixed weightsWeights are values applied to each back-end server to determine how Network Dispatcher will distribute the load between all the servers in a cluster. For example, if one server is set to a weight of 20, and the other server is set to a weight of 10, in every 30 connections, the first server will receive 20 and the second server will receive 10.

When you use the Manager, it sets the weight automatically based on information it gets from the Executor, the advisors and ISS. In the previous versions, if you wanted to work with fixed weights, you needed to stop the Manager. In that case, the advisors also are stopped, so the system can no longer determine whether the Web server is responding or not.


Now, you can set a fixed weight for your back-end servers and still have the Manager and the advisors working.

5.3.1 Scenario descriptionWe used a simple load balancing scenario to illustrate this feature (see Figure 5-8).

Figure 5-8 Fixed weights scenario

We have one Dispatcher load balancing to two back-end servers. We will set one server to a fixed weight of 20, and the other server to a fixed weight of 10.

9.24.105.39m23kk904weight:10

9.24.105.59rs60008

weight:20Web servers

9.24.105.62rs600037

NetworkDispatcher

Cluster:9.24.105.87

wses1


Table 5-3 shows more information on the machines we used.

Table 5-3 Machines used in the fixed weight scenario


5.3.2 ConfigurationWe used the basic configuration described in 4.1, “Load balancing” on page 70. We created a cluster (9.24.105.87), added port 80 to this cluster, and added two servers to port 80, 9.24.105.39 and 9.24.105.59 (see Figure 5-9).


Service


Dispatcher


Web server


Web server


Figure 5-9 Basic load balancing configuration

To set a fixed weight for the 9.24.105.39 server, click Server:9.24.105.39 in the left pane of the window, and click the Configuration Settings tab in the right pane of the window (see Figure 5-10).


Figure 5-10 Configuring fixed weights

Locate the Fixed weight field, change it to yes and click the Update Configuration button at the bottom of this pane.

Now that the server is set to work with a fixed weight, you can change the value of the Server weight field. Locate the Server weight field and type in the value you want for this server. We typed in the value 10 for the 9.24.105.39 server. Click the Update Configuration button to update the configuration.


Repeat the same procedure for all servers that will work with fixed weights. In our environment, we followed the same procedure for the second server, 9.24.105.59, and set its weight to 20.

You can use the ndcontrol manager report command to check the weight values for all the servers in a cluster (see Example 5-6).

Example 5-6 Checking the weight values for the back-end servers# ndcontrol manager report



Important: When setting a server to work with a fixed weight, do it in two steps as we described. You first set Fixed weight to yes, then you must update the configuration before setting the value of the weight you want. We ran tests on our machines and had several problems when setting both fields at the same time.


Configuration filesExample 5-7 shows the configuration file that you can use to reproduce the configuration described above.

Example 5-7 Fixed weight configuration filendcontrol executor startndcontrol executor set nfa 9.24.105.62



ndcontrol server add 9.24.105.87:80:9.24.105.39ndcontrol server set 9.24.105.87:80:9.24.105.39 fixedweight yndcontrol server set 9.24.105.87:80:9.24.105.39 weight 10

ndcontrol server add 9.24.105.87:80:9.24.105.59ndcontrol server set 9.24.105.87:80:9.24.105.59 fixedweight yndcontrol server set 9.24.105.87:80:9.24.105.59 weight 20





5.3.3 TestsAfter we set the fixed weight for both servers, we created traffic to simulate several simultaneous connections to this cluster using the shell script shown in Example 4-8 on page 103.

By running this shell script, we were able to create traffic to the cluster address and check the behavior of Network Dispatcher after we set the fixed weights for the back-end servers (see Example 5-8).

Example 5-8 Servers using fixed weights# ndcontrol manager report



Notice that the Dispatcher balances the load according to the weight we specified: server 9.24.105.59 gets twice the traffic that server 9.24.105.39 gets.


You can see the same information graphically using the server monitor, as shown in Figure 5-11.

Figure 5-11 Server monitor showing servers using fixed weights

5.4 Rules-based load balancingYou can use rules-based load balancing to fine-tune when and why packets are sent to which servers. Rules-based load balancing is a feature available only for the Dispatcher and Content Based Routing (CBR) components of Network Dispatcher. These components review any rules you add from first priority to last priority, stopping at the first rule that they find to be true; they then balance the load between any servers associated with the rule.


Using rules expands your ability to distribute connections and offers another possible way to manage the load distribution in a cluster. Rules are particularly useful when you want to distribute the load to a subset of your servers for any reason.

You must always use rules with the CBR component.

5.4.1 Types of rulesYou can use the following types of rules:

� Client IP address

You might want to use rules based on the client IP address if you want to screen the customers and allocate resources based on where they are coming from.

For example, you have noticed that your network is getting a lot of unpaid and therefore unwanted traffic from clients coming from a specific set of IP addresses. You can add a rule that instructs the Dispatcher not to serve those requests, or to distribute them among a limited subset of your servers.

� Time of day

You may want to use rules based on the time of day for capacity planning reasons. For example, if your Web site gets accessed most during the same group of hours every day, you might want to dedicate five servers to HTTP full-time, then add another five during the peak time period.

Another reason you might use a rule based on the time of day could be, for example, that you want to bring some of the servers down for maintenance every night at midnight. In this case, you would set up a rule that excludes those servers during the necessary maintenance period.

� Connections per second on a port

You might want to use rules based on connections per second on a port if you need to share some of your servers with other applications.

For example, you could set two rules:

a. If the number of connections per second on port 80 is greater than 100, then the load should be distributed on two specific Web servers.

b. If the number of connections per second on port 80 is greater than 2000, then the load should be distributed on 10 specific Web servers.


Or you might be using Telnet and want to reserve two of your five servers for Telnet use, except when the connections per second increase above a certain level. Using this rule, the Dispatcher would balance the load across all five servers at peak times.

Note that the Manager must be running for the above scenario to work.

� Total active connections for a port

You might want to use rules based on the total number of active connections on a specific port. This is very useful for when your servers get overloaded and start throwing away packets.

In fact, certain Web servers will continue accepting connections even though they do not have enough threads to respond to the requests. As a result, the client requests time out and the customer coming to your Web site is not served. You can set rules based on the total number of active connections to balance capacity within a pool of servers.

For example, you know from experience that your servers will stop serving requests after they have accepted 250 connections. In that case, you can create a rule that instructs the Dispatcher to use your current servers, but some additional servers are automatically added when the total number of active connections becomes greater than 250. Those additional servers will otherwise be used for other processing.

Note that the Manager must be running for the above scenario to work. This type of rule is available for both the Dispatcher and CBR components.

� Client port

You may want to use rules based on the client port if your clients are running software that uses a specific client port when making requests.

For example, you could create a rule that says that any requests with a client port of 10002 will have to use a set of special fast servers because you know that any client requests with that port is coming from an elite group of customers.

The client port rule type is available only for the Dispatcher component.

� Content of a request

Request content type rules are used to send requests to sets of servers specifically set up to handle some subset of the traffic of your site. For example, you may want to use one set of servers to handle all CGI requests, another set to handle all streaming audio requests, and a third set to handle all other requests. To do this, you will add one rule with a pattern that matches the path to your CGI directory, another that matches the file type for your streaming audio files, and a third rule, which will always be true, to handle the rest of the traffic. You will then add the appropriate servers to each of the rules.


This rule type is available only for the CBR component. For further information about configuring content type rules, see Chapter 11, “Content Based Routing (CBR)” on page 443.

� Type of service

Type of service rules allow you to load balance across servers based on the content of the Type of Service field in the IP header.

For example, if a client request comes in with one TOS value indicating normal service, it can be routed to one set of servers. If a client request comes in with a different TOS value that indicates a higher priority of service, it can be routed to a different set of servers.

This rule type is only available for the Dispatcher component.

� Capacity and bandwidth rules (in bytes per second)

The capacity rules function provides the ability to allocate bandwidth capacity to given servers. This rule is based on the number of bytes served by each server machine. This information will be gathered on a packet-by-packet basis.

This rule is configured by setting an upper and a lower boundary. When this rule is tested, if the current traffic load is within the higher and lower boundaries interval, the rule will fire and a server will be chosen from that rule’s server set.

When the traffic exceeds the reserved bandwidth, you can do either of the following:

– using a rule that is always true, send the traffic to another server that responds with a site busy type of response;

– share a specific amount of bandwidth at the cluster level or the executor level, using the shared bandwidth rule; when the overall shared bandwidth threshold is neared, using an rule that is always true, you can then direct the traffic to another server that responds with a site busy type of response.

� Always true

A rule can be created that is always true. Such a rule will always be selected, unless all the servers associated with it are down. For this reason, it should ordinarily be at a lower priority than other rules. You can even have multiple always-true rules, each with a set of servers associated with it. The first rule with an available server will be chosen.

For example, assume you have six servers. You want two of them to handle your traffic under all circumstances, unless they are both down. If the first two servers are down, you want a second set of servers to handle the traffic. If all four of these servers are down, then you will use the final two servers to


handle the traffic. To do this, you can set up three always-true rules. The first set of servers will then always be chosen, as long as at least one server is up. If they are both down, one server from the second set will be chosen, and so forth.

You can define more than one always-true rule, and thereafter adjust which one gets executed by changing their priority levels.

We recommend that you make a plan of the logic that you want Dispatcher to follow before you start adding rules to your configuration.

5.4.2 How rules are evaluatedAll rules have a name, type, priority, begin range and end range, and may have a set of servers. Rules are evaluated according to the priority order, with lower priority rules evaluated first. In other words, a rule with a priority of 1 will be evaluated before a rule with a priority of 2. The first rule that is satisfied will be used. Once a rule has been satisfied, no further rules are evaluated.

For a rule to be satisfied, it must meet two conditions:

1. The predicate of the rule must be true. That is, the value it is evaluating must be between the begin and end ranges. For always-true rules, the predicate is always satisfied, regardless of the begin and end ranges.

2. If there are servers associated with the rule, at least one of them must be available to which to forward packets.

If a rule has no servers associated with it, it needs to meet only the first condition in order to be satisfied. If no rules are satisfied, a server will be selected from the full set of servers available on the port.


5.4.3 Scenario descriptionTo illustrate the use of rules, we created a scenario with Network Dispatcher and three back-end servers, as shown in Figure 5-12.

Figure 5-12 Rules-based load balancing scenario

We created a rule stating that when the Dispatcher is receiving low traffic for this cluster, it will balance the traffic between two servers: 9.24.105.39 and 9.24.105.59. When the number of connections on this port gets higher than 5 connections per second, it will include a third server, which is 9.24.105.20.

9.24.105.39m23kk904

9.24.105.59rs60008

Webservers

9.24.105.62rs600037

NetworkDispatcherCluster:

9.24.105.87wses1

9.24.105.20rs600036


Table 5-4 shows more information on the machines we used.

Table 5-4 Machines used in the rules-based load balancing scenario


5.4.4 ConfigurationFirst, we configured the regular load balancing for these three servers by following the steps in 4.1, “Load balancing” on page 70. Figure 5-13 shows this configuration.


Service


Dispatcher


Web server


Web server


Web server


Figure 5-13 Basic configuration before creating rules

We tested the environment without using any rules to make sure that the three servers were working as expected (see Example 5-9).

Example 5-9 Configuration without rules# ndcontrol manager report

----------------------------------| HOST TABLE LIST | STATUS |----------------------------------| 9.24.105.39| ACTIVE|| 9.24.105.59| ACTIVE|| 9.24.105.20| ACTIVE|--------------------------------------------------------------------------------------------------------------------------------------| 9.24.105.87| WEIGHT | ACTIVE % 40 | NEW % 40 | PORT % 20 | SYSTEM % 0 |----------------------------------------------------------------------------------------------------


| PORT: 80 | NOW | NEW | WT | CONNECT | WT | CONNECT | WT | LOAD | WT | LOAD |----------------------------------------------------------------------------------------------------| 9.24.105.39| 8 | 8 | 10 | 0 | 9 | 1 | 2| 309| 0| 0|| 9.24.105.59| 10 | 10 | 10 | 0 | 10 | 1 | 13| 84| 0| 0|| 9.24.105.20| 10 | 10 | 10 | 0 | 10 | 1 | 13| 78| 0| 0|----------------------------------------------------------------------------------------------------| PORT TOTALS: | 28 | 28 | | 0 | | 3 | | 471 | | 0 |----------------------------------------------------------------------------------------------------


Once you confirm that the regular load balancing is working as expected for all back-end servers, you can start adding the rules.

Using the GUI, click Port:80 in the left pane of the window, then click Port:80->Add Rule->Total connections (per second) in the context menu. The window shown in Figure 5-14 is displayed.

Figure 5-14 Creating the first rule


We filled in the fields in this window as follows:

� Rule name

The Rule name field can contain any alphanumeric character, underscore, hyphen or period, but it cannot contain any blanks. It can be up to 20 characters long. We chose Conn5 to be the name of this rule.

� Priority

This field is optional. It contains an integer which represents the priority according to which this rule will be evaluated compared to the other rules. The order of evaluation is ascending (from the lowest to the highest).

If you do not fill in any value in this field, Network Dispatcher will do it automatically. The first rule you create receives priority 1; the next one receives priority 11 (which is 1 + 10); the next one receives priority 21 (last priority value + 10), and so forth. We filled in the value 10.

� Begin range

This is the lower value in the range that will determine whether or not the rule is true. Depending on the type of rule you are creating, you can use different values for this field:

– IP address: it is the address of the client. The default is 0.0.0.0.– Time: it is an integer, and the default is 0, which represents midnight.– Total connections: it is an integer, and the default is 0.– Active connections: it is an integer, and the default is 0.– Client port: it is an integer, and the default is 0.– Reserved bandwidth: it is an integer, and the default is 0.

You do not use this field when creating an always true, shared bandwidth or type of service rule.

We filled in the value 5, which means that this rule will be true when the number of connections per second is greater than 5 (and less than the value you specify in the end range field).

� End range

This is the upper value in the range that will determine whether or not the rule is true. Depending on the type of rule you are creating, you can use different values for this field:

– IP address: it is the address of the client. The default is 255.255.255.255.

– Time: it is an integer, and the default is 24, which represents midnight.

– Total connections: it is an integer, and the default is 2 to the 32nd power minus 1.

– Active connections: it is an integer, and the default is 2 to the 32nd power minus 1.


– Client port: it is an integer, and the default is 65535.

– Reserved bandwidth: it is an integer, and the default is 2 to the 32nd power minus 1.

You do not specify an end range for an always true, shared bandwidth or type of service (TOS) rule.

We left this field blank so that it would work with the default value mentioned above.

� Level to evaluate

This field is valid only for Total connections, Active connections and Shared bandwidth rules.

You can choose between evaluating all of the servers on the port or only the servers on the rule.

We selected All servers on port.

� One or more server addresses

This is the list of server(s) you have running. You can optionally select one or more from the list to be included with the rule.

We selected all servers, because when this rule is evaluated as true, it means that Network Dispatcher is receiving more than 5 connections per second for this port, and in this situation we want to balance the load to all three Web servers.

There are a couple of fields that were not shown in Figure 5-14 on page 183 because they are only available for specific rules. They are:

� TOS (valid only for the Type of service rule)

This is an 8-bit entry value consisting of 0, 1, or x.

� Level to share available bandwidth (valid only for the Shared bandwidth rule)

Set the level at which you want bandwidth to be shared. Choose either the cluster or executor level.


After filling in all the fields above, click OK to create the rule. Figure 5-15 shows the configuration including the new rule.

Figure 5-15 Configuration after adding the first rule

Now, we need to create an always true rule. This rule will be used when the previous rule is evaluated false, that means, when the number of connections per second is less than 5.

In this case, we only want the Dispatcher to use two servers, 9.24.105.39 and 9.24.105.59.


To add a new rule, click Port:80 in the left pane of the window, then click Port:80->Add rule->Always true in the context menu. The window shown in Figure 5-16 will be displayed.

Figure 5-16 Adding an always true rule

We filled in the fields in this window as follows:

� Rule name: LowConn.

� Priority: 20 (we want this rule to be evaluated after the first one).

� One or more server addresses: we selected the servers 9.24.105.59 and 9.24.105.39.


See Figure 5-17 for the configuration after adding the second rule.

Figure 5-17 Configuration after adding the always true rule


Configuration fileYou can use the commands in Example 5-10 to create the environment described above.

Example 5-10 Ruled-based load balancing configurationndcontrol executor startndcontrol executor set nfa 9.24.105.62



ndcontrol server add 9.24.105.87:80:9.24.105.39ndcontrol server add 9.24.105.87:80:9.24.105.59ndcontrol server add 9.24.105.87:80:9.24.105.20

ndcontrol rule add 9.24.105.87:80:Conn5 type connection priority 10 beginrange 5 endrange 2147483647ndcontrol rule useserver 9.24.105.87:80:Conn5 9.24.105.59ndcontrol rule useserver 9.24.105.87:80:Conn5 9.24.105.20ndcontrol rule useserver 9.24.105.87:80:Conn5 9.24.105.39

ndcontrol rule add 9.24.105.87:80:LowConn type true priority 20ndcontrol rule useserver 9.24.105.87:80:LowConn 9.24.105.59ndcontrol rule useserver 9.24.105.87:80:LowConn 9.24.105.39




5.4.5 TestsWe began our test by creating a low number of connections, to make sure that only the two servers were going to be used for balancing.

Figure 5-18 shows that only the servers 9.34.105.39 and 9.24.105.59 are being used for load balancing. Note that there is always one packet being sent to our third server, 9.24.105.20. This packet is sent by the Manager to collect the advisor information. Although this server is not currently being used for load balancing, the Dispatcher still needs to collect information from it.


Figure 5-18 Rules-based load balancing to two servers

Our previous test showed that the first part of our configuration is working. Next, we need to increase the number of connections to fire the rule and start using the third server.

We again used a shell script to send a high number of HTTP requests (see Example 4-8 on page 103).

Figure 5-19 shows the traffic after we increased the number of HTTP requests. Note that at the beginning of the graphic, when the load was still low, the 9.24.105.20 server was not being used. Then the load increased and this server started being used for load balancing (after time T-8 on the graphic).


Figure 5-19 Rules-based load balancing to three servers

The third server will only be used when the number of connections is higher than 5. If we decrease the load, it will become idle again.

5.5 Wide area Network Dispatcher supportNetwork Dispatcher provides a wide area network (WAN) Dispatcher enhancement that offers support for remote servers. A remote server consists of a remote Dispatcher machine and its locally attached servers. A client’s packet can now go from the Internet to a Dispatcher machine, from there to a geographically remote Dispatcher machine, then to one of its locally attached servers, and from the server directly back to the Internet and the client. See an example in Figure 5-20.


Figure 5-20 Example of layout for a wide area Network Dispatcher

This allows one cluster address to support all worldwide client requests while distributing the load to servers around the world.

The Dispatcher machine initially receiving the packet can still have local servers attached to it and can distribute the load among its local servers as well as the remote servers.

Note that wide area support is currently available only for the Dispatcher component of Network Dispatcher.

5.5.1 Scenario descriptionWe created a simple scenario to illustrate this feature. We have one Network Dispatcher machine, balancing the load to two local Web servers and two remote Web servers. To be able to load balance to remote Web servers, we need to configure another Network Dispatcher machine on the same network as the remote Web servers (see Figure 5-21).

Back endWeb servers

Network Dispatchers

Network Dispatcher


Figure 5-21 Wide area Network Dispatcher scenario

When a client sends an HTTP request, it is received by the local Network Dispatcher, 9.24.105.62. This Dispatcher will load balance between two local Web servers, 9.24.105.39 and 9.24.105.59, and two remote Web servers, 192.168.10.6 and 192.168.10.7. The load balancing of the two remote Web servers will actually be handled by the remote Network Dispatcher, 192.168.10.14.

The following table shows more information on the machines we used for this scenario.

Table 5-5 Machines used in the wide area network scenario


Service


Local Dispatcher


Local Web server

9.24.105.39m23kk904

9.24.105.59rs60008

LocalWeb

servers

Cluster:9.24.105.87

wses1R

9.24.105.14

192.168.10.1

192.168.10.14rs600010

192.168.10.7was02

192.168.10.6was01

9.24.105.62rs600037

RemoteWeb

servers

LocalNetwork

Dispatcher

RemoteNetwork

Dispatcher


5.5.2 ConfigurationFirst, we configured the local Dispatcher (9.24.105.62) based on the configuration described in 4.1, “Load balancing” on page 70. We added only the two local Web servers, as shown in Figure 5-22.


Local Web server


Remote Dispatcher

was01 192.168.10.6 Windows NT 4.0WebSphere Application Server V3.5 with PTF3

Remote Web server

was02 192.168.10.7 Windows NT 4.0WebSphere Application Server V3.5 with PTF3

Remote Web server

fw01 9.24.105.14192.168.10.1

Windows NT 4.0IBM Firewall for NT V4.1

Router between networks


Service


Figure 5-22 Basic configuration of the first Dispatcher

The next thing we did was to set the wide area port number. This port will be used by the local and the remote Dispatchers to communicate. You can select any port number, as long as it is not currently being used in any of the Dispatchers. You can check that by using the command netstat -an.

We selected the port number 23456. To add the port number, click Executor:9.24.105.62 in the left pane of the window, click the Configuration Settings tab in the right pane and locate the field Wide port number (see Figure 5-23).


Figure 5-23 Setting the Wide port number field

Fill in the port number you selected, then click the Update Configuration button at the bottom of the window to activate the change. We typed in the port number 23456.

In this configuration, we will not add the remote servers directly to the cluster on this Dispatcher. We will add the remote Dispatcher as if it were a server, and both Dispatchers will exchange information about the requests that need to be sent to the remote Web servers.


To include the remote Dispatcher, click Port:80 in the left pane of the window, then click Port:80->Add server..., and fill in the Server address field with the non-forwarding address of the remote Dispatcher; select the Network router address checkbox, and fill in the IP address of the router that you use to get to the remote network (see Figure 5-24).

Figure 5-24 Adding the remote Dispatcher as a server

The next step is to configure the remote Dispatcher. Follow the steps described in 4.1, “Load balancing” on page 70, but take note of a few changes:

� When adding the cluster, use the same cluster IP address that you used for the local Dispatcher, which is 9.24.105.87 (see Figure 5-25).

Figure 5-25 Adding the cluster on the second Dispatcher

� The primary host for this cluster is the local Dispatcher we configured (the one that will receive the requests from the Web browsers). In our scenario, the primary host for this cluster is the Dispatcher 9.24.105.62.


� Note that this cluster must not be configured, so do not select the Configure this cluster? checkbox.

After you have added the cluster, proceed with adding the port you will use (in our scenario, we are using port 80) and the back-end servers. In this scenario, the back-end servers are the servers 192.168.10.6 and 192.168.10.7. See Figure 5-26 for the complete configuration of the remote Dispatcher.

Figure 5-26 Configuration of the remote Dispatcher


The next step is to set the wide area port number for this remote Dispatcher the same way we did for the local Dispatcher. To add the port number, click Executor:192.168.10.14 in the left pane of the window, and click the Configuration Settings tab in the right pane. Locate the Wide port number field and fill in the same number that you used for the local Dispatcher. Since we used 23456 on the local Dispatcher, we also set this port number on the remote Dispatcher (see Figure 5-27).

Figure 5-27 Setting the Wide port number on the second Dispatcher

Working with advisorsNote that we started the Manager and the advisors on both Dispatchers. The first Dispatcher will work with advisors without any further configuration, but the second Dispatcher (and all remote Dispatchers) will not, since it does not have the IP alias on its network interface.


The remote Dispatcher cannot have an IP alias on its network interface (the IP alias can only be added to the local Dispatcher), so we added the IP alias to the loopback adapter. Refer to “Configuration of the back-end servers” on page 95 for more information about adding an IP alias.

Since we are using AIX, we ran the following command to add an IP alias to the loopback adapter:


This command adds an IP alias to the loopback adapter, but it also adds a route to the whole 9.24.105.0 network using the local IP alias as a gateway. This route disables all communication between this machine and the 9.24.105.0 network, so we must remove it, using the following command:

# route delete 9.24.105.0 9.24.105.87

In its place, we must add a host route with destination to the cluster IP address (9.24.105.87) using the loopback as gateway. We used the following command:

# route add -host 9.24.105.87 127.0.0.1

You should also add these commands to the /etc/rc.net file so that they will be run every time the machine is rebooted, or every time the network is reconfigured.

Edit the /etc/rc.net file and add the following lines at the bottom of the file:

ifconfig lo0 alias 9.24.105.87 netmask 255.255.255.0route delete 9.24.105.0 9.24.105.87route add -host 9.24.105.87 127.0.0.1


Now all advisors will be able to detect whether or not the local and remote Web servers are available. Figure 5-28 shows the advisor status on the local Dispatcher. Note that it gets information from all back-end servers, including the remote ones (it shows the IP address of the remote Dispatcher).

Figure 5-28 Advisor on the local Dispatcher


Figure 5-29 shows the advisor status on the remote Dispatcher. Note that this screen shows the information for each remote Web server individually.

Figure 5-29 Advisor on the remote Dispatcher

Firewall issuesIf you are using a firewall or a router with Access Control List, you need to create filters to allow communication between the Dispatchers.


Figure 5-30 shows the communication that we identified between the Dispatchers in our scenario. Use this as a reference to create the filters on your firewall.

Figure 5-30 Communication between the two Dispatchers

Configuration filesIn this section, we show you the configuration files for both Dispatchers.

192.168.10.14rs600010

9.24.105.62rs600037

LocalNetwork Dispatcher

RemoteNetwork Dispatcher

R

Protocol 47

Echo request (ping)

Echo reply (ping)


Example 5-11 shows the configuration file for the local Dispatcher, which is the one that will receive all requests from the clients. In our scenario, the local Dispatcher is 9.24.105.62.

Example 5-11 Configuration file for the local Dispatcherndcontrol executor startndcontrol executor set nfa 9.24.105.62ndcontrol executor set wideportnumber 23456




ndcontrol server add 9.24.105.87:80:192.168.10.14 router 9.24.105.14ndcontrol cluster configure 9.24.105.87 en0 255.255.255.0



Example 5-12 shows the configuration file for the remote Dispatcher, which is the one located in the remote network. You can use the same commands for all your remote Dispatchers (in case you are working with more than one remote network).

Example 5-12 Configuration file for the remote Dispatcherndcontrol executor startndcontrol executor set nfa 192.168.10.14ndcontrol executor set wideportnumber 23456

ndcontrol cluster add 9.24.105.87 primaryhost 9.24.105.62ndcontrol cluster set 9.24.105.87 primaryhost 9.24.105.62






5.5.3 TestsWe used the script shown in Example 4-8 on page 103 to generate traffic for this test. We sent several HTTP requests to 9.24.105.87, which is the IP address of the cluster. The requests were received by the local Dispatcher, 9.24.105.62, which balanced them between the two local Web servers and the remote Web servers, managed by the remote Dispatcher, 192.168.10.14.

Figure 5-31 shows the server monitor on the local Dispatcher, and how the load is being balanced among the servers. Note that the graphic does not show the two remote Web servers, it only shows the IP address of the remote Dispatcher.

Figure 5-31 Server monitor on the local Dispatcher


If you want to monitor the load balancing on the remote Web servers, you need to run the server monitor on the remote Dispatcher. Figure 5-32 shows the server monitor running on the remote Dispatcher. Note that it now shows the load on the remote Web servers individually.

Figure 5-32 Server monitor on the remote Dispatcher

5.6 Mutual high availabilityThe mutual high availability feature of Network Dispatcher provides the administrator the opportunity to use both Dispatcher machines in a high availability configuration, the primary and the backup machines, to work simultaneously to balance the load for different clusters while also providing backup for each other.

In case one server fails, the other one will balance the load for the clusters assigned to it, and also the clusters assigned to the failed Dispatcher.


5.6.1 Scenario descriptionWe created a scenario based on the scenario we described in 4.2, “High availability” on page 104. We assigned a new cluster, wses2.itso.ral.ibm.com, to the backup machine (rs600010), as shown in Figure 5-33, and two Web servers to this new cluster.

Figure 5-33 Mutual high availability scenario

9.24.105.39m23kk904

9.24.105.59rs60008

Web servers

Network Dispatcher


wses1Primary:rs600037Backup:rs600010

9.24.105.62rs600037

9.24.105.60rs600010

9.24.105.20rs600036

9.24.105.63rhlinux1


wses2Primary:rs600010Backup:rs600037




5.6.2 ConfigurationConfiguring mutual high availability is similar to configuring regular high availability. The only difference is that when adding the clusters, we need to inform each server as to which primary machine will handle each cluster.

This will allow us to keep some clusters active on one Dispatcher, and other clusters active on the other Dispatcher, each Dispatcher being the backup of the other.

Configuring the clusters on the DispatchersWe created the load balancing configuration on both Dispatcher servers independently. We used a remote configuration client so we could work on both machines at the same time.

First, we configured the 9.24.105.62 server (see 4.1, “Load balancing” on page 70). We added the 9.24.105.87 cluster with two servers, 9.24.105.39 and 9.24.105.59.


Service


Primary Dispatcher for cluster wses1


Primary Dispatcher for cluster wses2


Web server for cluster wses1



rhlinux1 9.24.105.63 Red Hat Linux V6.2Apache HTTP Server





Next, we configured the 9.24.105.60 server. We added the 9.24.105.88 cluster and two servers, 9.24.105.63 and 9.24.105.20. We also started the Manager and the HTTP advisor on both Dispatchers. The basic configuration of both Dispatchers is shown in Figure 5-34.

Figure 5-34 Basic configuration of both Dispatchers

Once we have both Dispatchers working for each cluster, we can start configuring high availability on each of them.

First, we remove the IP alias that we had created on each Dispatcher for its corresponding cluster. Since we created the IP alias using the GUI, we also remove it using the GUI. Locate the cluster that you configured, and click Cluster:[IP address]->Unconfigure Cluster Address.


Make sure that all IP aliases are removed before continuing. Refer to “Checking the IP aliases on the Dispatcher” on page 90 for more information on how to check the IP aliases.

The next thing you need to do is to create the configuration of the clusters on each backup machine. In our environment, we first add the 9.24.105.88 cluster to the first Dispatcher (24.105.62). Note that when creating this cluster on 9.24.105.62, we informed the 9.24.105.62 Dispatcher that the primary machine for this cluster is the other Dispatcher, 9.24.105.60 (see Figure 5-35).

Figure 5-35 Configuring the 9.24.105.88 cluster on the 9.24.105.62 Dispatcher

Note that you must not select the Configure this cluster? checkbox. We will configure all IP aliases later using the high availability scripts.


Now, continue the configuration of this cluster by adding the port and the servers as you did on the second Dispatcher. See Figure 5-36 for the complete configuration of this new cluster on the 9.24.105.62 Dispatcher.

Figure 5-36 New cluster on the Dispatcher 9.24.105.62

Now you must do the same thing on the second Dispatcher (9.24.105.60). You must add the cluster for which it will act as a backup. In our scenario, this is the 9.24.105.87 cluster.


When you add the cluster, remember to use 9.24.105.62 as the primary machine for this cluster, as shown in Figure 5-37.

Figure 5-37 Adding the cluster 9.24.105.87 to the Dispatcher 9.24.105.60


Add the port and the servers to this cluster. After you finish, compare this to Figure 5-38, which shows the configuration of the clusters on the Dispatcher 9.24.105.60.

Figure 5-38 Configuration of the clusters on the second Dispatcher

Once both Dispatchers are configured with all the clusters, we can work on the high availability scripts.

Creating the high availability scriptsThe high availability scripts work similarly in mutual high availability (see “Creating the high availability scripts” on page 120 for more information about these scripts). You need to create the goActive, goStandby and goInOp on both machines, and place them in the directory <install path>/dispatcher/bin.

The main difference with the mutual high availability scripts is that Network Dispatcher calls them with a parameter identifying the primary machine IP address. The scripts must query this parameter and perform the ifconfig and ndconfig commands for the cluster IP addresses associated with that primary machine.

First, we created the scripts on the 9.24.105.62 Dispatcher. This Dispatcher is primary for the 9.24.105.87 cluster and backup for the 9.24.105.88 cluster (see Figure 5-33 on page 207).

The first script, goActive, must add the IP aliases of the active clusters to the network interfaces. This script checks the parameter that is passed by Network Dispatcher, and it will add the IP aliases of the corresponding clusters (se)e Example 5-13.

Example 5-13 goActive script on the 9.24.105.62 Dispatcher #!/bin/ksh# goActive script#LOCAL_NFA=9.24.105.62REM_NFA=9.24.105.60

CLUSTER1=9.24.105.87CLUSTER2=9.24.105.88

NETMASK1=255.255.255.0NETMASK2=255.255.255.0

INTERFACE=en0

if [[ "$1" = $LOCAL_NFA ]]; then # Activating local cluster ifconfig lo0 delete $CLUSTER1 ifconfig $INTERFACE alias $CLUSTER1 netmask $NETMASK1fi

if [[ "$1" = $REM_NFA ]]; then # Activating remote cluster ifconfig lo0 delete $CLUSTER2 ifconfig $INTERFACE alias $CLUSTER2 netmask $NETMASK2fi

We defined two variables, LOCAL_NFA, which represents the local IP address of the Dispatcher, and REM_NFA, which represents the IP address of the second Dispatcher. Note that both variables will change on the second Dispatcher.

CLUSTER1 is the local cluster, and CLUSTER2 is the remote cluster. NETMASK1 is the netmask for CLUSTER1, and NETMASK2 is the netmask for CLUSTER2.


The if clauses check which parameter was passed (the local or the remote IP address), and will add the appropriate IP aliases.

The next script, goStandby, uses the same variables and displays the same behavior, but this script will remove the IP aliases from the network interfaces and add them to the loopback interface (see Example 5-14).

Example 5-14 goStandby script on the 9.24.105.62 Dispatcher#!/bin/ksh# goStandby script#LOCAL_NFA=9.24.105.62REM_NFA=9.24.105.60

CLUSTER1=9.24.105.87CLUSTER2=9.24.105.88

NETMASK1=255.255.255.0NETMASK2=255.255.255.0

INTERFACE=en0

if [[ "$1" = $LOCAL_NFA ]]; then # Deactivating local cluster ifconfig $INTERFACE delete $CLUSTER1 ifconfig lo0 alias $CLUSTER1 netmask $NETMASK1fi

if [[ "$1" = $REM_NFA ]]; then # Deactivating remote cluster ifconfig $INTERFACE delete $CLUSTER2 ifconfig lo0 alias $CLUSTER2 netmask $NETMASK2fi

Note that on goStandby, the variables remain the same. The only difference is in the ifconfig commands.


The last script to be created on this server is goInOp. This script will remove all IP aliases that were added to the local network interface and to the loopback interface, so we do not need to check the parameter this time (see Example 5-15).

Example 5-15 goInOp script on the 9.24.105.62 Dispatcher#!/bin/ksh# goInOp script#CLUSTER1=9.24.105.87CLUSTER2=9.24.105.88

NETMASK1=255.255.255.0NETMASK2=255.255.255.0

INTERFACE=en0

# Deactivating local clusterifconfig $INTERFACE delete $CLUSTER1ifconfig lo0 delete $CLUSTER1

# Deactivating remote clusterifconfig $INTERFACE delete $CLUSTER2ifconfig lo0 delete $CLUSTER2

Now that the scripts are all ready on the first Dispatcher, we need to do the same configuration on the second Dispatcher, 9.24.105.60.

The scripts are similar, except for the IP addresses that we associate to the variables.


The first script, goActive, works exactly the same way as it did on the 9.24.105.62 Dispatcher, but we had to change the IP addresses of the variables (see Example 5-16).

Example 5-16 goActive script on the 9.24.105.60 Dispatcher#!/bin/ksh# goActive script#LOCAL_NFA=9.24.105.60REM_NFA=9.24.105.62

CLUSTER1=9.24.105.88CLUSTER2=9.24.105.87

NETMASK1=255.255.255.0NETMASK2=255.255.255.0

INTERFACE=en0

if [[ "$1" = $LOCAL_NFA ]]; then # Activating local cluster ifconfig lo0 delete $CLUSTER1 ifconfig $INTERFACE alias $CLUSTER1 netmask $NETMASK1fi

if [[ "$1" = $REM_NFA ]]; then # Activating remote cluster ifconfig lo0 delete $CLUSTER2 ifconfig $INTERFACE alias $CLUSTER2 netmask $NETMASK2fi

We associated LOCAL_NFA to the 9.24.105.60 IP address and REM_NFA to the 9.24.105.62 IP address. We also changed CLUSTER1 to 9.24.105.88 (which is the local cluster) and CLUSTER2 to 9.24.105.87 (which is the remote cluster). In our scenario, the netmasks are the same, so check if you need to change them also.


The next script, goStandby, is shown in Example 5-17.

Example 5-17 goStandby script on the 9.24.105.60 Dispatcher#!/bin/ksh# goStandby script#LOCAL_NFA=9.24.105.60REM_NFA=9.24.105.62

CLUSTER1=9.24.105.88CLUSTER2=9.24.105.87

NETMASK1=255.255.255.0NETMASK2=255.255.255.0

INTERFACE=en0

if [[ "$1" = $LOCAL_NFA ]]; then # Deactivating local cluster ifconfig $INTERFACE delete $CLUSTER1 ifconfig lo0 alias $CLUSTER1 netmask $NETMASK1fi

if [[ "$1" = $REM_NFA ]]; then # Deactivating remote cluster ifconfig $INTERFACE delete $CLUSTER2 ifconfig lo0 alias $CLUSTER2 netmask $NETMASK2fi

Note that the changes we made to goActive were also made to this script.


The last script, goInOp, is shown in Example 5-18.

Example 5-18 goInOp script on the 9.24.105.60 Dispatcher#!/bin/ksh# goInOp script#CLUSTER1=9.24.105.88CLUSTER2=9.24.105.87

NETMASK1=255.255.255.0NETMASK2=255.255.255.0

INTERFACE=en0

# Deactivating local clusterifconfig $INTERFACE delete $CLUSTER1ifconfig lo0 delete $CLUSTER1

# Deactivating remote clusterifconfig $INTERFACE delete $CLUSTER2ifconfig lo0 delete $CLUSTER2

Important: If you are working in a UNIX environment, make sure that the high availability scripts have execute permission. If they do not, run the following command inside the <install path>/dispatcher/bin directory of each Dispatcher:

# chmod 700 go*

Configuring mutual high availabilityNow that all scripts are ready, you can start adding the high availability configuration to each Dispatcher.

In our scenario, we first added the heartbeat on the first Dispatcher, which is 9.24.105.62. Click Executor:9.24.105.62, then click Executor:9.24.105.62->Add Heartbeat in the context menu. The heartbeat is configured using the local IP address (9.24.105.62) as source and the second Dispatcher IP address (9.24.105.60) as destination (see Figure 5-39).

Figure 5-39 Adding the heartbeat on the first Dispatcher

Do the same thing on the second Dispatcher, using the local IP address (9.24.105.60) as source and the remote IP address (9.24.105.62) as destination (see Figure 5-40).

Figure 5-40 Adding the heartbeat on the second Dispatcher

Now add the reach target to both Dispatchers. We chose the reach target in our scenario to be our default gateway router (9.24.105.1).


Click Executor:9.24.105.62, then click Executor:9.24.105.62->Add Reach Target in the context menu, and fill in the IP address you have chosen, as shown in Figure 5-41.

Figure 5-41 Adding the reach target

To add the reach target on the second Dispatcher, click Executor:9.24.105.60, then click Executor:9.24.105.60->Add Reach Target in the context menu, and fill in the same IP address you used on the first Dispatcher.

Finally, add the backup information to each Dispatcher as shown below.

On the first Dispatcher, click Executor:9.24.105.62, then click Executor:9.24.105.62->Add High Availability Backup in the context menu. The window shown in Figure 5-42 is displayed.

Figure 5-42 Adding backup information

Choose a port number that is not in use in any of the Network Dispatcher machines, and enter it in the Port number field. In the Role field, select Both for mutual high availability. In the Recovery Strategy field, we selected Auto. Refer to 4.3.2, “Configuration” on page 129 for more details about these fields.


On the second Dispatcher, click Executor:9.24.105.60, then click Executor:9.24.105.60->Add High Availability Backup in the context menu, and add the same information you did for the first server (see Figure 5-42).

As soon as you add the backup information, the high availability scripts are also run.

You can check the high availability status on the first Dispatcher by clicking Executor:9.24.105.62, and clicking the High Availability tab in the right pane of the window, as shown in Figure 5-43.

Figure 5-43 High availability configuration on the first Dispatcher


Click Refresh Statistics to see the current information.

Note that this Dispatcher is in Active state for the 9.24.105.87 cluster and in Standby state for the 9.24.105.88 cluster. You can also see this information by running the command ndcontrol high status, as shown in Example 5-19.

Example 5-19 Status of high availability for the first DispatcherHigh Availability Status:-------------------------Role ................. PrimaryRecovery strategy .... AutoState ................ ActiveSub-state ............ SynchronizedPrimary host ......... 9.24.105.62Port ................. 12345Preferred target ..... 9.24.105.60





Make sure that the IP aliases were created by the scripts. Use the command ifconfig -a, as shown in Example 5-20.

Example 5-20 IP aliases on the first Dispatcher# ifconfig -alo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0 inet 9.24.105.88 netmask 0xffffff00 broadcast 9.24.105.255en0: flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,PSEG> inet 9.24.105.62 netmask 0xffffff00 broadcast 9.24.105.255 inet 9.24.105.87 netmask 0xffffff00 broadcast 9.24.105.255

Note that the local cluster IP alias (9.24.105.87) is in the local network interface, en0, while the remote cluster IP alias (9.24.105.88) is in the loopback interface.

Now check the configuration on the second Dispatcher by clicking Executor:9.24.105.60, then clicking the High Availability tab in the right pane of the window, as shown in Figure 5-44.

Figure 5-44 High availability configuration on the second server

Click Refresh Statistics to see the current information.


Note that this Dispatcher is in Active state for the 9.24.105.88 cluster and in Standby state for the 9.24.105.87 cluster. You can also see this information by running the command ndcontrol high status, as shown in Example 5-21.

Example 5-21 Status of high availability for the second DispatcherHigh Availability Status:-------------------------Role ................. PrimaryRecovery strategy .... AutoState ................ ActiveSub-state ............ SynchronizedPrimary host ......... 9.24.105.60Port ................. 12345Preferred target ..... 9.24.105.62





Make sure that the IP aliases were also created by the scripts on this machine, as shown in Example 5-22.

Example 5-22 IP aliases on the second Dispatcher# ifconfig -alo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0 inet 9.24.105.87 netmask 0xffffff00 broadcast 9.24.105.255en0: flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,PSEG> inet 9.24.105.60 netmask 0xffffff00 broadcast 9.24.105.255 inet 9.24.105.88 netmask 0xffffff00 broadcast 9.24.105.255

Note that the local cluster IP alias (9.24.105.88) is in the local network interface, en0, while the remote cluster IP alias (9.24.105.87) is in the loopback interface.

Once this configuration is finished, if either of the Dispatchers fails to respond to the network, the other Dispatcher will take over all clusters.

Configuration filesAfter we saved the configuration on both servers, the following files were generated.

The configuration file generated on the first Dispatcher, 9.24.105.62, is shown in Example 5-23:

Example 5-23 Configuration file of the first Dispatcherndcontrol executor startndcontrol executor set nfa 9.24.105.62









ndcontrol highavailability heartbeat add 9.24.105.62 9.24.105.60ndcontrol highavailability backup add both auto 12345ndcontrol highavailability reach add 9.24.105.1


The configuration file generated on the second Dispatcher, 9.24.105.60, is shown in Example 5-24:

Example 5-24 Configuration file of the second Dispatcherndcontrol executor startndcontrol executor set nfa 9.24.105.60









ndcontrol highavailability heartbeat add 9.24.105.60 9.24.105.62ndcontrol highavailability backup add both auto 12345ndcontrol highavailability reach add 9.24.105.1

5.6.3 TestsRefer to 4.2.3, “Tests” on page 123 for more information on testing a high availability environment.

Since we are using the auto recovery strategy, we tested the environment by simulating the failure of one of the servers.

We stopped the Ethernet interface on the second Dispatcher (9.24.105.60) using the command below.

# ifconfig en0 down

This forces the first Dispatcher to take over the 9.24.105.88 cluster.


After the first Dispatcher stops receiving the heartbeat from the second Dispatcher, it takes over all clusters, as shown in Example 5-25.

Example 5-25 High availability status after the failure of the second Dispatcher# ndcontrol high status

High Availability Status:-------------------------Role ................. PrimaryRecovery strategy .... AutoState ................ ActiveSub-state ............ Not SynchronizedPrimary host ......... 9.24.105.62Port ................. 12345Preferred target ..... n/a

High Availability Status:-------------------------Role ................. BackupRecovery strategy .... AutoState ................ ActiveSub-state ............ Not SynchronizedPrimary host ......... 9.24.105.60Port ................. 12345Preferred target ..... n/a




The first Dispatcher also adds the IP alias of the 9.24.105.88 cluster to its local network interface, since it runs the goActive script passing as parameter the IP address of the failed Dispatcher, as shown in Example 5-26.

Example 5-26 ifconfig after the failure of the second Dispatcher# ifconfig -alo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0en0: flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,PSEG> inet 9.24.105.62 netmask 0xffffff00 broadcast 9.24.105.255 inet 9.24.105.87 netmask 0xffffff00 broadcast 9.24.105.255 inet 9.24.105.88 netmask 0xffffff00 broadcast 9.24.105.255

After we brought the second Dispatcher back up, it took over its cluster (remember that we used auto recovery) and the first Dispatcher was once again in standby mode for this cluster.

5.7 AffinityThe affinity feature is enabled by configuring a cluster’s port to be sticky. When you set a port to be sticky, all subsequent requests sent to it will be forwarded to the same back-end server.

You can enable the classical affinity by setting the port stickytime to the amount of time (in seconds) you want Network Dispatcher to keep all connections coming from the same IP address to be forwarded to the same back-end server. You can disable this feature by setting the port stickytime to zero.

When Network Dispatcher is not using affinity, whenever a new TCP connection is received from a client, Network Dispatcher picks the right back-end server at that moment and forwards the packet to it. If the same client opens a new connection to the same cluster, it is treated as an unrelated connection, and Network Dispatcher will again analyze it and pick the right back-end server at that time.

With affinity enabled, Network Dispatcher will forward the new connection to the same back-end server as the previous connection. Over time, the client will stop sending packets, and the affinity record will be erased. Each affinity record will be kept for the duration of the stickytime (in seconds). The new connections received within the stickytime are still forwarded to the same back-end server. If a new connection is received after that time, the affinity record is purged and a new back-end server is selected for it.

Enhancements have been made to the classical affinity. We discuss them in the next sections.

5.7.1 Server Directed Affinity API to control client-server affinityThe Server Directed Affinity API only applies to the Dispatcher component.

The SDA feature provides an API that allows an external agent to influence the Dispatcher affinity behavior.

SDA featuresYour application may have indicated that its server systems have the knowledge to direct client requests to particular server machines better than the Dispatcher can. Rather than having a client forwarded to the same server selected by Dispatcher’s load balancing, you may want the client to be forwarded to the server of your choice. The SDA feature provides this API. You can now write your own software to implement an SDA agent, which communicates with a listener in Dispatcher. It can then manipulate the Dispatcher affinity tables to:

� Query the contents

� Insert new records

� Remove records

Records inserted in an affinity table by an SDA agent remain in the table indefinitely. They do not time out. They are removed only when the SDA agent removes them or if a Dispatcher advisor detects that the server is dead.

Note: Affinity is based on the IP address of the client. If the client is using a proxy or SOCKS server, all connections from the same network will use the same source IP address. If you are using affinity, all clients from those networks will get sticky to the same server on your cluster. Consider this when configuring your environment.


Dispatcher’s SDA componentsThe Dispatcher implements a new socket listener to accept and handle requests from an SDA agent. When an SDA agent opens a connection with the Dispatcher, the listener will accept it and leave the connection open. Multiple requests and responses can flow over this persistent connection. The socket will close when the SDA agent closes it or if the Dispatcher detects an unrecoverable error.

Inside the Dispatcher, the listener takes each request from the SDA agent, communicates with the appropriate affinity table in the Dispatcher executor kernel, and prepares a response for the SDA agent. For more information, refer to the following files after the Dispatcher has been installed.

� the API: samples/SDA/SDA_ API.htm

� the sample code for an SDA agent: samples/SDA/SDA_SampleAgent.java

The files can be found in the Dispatcher’s install directory tree.

5.7.2 Cross port affinityCross port affinity only applies to the Dispatcher component.

Cross port affinity is the sticky feature that has been expanded to cover multiple ports. For example, if a client request is first received on one port and the next request is received on another port, cross port affinity allows the dispatcher to send the client request to the same server. In order to use this feature, the ports must:

� share the same cluster address

� share the same servers

� have the same stickytime value (not zero)

� have the same stickymask value

More than one port can link to the same cross port. When subsequent connections come in from the same client on the same port or a shared port, the same server will be accessed. The following is an example of the configuration of multiple ports with a cross port affinity to port 80:

# ndcontrol port set cluster:20 crossport 80# ndcontrol port set cluster:30 crossport 80# ndcontrol port set cluster:40 crossport 80


After cross port affinity has been established, you have the flexibility to modify the stickytime value for the port. However, it is recommended that you change the stickytime values for all shared ports to the same value, otherwise unexpected results may occur.

To remove the cross port affinity, set the cross port value back to its own port number.

5.7.3 Affinity address maskAffinity address mask only applies to the Dispatcher component.

Affinity address mask is a sticky feature enhancement to groups of clients based upon common subnet addresses. Specifying stickymask in the ndcontrol port command allows you to mask the common high-order bits of the 32-bit IP address. If this feature is enabled, when a client request first makes a connection to the port, all subsequent requests from clients with the same subnet address (represented by that part of the address which is being masked) will be directed to the same server.

For example, if you want all incoming client requests with the same network Class A address to be directed to the same server, you set the stickymask value to 8 (bits) for the port. To group client requests with the same network Class B address, set the stickymask value to 16 (bits). To group client requests with the same network Class C address, set the stickymask value to 24 (bits).

For best results, set the stickymask value when first starting the Network Dispatcher. If you change the stickymask value dynamically, results may be unpredictable.

Keep in mind that there is interaction with cross port affinity: if you are enabling cross port affinity, stickymask values of the shared ports must be the same. See 5.7.2, “Cross port affinity” on page 233 for more information.

To enable affinity address mask, issue an ndcontrol port command similar to the following:

# ndcontrol port set cluster:port stickymask 8

Possible stickymask values are 8, 16, 24 and 32. A value of 8 specifies that the first 8 high-order bits of the IP address (network Class A address) will be masked. A value of 16 specifies that the first 16 high-order bits of the IP address (network Class B address) will be masked. A value of 24 specifies that the first 24


high-order bits of the IP address (network Class C address) will be masked. If you specify a value of 32, you are masking the entire IP address, which effectively disables the affinity address mask feature. The default value of stickymask is 32.

5.7.4 Rule affinity overrideWith rule affinity override, you can override the stickiness of a port for a specific server. For example, let us say that you are using a rule to limit the number of connections to each application server, and you have an overflow server with an always true rule that says please try again later for that application. The port has a stickytime value of 25 minutes, but you don’t want the client to be sticky to the overflow server. With rule affinity override, you can change the overflow server to override the affinity normally associated with that port. The next time the client requests the cluster, it is load balanced to the best available application server, not to the overflow server.

5.7.5 Cookie affinityThe cookie affinity feature applies only to Content Based Routing with WTE (for HTTP), which supports load balancing based on rules. It provides a new way to make clients sticky to a particular server. This function is enabled by setting the stickytime of a rule to a positive number, and setting the affinity to cookie. This can be done when the rule is added, or by using the rule set command.

Once a rule has been enabled for cookie affinity, new client requests will be load-balanced using standard CBR algorithms, while succeeding requests from the same client will be sent to the initially chosen server. The chosen server is stored as a cookie in the response to the client. As long as the client’s future requests contain the cookie, and each request arrives within the stickytime interval, the client will maintain affinity with the initial server.

Cookie affinity is used to ensure that a client continues to be load balanced to the same server for some period of time. This is accomplished by sending a cookie to be stored by the client browser. The cookie contains:

� the cluster:port:rule that was used to make the decision,

� the server that was load balanced to, and

� a timeout timestamp for when the affinity is no longer valid.

Whenever a rule fires that has cookie affinity turned on, the cookies sent by the client are examined. If a cookie is found to contain the identifier for the cluster:port:rule that fired, then the server that was load balanced to, and the expires timestamp are extracted from the cookie. If:


� the server is still in the set used by the rule, and

� its weight is greater than zero, and

� the expires timestamp is greater than the current time,

the server in the cookie is chosen for load balancing. If any of the preceding three conditions are not met, a server is chosen using the normal algorithm.

Once a server has been chosen, a new cookie is constructed using the five pieces of information (IBMCBRcluster:port:rule=timestamp), the timestamp will be the time that affinity expires. The “cluster:port:rule” and the “server_chosen” are encoded so that no information about the CBR configuration is revealed. An expires parameter is also inserted in the cookie. This parameter is in a format the browser can understand, and causes the cookie to become invalid two hours after the expires timestamp. This is done to avoid cluttering the client’s cookie database.

This new cookie is then inserted in the headers that go back to the client. If the client browser is configured to accept cookies, it will send back subsequent requests.

How to enable cookie affinityTo enable cookie affinity for a particular rule, use the rule set command:

# rule set cluster:port:rule stickytime 60 # rule set cluster:port:rule affinity cookie

Stickiness is set per rule. Two different rules can send the same client to two different servers. For one rule, the client will be sent to one server, for another rule the client could be sent to a different server. This is a requirement, and for CBR, the servers for rules will be different, so a server using a rule will probably not even exist for another rule. This is desirable because you can set up your rule that handles CGI or servlet requests to be sticky. Your rule for normal HTTP requests will use normal load balancing.

Why use cookie affinityA sticky rule would normally be used for CGI or servlets that store client state on the server. The state is identified by a cookie ID (these are server cookies). The client state is only on the selected server, so the client needs the cookie from that server to maintain that state between requests.

Note: Refer to 11.2, “CBR for HTTP” on page 445 for a scenario using cookie affinity.


Normal client IP affinityNormal client IP affinity for CBR is set by rule. To enable normal affinity use:

# rule set cluster:port:rule stickytime 300# rule set cluster:port:rule affinity clientip

The default for rule affinity is the client IP, so if you have never set it to cookie you do not need to set it to clientip.

5.8 LoggingNetwork Dispatcher provides several logging methods. It writes information to text log files by default. You can also use a binary logging feature to analyze server statistics.

We discuss both methods in the next sections.

5.8.1 Basic loggingSeveral components of Network Dispatcher generate logs. These logs are recorded in the following path for each platform:

AIX: /usr/lpp/nd/<component>/logs

Sun: /opt/nd/<component>/logs

Linux: \Program Files\ibm\nd\<component>\logs

NT/2000: /opt/nd/<component>/logs

Where <component> can be one of the following: Dispatcher, CBR or ISS.

You can change the directory where the log files are created. In AIX, Linux and Solaris, you can edit the /usr/bin/ndserver script. Locate the ND_LOGDIR variable, and set it to the new path. On Windows platforms, you can edit the C:\WINNT\SYSTEM32\ndserver.cmd script and look for the same variable.

For all operating systems, make sure that there are no spaces on either side of the equal sign and that the path ends in a slash (see Example 5-27).

Example 5-27 The ND_LOGDIR variableFor UNIX platforms:ND_LOGDIR=/home/mylogdir/

For Windows platforms:set ND_LOGDIR=C:\MyLogDir\

The following list shows the common log files that are created in the directories above:

manager.log logs the Manager activities

Http_80.log logs the HTTP advisor activities

Telnet_23.log logs the Telnet advisor activities

server.log logs the ndserver activities

hamon.log logs the high availability monitor activities (high availability scripts execution - refer to “Creating the high availability scripts” on page 120)

Information can be stored in the log files at different levels:

� Level 0: logs only errors.

� Level 1 (the default): logs errors, headers and records of events that happen only once.

� Levels 2 through 4: log progressively more information, but are not always used.

� Level 5: the maximum verbosity level.

You can change the verbosity of the logs by setting the loglevel.

# ndcontrol set loglevel 5# ndcontrol manager set loglevel 3# ndcontrol manager reach set loglevel 2# ndcontrol advisor loglevel <advisor> <port> 4

You can use the same options for CBR, but you need to use cbrcontrol instead of ndcontrol.

5.8.2 Binary loggingThe Dispatcher and CBR components of Network Dispatcher provide an optional binary logging facility. The purpose of this facility is to allow you to analyze server usage and trends. The Manager component must be running in order for information to be entered into the binary logs.

Note: Logging consumes resources and disk space. We recommend that you use the default values, unless you need to troubleshoot a problem.

Starting the logging facilityWe enabled the logging facility with the Network Dispatcher GUI for a small cluster environment that we used in 4.1, “Load balancing” on page 70. To start capturing data to the log files, we right-clicked Manager in the left pane of the window and from the pop-up menu selected Start Logging... as shown in Figure 5-45.

Figure 5-45 Starting the binary logging facility

You can also enable logging by running the following command:

# ndcontrol log start

Several other log commands are also available:

# ndcontrol log stop# ndcontrol log status


# ndcontrol log set interval seconds# ndcontrol log set retention hours

Statistics will be placed in the logs at a user-specified interval. The interval can be changed via the GUI or the command line.

Examining the log filesWhen the logging facility is turned on, one log is created at the start of every hour with the date and time as the name of the file. The logs are placed in the <install path>/<component>/logs directory, where <install path> varies by operating system and <component> can be Dispatcher, ISS or CBR.

Using the LOG_SampleReader sample Java programThe data is recorded in the log in binary format; therefore, users are not able to read the raw log files directly. For this reason, a sample program is provided to be used primarily as a reference for you when writing your own program to read the binary logs. It can also be used to extract one type of log entry. When the Dispatcher is installed, the sample program, called LOG_SampleReader.java, is placed in the location <install path>/lib/BinaryLog.

In this section, we explain how the LOG_SampleReader sample Java program extracts the data from the binary logs. We also show how to invoke LOG_SampleReader and interpret its output.

� Log Data Format

The installbase/lib/ibmnd.jar file for the Dispatcher component and the installbase/lib/ibmcbr.jar file for the CBR component provide the utilities (objects and their associated methods) that LOG_SampleReader uses to read the data from the log. The comments embedded in LOG_SampleReader.java inform us that the following types of records can be retrieved from the log:

– LOG_TimestampRecord– LOG_ExecutorIDRecord– LOG_ClusterIDRecord– LOG_PortIDRecord– LOG_ServerReportRecord

Each of these types has associated with it one or more methods which can be used to determine its value from the LOG_Record object. For example, the LOG_ServerReportRecord implements the following eight methods:

c. getIPAddress() returns the server IP address.d. getWeight() returns the server weight of the server when the record was

written.e. getTotalConnections() returns the total number of server connections

when the record was written.

f. getActiveConnections() returns the number of active connections.g. getPortLoad() returns the port load of the server when the record was

written.h. getHasPortLoad() returns a Boolean value indicating whether or not port

loads were being provided for the server when the record was written to the binary log (port loads are obtained through an advisor).

i. getSystemLoad() returns the system load of the server when the record was written.

j. getHasSystemLoad() returns a Boolean value indicating whether or not system loads were being provided for the server when the record was written to the binary log (system loads can be provided, for example, using ISS).

LOG_SampleReader.java extracts ServerReport data from the logs by using the LOG_Reader and LOG_Record objects and their associated methods. These methods scan through the log records searching for a record of type LOG_ServerReportRecord. When one is found, it is printed.

� Invoking the Sample Program

The Dispatcher and CBR components <install path>/<component>/samples/BinaryLog directories each contain a small shell script that serves as a wrapper for LOG_SampleReader. It sets up the required CLASSPATH elements and location of the log directory before invoking LOG_SampleReader through the java command.

Example 5-28 shows the Dispatcher wrapper file, ndlogreport.cmd, from our Windows NT system.

Example 5-28 ndlogreport.cmd script on Windows NT@echo offrem Set the NDBLOGDIR variable below to the directory that contains rem your log reader.set NDBLOGDIR=%IBMNDPATH%\dispatcher\samples\BinaryLogset NDCLASSPATH=%NDBLOGDIR%;%IBMNDPATH%\dispatcher\lib\ibmnd.jarjava -DEND_LOG_DIRECTORY=%IBMNDPATH%\dispatcher\logs -cp %NDCLASSPATH% LOG_SampleReader %1 %2 %3 %4

Example 5-29 shows the ndlogreport script from our AIX system, which is similar to the one provided in Linux and Solaris.

Example 5-29 ndlogreport shell script on AIX#!/bin/ksh# Set the NDBLOGDIR variable below to the directory that contains# your log reader.

NDBLOGDIR=/usr/lpp/nd/dispatcher/samples/BinaryLog

export CLASSPATH=\$NDBLOGDIR:\/usr/lpp/nd/dispatcher/lib/ibmnd.jarjava -DEND_LOG_DIRECTORY=/usr/lpp/nd/dispatcher/logs LOG_SampleReader $1 $2 $3 $4

We invoked this script with four arguments representing the start date and time and end date and time of the server records we wanted to see from the log. We used the following command in a Dispatcher running on Windows NT:

Example 5-30 Generating a reportC:\> cd program files\ibm\nd\dispatcher\samples\BinaryLogC:\> ndlogreport.cmd 2001/04/03 13:00 2001/04/03 20:00

� Interpreting LOG_SampleReader Output

The output from the above command is shown in Example 5-31:

Example 5-31 Output from ndlogreport command2001/04/03-18:43:07.879,9.24.105.82,9.24.105.87,80,9.24.105.39,9,18,0,541,true,0,false2001/04/03-18:43:07.879,9.24.105.82,9.24.105.87,80,9.24.105.59,10,18,0,130,true,0,false2001/04/03-18:44:08.316,9.24.105.82,9.24.105.87,80,9.24.105.39,9,26,0,520,true,0,false2001/04/03-18:44:08.316,9.24.105.82,9.24.105.87,80,9.24.105.59,10,26,0,131,true,0,false2001/04/03-18:45:09.944,9.24.105.82,9.24.105.87,80,9.24.105.39,9,33,0,521,true,0,false2001/04/03-18:45:09.944,9.24.105.82,9.24.105.87,80,9.24.105.59,9,33,0,130,true,0,false2001/04/03-18:45:09.944,9.24.105.82,9.24.105.87,80,9.24.105.20,10,5,0,130,true,0,false2001/04/03-18:46:10.391,9.24.105.82,9.24.105.87,80,9.24.105.39,10,63,0,511,true,0,false2001/04/03-18:46:10.391,9.24.105.82,9.24.105.87,80,9.24.105.59,10,62,0,130,true,0,false2001/04/03-18:46:10.391,9.24.105.82,9.24.105.87,80,9.24.105.20,0,22,0,130,true,0,false

There are 13 fields on each line:

a. date

b. time

c. Dispatcher nonforwarding address

d. cluster IP address

e. port number

f. server IP address


g. server weight

h. total connections

i. active connections

j. port load of the server

k. whether the Dispatcher is receiving port load information

l. server load

m. whether the Dispatcher is receiving server load information

We can determine from the above output that at the start of the period, there were only two servers for the cluster, 9.24.105.39 and 9.24.105.59. Later on, another server was added to this cluster: 9.24.105.20. We can also determine that all servers were responding to the advisor (see the true value for field k) and they were not using ISS (see the false value for field m).

This program can be modified to process the binary log information for any kind of analysis that you require.


Part 3 Web Traffic Express

In this part, we describe the Web Traffic Express component of WebSphere Edge Server.

The Web Traffic Express component of WebSphere Edge Server for Multiplatforms is also referred to in many publications as the caching proxy.

Part 3


Chapter 6. Web Traffic Express installation

This chapter covers the installation steps for Web Traffic Express in the following environments:

� AIX: see 6.1, “Installing Web Traffic Express on AIX” on page 248.

� Windows NT: see 6.2, “Installing Web Traffic Express on Windows NT” on page 262.

� Linux: see 6.3, “Installing Web Traffic Express on Linux” on page 274.

Procedures for starting and stopping services and for uninstallation are also covered.

6


6.1 Installing Web Traffic Express on AIXIn this section, we describe the hardware and software requirements for Web Traffic Express, the installation process, and how to start and stop Web Traffic Express on an AIX platform.

6.1.1 System hardware and software requirements� Any IBM RS/6000-based machine

� AIX Version 4.3.2 or higher (32-bit only)

� Asynchronous I/O enabled on the WTE machine. To verify that the required bos.rte.aio fileset is installed, issue the following command:

$ lslpp -l bos.rte.aio

To verify that asynchronous I/O is enabled, issue the following command:

# smit chgaio

Set the state to available in the STATE to be configured at system restart field.

� If running WTE in a non-English language environment, edit the /etc/environment file, setting the LC__FASTMSG environment variable to the value false.

� 50 MB of available disk space for installation and documentation, plus additional space for logs.

If you do not create a separate file system for WTE and you plan to implement filtering using CyberPatrol (refer to 8.1.4, “Blocking URLs using the CyberPatrol plug-in” on page 336), you may need to increase your root filesystem by 30 MB.

� Any communication hardware adapter configured to use the TCP/IP stack for making network connections.

Note: You must have super user root as the local machine to configure asynchronous I/O. Note that changes to this setting will not be effective until the system has been restarted.

Note: We suggest you create a separate filesystem called /opt, to install WTE.


� A minimum of 64 MB RAM, though greater amounts yield better performance. The following functions require an amount of RAM over and above the amount required for basic WTE operation:

– Caching of any type. WTE uses additional RAM as overhead for internal record keeping about each cache device. For cache devices smaller than 1 GB, the requirement for each device is 8.5 MB plus about 1% of the cache size on the device. For devices 1 GB and larger, the additional RAM requirement is 1% of the cache size on the device.

– Memory caching. The appropriate cache size (amount of additional RAM required) depends on the size and number of files that users retrieve from Web servers. Larger caches generally have higher cache hit rates. A suggested minimum is 17 MB for reverse proxy, and the greater of 17 MB or 2 MB per user for forward proxy. If transparent proxy is supported, the suggested minimum is the same as for forward proxy.

� Free disk space for paging: a minimum of twice the amount of RAM is required.

� Free disk space for disk or file caching: this depends on the size and number of files that users retrieve from Web servers. Larger caches generally have higher cache hit rates. Suggested minimum values are the same as described previously for memory caching.

� Netscape V4.7 or later in order to configure WTE using the Configuration and Administration forms.

� Content Based Routing (CBR) calls for the JRE (Java Runtime Environment) required by Network Dispatcher and the ND CBR component.

Note: The fact that overhead is required for each device implies that it is more efficient to use fewer large devices than many small ones.

Note: For WTE to cache to disk or to a file, you must format the disk or file by using the htcformat command as described in Chapter 7, “Web Traffic Express basic configuration” on page 287.

Note: Netscape Version 6 is not supported for use with Web Traffic Express Configuration and Administration forms.

Chapter 6. Web Traffic Express installation 249

6.1.2 InstallationThis section describes the steps to follow to install Web Traffic Express on the AIX platform.

Before starting the installation process, you must choose which type of proxy your configuration calls for. The following are the different types of proxies that can be chosen during the installation process (refer to Figure 6-7 on page 256):

� Forward proxy:

Intercepts HTTP requests generated by Web browser clients that are configured to use it as the proxy. Figure 6-1 illustrates this configuration.

Figure 6-1 Web Traffic Express acting as a forward proxy

WTEForward Proxy

Server

WEB Clients

Router

Content Server

Internet


� Reverse proxy:

Intercepts HTTP requests destined for your content host. The following screen illustrates this scenario:

Figure 6-2 Web Traffic Express acting as a reverse proxy

� Transparent proxy:

Intercepts HTTP requests generated by Web browser clients that are not configured to use it as a proxy. This is possible because the router is configured to redirect all requests destined for port 80 to the proxy server. Figure 6-3 illustrates this scenario:

Figure 6-3 Web Traffic Express acting as a transparent proxy

WTEReverse Proxy

Server

WEB Clients

Content Server

RouterInternet

WTETransparent Proxy

Server

WEB Clients Content Server

InternetRouter


Once you have chosen the type of proxy for your configuration, you can proceed with the installation process.

Our environment consists of an IBM RISC/6000 43P Model 150 with 768 MB RAM, two 9.1 GB hard disks, and one token-ring adapter. It is running AIX 4.3.3.0.

1. Log in as the root user:

% su root

Password: root_password

2. Create a directory on which to mount the CD-ROM, if it does not already exist:

# mkdir /cdrom

3. Insert the WebSphere Edge Server media in the CD-ROM drive.

4. Mount the CD-ROM:

# mount -v cdrfs -p -r /dev/cd0 /cdrom# cd /cdrom

5) Start the InstallShield Wizard:

# ./setup


In the first window, you have to choose a setup language (see Figure 6-4).

Figure 6-4 Setup language

Click the Next button to continue installation.

The next window is the Software License Agreement. After you read and accept this license, click the Accept button to continue.


WebSphere Edge Server consists of two components: Network Dispatcher and Web Traffic Express. In the next window (shown in Figure 6-5), select which component you want to install.

Figure 6-5 Choosing the WebSphere Edge Server component

Select Web Traffic Express, which is the caching proxy component, and click the Next button.


The following window contains descriptions of the different types of configurations supported by Web Traffic Express.

Figure 6-6 Information on the types of proxy configurations

After reading this information and clicking Next, you are presented with a window where you can choose how the proxy server will function.


Figure 6-7 Selecting the function of the proxy server.

For a description of each type of proxy, refer to 6.1.2, “Installation” on page 250. We selected the Forward Proxy option. Click Next.


This brings you to the port selection window (see Figure 6-8). Here, you specify the port number on which Web Traffic Express will listen for requests from the clients. The default port is 80. You can change this to any available port.

Figure 6-8 Choosing the port number


You can customize the Web Traffic Express component using the browser-based Configuration and Administration forms. In the next window (Figure 6-9), you define the user ID and password which the WTE administrator will use to access this feature.

Figure 6-9 Defining user ID and password

Type the password and click Next.


The InstallShield displays a summary of all selections and asks you to change them or proceed. This is the last chance to change anything before the installation of the files.

Figure 6-10 Confirming before copying files

After you confirm the installation options, InstallShield will copy all files to the /opt/IBMWTE directory.


Figure 6-11 Installation Summary window

The last window of the installation process shows the path of the readme file. This file contains the latest information about any known problems.

6.1.3 Starting and stopping Web Traffic ExpressOn AIX, Web Traffic Express is a process called ibmproxy. The InstallShield Wizard starts the ibmproxy process as the final step of the installation with the default configuration settings. It also creates a Web Traffic Express initialization file in the /etc directory and includes it in the machine’s startup sequence.

To stop this process, enter the following command:

# stopsrc -s ibmproxy

Note: Web Traffic Express installation creates two log files in the /opt/IBMWTE directory. One log file, called trace.log, contains all the steps performed in the installation process. Another file, called install.log, lists all filesets installed.


To start this process, enter:

# startsrc -s ibmproxy

6.1.4 UninstallationTo uninstall Web Traffic Express, follow these steps:


% su root


2. Stop the Web Traffic Express process as described in 6.1.3, “Starting and stopping Web Traffic Express” on page 260.

3. Create a directory on which to mount the CD-ROM, if it does not already exist.

# mkdir /cdrom


5. Mount the CD-ROM.

# mount -v cdrfs -p -r /dev/cd0 /cdrom# cd /cdrom

6. Issue the setup command with the following option:

# ./setup -wteuninstall

You can also uninstall using SMIT. The following filesets must be deleted:

� ibmwte.base.exe 3.6.0.0

� ibmwte.base.ssl 3.6.0.0

� ibmwte.doc.en_US 3.6.0.0

� ibmwte.msg.en_US.base 3.6.0.0

Note: When Web Traffic Express starts, a file called ibmproxy-pid is created in the /opt/IBMWTE/usr/internet/server-root directory. This file contains the process ID (PID).

Note: Web Traffic Express uninstallation creates one log trace file called unintsall.log on /opt/IBMWTE. This file contains all the steps performed in the uninstallation process.


6.2 Installing Web Traffic Express on Windows NTIn this section, we describe the installation requirements and how to start and stop Web Traffic Express on a Windows NT platform.

6.2.1 System hardware and software requirements� Any Intel x86 PC, including SMP models supported by Windows 2000 or

Windows NT.

� Windows NT 4.0 with Service Pack 5 or higher, or Windows 2000.

� 50 MB of available disk space for installation and documentation, plus additional space for logs.





� Free disk space for paging: a minimum of twice the amount of RAM is required.




� Netscape V4.72 or later, or Microsoft IE V5.0 or later is required to configure WTE using the Configuration and Administration forms.

� Content Based Routing (CBR) demands the JRE (Java Runtime Environment) required by Network Dispatcher and the ND CBR component.

6.2.2 InstallationThis section describes the steps to follow to install Web Traffic Express on the Windows NT platform.

Our environment is an IBM PC 300PL with 256 MB RAM, 1.5 GB and 600 MB hard disk, and one Token Ring adapter. It is running Windows NT 4.0 with Service Pack 5.

1. Insert the WebSphere Edge Server media into the CD-ROM drive.

2. Click Start-> Run...-> Browse and choose setup.exe on the CD-ROM drive as shown in Figure 6-12.

Figure 6-12 Installation program name and path

Notice that D is the drive letter assigned to the CD-ROM drive. Click OK to start the installation.


Note: Netscape Version 6 is not supported for use with Web Traffic Express Configuration and Administration forms.


The first window shows the Software License Agreement. After you read and accept this license, click Accept to continue the installation.

In the next window, choose a setup language, as shown in Figure 6-13. Click Next.

Figure 6-13 Choosing a setup language


The InstallShield Wizard lists the WebSphere Edge Server Window components (refer to Figure 6-14).

WebSphere Edge Server consists of two components: Network Dispatcher and Web Traffic Express. In this window, you select which component you want to install.


Select the Web Traffic Express option, which is the caching proxy component. Click Next.


InstallShield then prompts you to choose an installation directory. The default location is C:\ProgramFiles\IBM as shown in Figure 6-15.

Figure 6-15 Choosing the destination location

Click Next to proceed with the installation.


The next window gives a description of the different types of configurations that are supported with Web Traffic Express (see figure below).

Figure 6-16 Information about types of proxy





Important: The Transparent proxy configuration type is not available in the Windows environment.


This brings you to the port selection window (see Figure 6-18). Here, you specify the port number on which Web Traffic Express will listen for requests from the clients. The default port is 80. You can change this to any available port.



You can customize the Web Traffic Express component using the browser-based Configuration and Administration forms. In the next window (Figure 6-19), you define the user ID and password used by the WTE administrator to access this feature.

Figure 6-19 Defining user ID and password to access configuration and administration forms

Type the password and click Next.


The InstallShield displays a summary of all selections and asks you to change them or proceed. This is the last chance to change anything before the installation of files.


After you confirm the installation options, the InstallShield Wizard will copy all files to the destination location. When installation is complete, the InstallShield will display a window asking you to reboot the machine.



The last window of the installation process shows the path of the readme file.

6.2.3 Starting and stopping Web Traffic ExpressWeb Traffic Express is added as an NT Service called IBM Web Traffic Express. It is automatically started during the boot process. You can stop the service by selecting it and clicking Stop (see below).

Note: Web Traffic Express installation creates one log trace file called trace.log on C:\ProgramFiles\IBM\WTE. This file contains all the steps performed in the installation process.


Figure 6-22 IBM Web Traffic Express Service

6.2.4 UninstallationThe Web Traffic Express program group contains readme icons, Uninstall IBM Web Traffic Express and Key Management Utility.

To uninstall, click Start -> Programs -> IBM WebSphere -> Edge Server -> Web Traffic Express -> Uninstall IBM Web Traffic Express.

The window shown on Figure 6-23 is displayed.

Figure 6-23 Window to confirm the uninstallation


The following window displays the progress of the WTE uninstallation.

Figure 6-24 Uninstallation progress

If you click the Details... button you can see that the log files were not deleted. Also notice that the configuration files in the c:\Program Files\IBM\WTE directory, as well as some registry entries, were not deleted.

If you would like to delete these registry entries, look for the following:

� EventMessage File

� Library

� NLSPath

� ImagePath

6.3 Installing Web Traffic Express on LinuxIn this section, we describe the installation requirements and how to start and stop Web Traffic Express on a Linux platform.


6.3.1 System hardware and software requirements� Any Intel x86 PC including SMP models supported by one of the following

Linux releases:

– Red Hat Linux 6.1 (kernel Version 2.2.12-20)– Red Hat Linux 6.2 (kernel Version 2.2.16-3)– SuSE Linux 6.4 (kernel Version 2.2.14)

� Free disk space for software and documentation: 50 MB, plus additional space for logs.





� Free disk space for paging: a minimum of twice the amount of RAM.





6.3.2 Installation This section describes the steps to follow to install Web Traffic Express on the Linux NT platform.

Our installation environment is a NetFinity 3000 with 256 MB RAM, a 9.1GB hard disk, and one Token Ring adapter. It is running Red Hat 6.2.


% su root


2. Create a directory in which to mount the CD-ROM, if it does not already exists.

# mkdir /mnt/cdrom



# mount /dev/cdrom /mnt/cdrom

5. If you have Linux autorun feature enabled on the machine, the InstallShield wizard starts automatically, as shown in Figure 6-25.

Figure 6-25 Autorun feature

Click Yes to continue the installation.

If you do not have the Linux autorun feature enabled, issue the following commands:

# cd /mnt/cdrom# ./setup


In the first window, you choose a setup language.

Figure 6-26 Choosing a setup language

Click Next to continue the installation.


The next window shows the Software License Agreement. After you read and accept this license, click Accept to continue the installation.

The WebSphere Edge Server has two components: Network Dispatcher and Web Traffic Express. In the next window, you select which component you want to install.


Select the Web Traffic Express option, which is the caching proxy component, and click Next.


The next window gives a description of the different types of configurations that are supported with Web Traffic Express.

Figure 6-28 Information about type of proxy



This brings you to the window shown in Figure 6-30. Here, you can specify the port number on which Web Traffic Express will listen for requests from the clients. The default port is 80. You can change this to any available port.



You can customize the Web Traffic Express component using the browser-based Configuration and Administration forms. In the next window (Figure 6-31), you define the user ID and password to be used by the WTE administrator to access this feature.

Figure 6-31 Defining user ID and password to access configuration and administration forms

Enter a password and click Next.


The InstallShield displays a summary of all selections and ask you to change them or proceed. This is the last chance to change anything before the installation of files.


After you confirm the installation options, the InstallShield will copy all files to the /opt/IBMWTE directory.



The last window shows the location of the readme file. This file contains the latest information about any known problems.

6.3.3 Starting and stopping Web Traffic ExpressOn Linux, Web Traffic Express runs as a process called ibmproxy. The InstallShield Wizard starts the ibmproxy process as the final step of the installation. It also creates a Web Traffic Express initialization file in the /etc/rc.d/init.d directory and includes it in the machine’s startup sequence.

Note: The Web Traffic Express installation creates one log trace file called trace.log on /opt/IBMWTE. This file contains all the steps followed in the installation process.

Important: Linux includes an HTTP server that binds to port 80. Since WTE use the same port, you need to stop the HTTP server before starting WTE.


To stop the WTE process, use the following command:

# /etc/rc.d/init.d/ibmproxy stop

To start the WTE process, use this command:

# /etc/rc.d/init.d/ibmproxy start

6.3.4 UninstallationTo uninstall Web Traffic Express, follow these steps:


% su root


2. Stop the Web Traffic Express process as described in 6.3.3, “Starting and stopping Web Traffic Express” on page 284.

3. Create a directory on which to mount the CD-ROM, if it does not already exist:

# mkdir /mnt/cdrom

4. Insert the WebSphere Edge Server in the CD-ROM drive.


# mount /dev/cdrom /mnt/cdrom# cd /mnt/cdrom

6. If you have Linux’s autorun feature enabled on the machine, the InstallShield Wizard starts automatically. Click Cancel.

7. Issue the setup command with the following option:

# ./setup -wteuninstall

Note: When the Web Traffic Express starts, a file called ibmproxy-pid is created in the /opt/IBMWTE/usr/internet/server-root directory. This file contains the process ID (PID).

Note: The Web Traffic Express uninstallation creates one log trace file called uninstall.log on /opt/IBMWTE. This file contains all the steps followed in the uninstallation process.


Chapter 7. Web Traffic Express basic configuration

In this chapter, we discuss the basic configuration and administration of WTE using both the command line interface and the Configuration and Administration forms browser interface.

7


7.1 Basic configurationWeb Traffic Express can be configured and customized using the browser-based Configuration and Administration forms, or by editing the Web Traffic Express configuration file ibmproxy.conf. Both of these configuration methods will be discussed in this chapter.

7.1.1 Configuration environmentIn our test scenario environment, we have three machines with Web Traffic Express installed. Table 7-1 and Figure 7-1 describe the specifics of our network configuration. This configuration will be used for all the following WTE chapters.

Table 7-1 Description of all machines

All machines are on the same network and in the itso.ral.ibm.com domain.

Hostname IP Address Operating System Service

rs600034 9.24.105.61 AIX 4.3.3 Caching Proxy Server

m238P4xl 9.24.105.83 Windows NT 4.0 Caching Proxy Server

rhlinux1 9.24.105.63 Linux Red Hat 6.2 Caching Proxy Server

m23kk904 9.24.105.39 Windows NT 4.0 Web Client

m238p4yl 9.24.105.82 Windows NT 4.0 Web Client

rs60008 9.24.105.59 AIX 4.3.3 Web Server


Figure 7-1 Test scenario

7.1.2 Using the Configuration and Administration formsThe Configuration and Administration forms GUI is a common way to set up and maintain the ibmproxy.conf configuration file. It presents a graphical interface using your Web browser. To access the administration forms, Web Traffic Express must be running on the server (for more information on starting and stopping WTE, refer to the corresponding platform in Chapter 6, “Web Traffic Express installation” on page 247). You need only a Web browser.

To access these HTML forms, type either of the following URLs:

� http://your.server.name:port

� http://your.server.name/Frntpage.html:port

In our case, we used the following URL: http://rs600034. By default, the port is port 80. We did not change this in our installation.

WTE Proxy Servers

9.24.105.61RS600034

9.24.105.83M238P4XL

9.24.105.63RHLINUX1

WEB Clients

9.24.105.39M23KK904

9.24.105.82M238P4YL

9.24.105.59RS60008

WEB Server

Chapter 7. Web Traffic Express basic configuration 289

Figure 7-2 shows the default front page supplied with Web Traffic Express.

Figure 7-2 Web Traffic Express front page

With this front page, you can view the documentation, visit the product support Web site or access the Configuration and Administration forms.

To access the Administration forms, you must have a user ID and password. Both of these can be defined during production installation. Refer to Figure 6-9 on page 258 to see where the user ID and password are initially defined for our AIX installation.


7.1.3 Web Traffic Express Administration settingsOnly authorized users can configure the ibmproxy.conf file via the Configuration and Administration forms interface. This section describes how to define another user ID and password to administer or modify Web Traffic Express. You can use the command line interface or the Configuration and Administration forms interface to define a different administrator user name and password. We show how to add a user with the Configuration and Administration forms in 8.1.2, “Basic user authentication” on page 320. In this section, we will use the command line interface.

The WTE command to add an authorized user is htadm. If you input this command with no parameters, a short description of how to use the command is displayed, as shown on the following example.

Example 7-1 htadm syntax# htadmAdministrative tool for Server access authorizationUsage: -adduser pwfile [user [password [real name]]] -deluser pwfile [user] -passwd pwfile [user [password]] -check pwfile [user [password]] -create pwfile

Setting administrator user ID and password on AIXOn AIX systems, WTE administrator user names and passwords are stored in the WTE password file. The default file is webadmin.passwd. This password file is located in the /opt/IBMWTE/usr/internet/server_root/protect directory. To define a different administrator user name and password for use with the Configuration and Administration forms from a Web browser, use the htadm command as follows:

htadm -adduser password_file user-name password real-name

The htadm executable file is automatically installed in the /usr/sbin directory. However, for password_file, you must specify the full path or execute htadm in the directory where the password file resides.

Note: The proxy server maintains its own list of user names, passwords, and group names.


The real-name variable is not required. It is a comment or a name you want to use to identify the user name. WTE writes this variable to the password file, but does not do anything with it. If you do not specify the real-name variable, the system will prompt you for it, at which time you may press Enter to signify no identity, or enter a real user name.

We entered the following commands:

# cd opt/IBMWTE/usr/internet/server_root/protect# htadm -adduser webadmin.passwd wteadmin admin "WTE Administrator"

We entered wteadmin as the user name and admin as the password. We now have two administrators defined to WTE. The password is written into the password file, webadmin.passwd. This file on our system contained the following two lines after entering the above command:

admin:bUckfUvSwcgZ.:administratorwteadmin:FGVS7ptGfgzOE:WTE Administrator

The password is encrypted in the password file. Interestingly, WTE encrypts each new password with a different key. You can see this by defining two users with the same password.

Setting administrator user ID and password on Windows NTOn Windows NT systems, user names and passwords are stored in the caching proxy server default password file, which is admin.pwd. This file is located in the Windows NT installation directory, which by default is c:\Program Files\IBM\WTE. To define a different administrator user name and password for use with the Configuration and Administration forms from a Web browser, use the htadm command as follows:

htadm -adduser password_file user-name password real-name

The htadm.exe is installed in the C:\Program Files\IBM\WTE\bin directory. This path is automatically added in the path system environment variable when WTE is installed. However, for password_file, you must specify the full path or execute htadm in the directory where the password file resides.

The real-name variable is not required. It is a comment or a name you want to use to identify the user name. WTE writes this variable to the password file, but does not do anything with it. If you don’t specify the real-name variable, the system will prompt you for it, at which time you may press Enter to signify no identity, or enter a real user name.

We entered the following commands:

cd c:\Program Files\ibm\wtehtadm -adduser admin.pwd wteadmin admin "WTE Administrator"


We entered wteadmin as the user name and admin as the password. We now have two administrators defined to WTE. The password that you enter is written into the password file, admin.pwd. This file on our system contained the following two lines after entering the above command:

admin:bFnr3RsF76DFg:administratorwteadmin:UmXJL9v2p5tpI:WTE Administrator

The password is encrypted in the password file. Interestingly, WTE encrypts each new password with a different key. You can see this by defining two users with the same password.

Setting administrator user ID and password on LinuxDefining a new administrator user ID and password in the Linux environment is very similar to the same process in the AIX environment. See “Setting administrator user ID and password on AIX” on page 291.

7.1.4 Configuring the basic settingsClick the CONFIGURATION AND ADMINISTRATION FORMS option on the home page (refer to Figure 7-2). The window shown in Figure 7-3 is displayed. You will be asked to authenticate; enter a username and the password of the administrator.

Figure 7-3 Entering Web Traffic Express administrator and password

We entered admin in the User Name field and admin in the Password field, and clicked OK. If the authentication is validated, the window shown in Figure 7-4 is displayed.


Figure 7-4 Web Traffic Express Introduction window

Almost all customization is performed using the groups listed in the left pane. For the changes to take effect, you must restart the WTE server by clicking the Restart Server button. This restarts the ibmproxy daemon which will re-read the ibmproxy.conf file.


All updates made using Configuration and Administration forms are stored in ibmproxy.conf. This file contains directives that define the functionality of the WTE server.

Table 7-2 Location of the ibmproxy.conf file

Some directives are not refreshed with the Restart Server button. You must stop the server and restart it as described in Chapter 6, “Web Traffic Express installation” on page 247. All directives that are not refreshed are listed in Table 7-3.

Table 7-3 Directives not refreshed upon restart

In the following sections, we describe configuration options using the groups available in the Configuration and Administration forms.

Operating System Location

Windows NT 4.0 c:\program files\ibm\wte

AIX 4.3.3 /etc (the file in this directory is a link to /opt/IBMWTE/usr/internet/etc)

Linux Red Hat 6.2 /etc (the file in this directory is a link to /opt/IBMWTE/usr/internet/etc)

Note about Netscape 4.x browsers: If the Netscape browser’s window is resized, the screen clears and a message is displayed informing the user that the browser does not support JavaScript. In order to properly redisplay the form, click the browser’s Reload button (Netscape 6 not supported).

Task Group Directives

CGI DisinheritEnv, InheritEnv

Caching Caching

Logging AcessLog, CacheAccessLog, ErrorLog, ProxyAccessLog, ServerRoot

Network Access BindSpecifc, Hostname, ListenBacklog, Port

Performace MaxActiveThreads

SNMP SNMP, SMNPCommunity, WebMasterEmail

UNIX Process Control GroupID, UserID

Miscellaneous TransparentProxy


Configuration host nameIn a Windows NT installation, this directive is already defined. However, on the AIX and Linux platforms, you must configure it. Choose Server Configuration -> Basic Settings. The following window is displayed.

Figure 7-5 Basic Settings window

We input rs600034.itso.ral.ibm.com in the Host name field. Click Submit (not shown on Figure 7-5) to update the configuration file, ibmproxy.conf.


The default of the Process ID File Location field is <WTE-root>/ibmproxy-pid. However, if you do not fill in this field, Configuration and Administration forms will generate an empty PidFile directive at the end of the ibmproxy.conf file. The same will happen with the InheritEnv and DisInheritEnv fields. After restarting the proxy server, the following error messages will be generated in the error log file:

Example 7-2 error.Apr042001[04/Apr/2001:09:01:33 +0500] [OK] [host: ] Insufficient parameters for directive: PidFile[04/Apr/2001:09:01:33 +0500] [OK] [host: ] Insufficient parameters for directive: InheritEnv[04/Apr/2001:09:01:33 +0500] [OK] [host: ] Insufficient parameters for directive: DisInheritEnv

We recommend that you edit the ibmproxy.conf file and uncomment the lines of the directives shown in Example 7-3. The default values will then be used and the errors will not be logged.

Example 7-3 iibmproxy.conf...# PidFile# InheritEnv# DisInheritEnv

For the changes to take effect, you must restart the WTE server by clicking the Restart button.

Enable proxy functionTo configure this option, choose Proxy Configuration -> Proxy Settings. The following window will be displayed.

Figure 7-6 Proxy Settings window


WTE is a proxy server when installed. The default setting automatically provides the proxy function for the following protocols:

� HTTP: process HTTP requests

� FTP: process FTP requests

� Gopher: process gopher requests

� SSL Tunneling: use any port to pass encrypted data, unaltered, between a client and a content server

When the proxy returns dynamic data, such as output data from CGI programs, API programs, server-side includes, and Java servlets, it must buffer the data. You set the value of the buffer size in the Proxy buffer size field. We accepted the default value, which is 100 KB.

The next field in the Proxy Settings form is the Proxy access log. This field allows you to specify the file name where WTE will log access requests that are proxied on this server. Refer to Figure 7-14 on page 312 to see how this information may be viewed. The fully qualified path is required. Each day at midnight, the WTE server, if it is running, starts a new log file. It uses the specified file name and appends a date suffix as an extension. Table 7-4 shows the default values for this field.

Table 7-4 Default proxy access log location

We modified the value of the Proxy access log field and set it to /wte/logs/proxy. The /wte file system and /log directory were created earlier.

Permissions are important when changing the default directory where the log files will reside. The WTE server will write to that directory as the server’s user ID and group ID specified in the ibmproxy.conf configuration file (nobody/nobody by default; refer to Figure 7-5 on page 296). Therefore, if you have created a new directory for the log files, you must ensure that the WTE server’s user ID can write to that directory.

Platform Default path and file

AIX /opt/IBMWTE/usr/internet/server_root/logs/proxy

Windows NT \Program Files\IBM\WTE\logs\proxy

Linux /opt/IBMWTE/usr/internet/server_root/logs/proxy

Note: This log file can take up a significant amount of space on your file system. It is recommended that a different file system be created to hold the log files.


For the configuration changes to be written in the configuration files, click Submit. You will receive a reply reporting that the requested configuration changes have been completed successfully. This directive requires you to stop and start the server, as shown in Table 7-3 on page 295.

Enable pure proxy functionThe WTE server’s function is to be a caching and filtering proxy server. In theory, WTE could even work as a content server. However, for best performance, it is recommended that you use it only as a proxy server (no caching). In fact, by default, the initial settings instruct this component to act as a pure proxy server. You can see this if you select the Proxy Performance form from the navigation menu, as shown in Figure 7-7:

Figure 7-7 Performance settings window


We accepted the default configuration, so we did not need to restart the server for these settings.

Configuring cache functionCaching is done by locally saving copies of the files that clients request. These files may be saved in memory, to a raw disk partition, or to a disk file. WTE uses the cached files, when they are subsequently requested, to serve its clients quickly because it does not need to fetch the files from a remote Web server again. To enable the caching capabilities of WTE, you must define where the requested files will be cached. By default, the cache is defined in memory. In addition, the system memory is used to hold a cache index which reduces processing time for finding cached files. You configure the amount of memory used for the cache index. If memory caching is also configured, WTE manages both the index and the cache in the configured amount of memory.

You may also configure WTE to maintain the cache in a raw disk device (unformatted partition), or in an OS file. These devices or files must be prepared using the custom formatting command htcformat.

How to use htcformatOn the AIX and Linux platforms, the htcformat executable file is automatically installed in the /usr/sbin directory. On the Windows platform, the htcformat.exe file is installed in the C:\Program Files\IBM\WTE\bin directory. This path is automatically added to the path system environment variable when WTE is installed.

If you input this command in the command line, a short description of how to use it is displayed.

Example 7-4 htcformat syntaxhtcformatUsage: htcformat <devpath> [-blocksize <block size>] [-blocks <number blocks>] [-help] htcformat -file <filepath> [-blocksize <block size>] -blocks <number blocks> [-help]

To format the device or file, execute the command using the syntax described in Example 7-4.

� For a raw disk partition:

The -blocksize and -blocks arguments are optional. The default block size is 8192 bytes. If you do not specify the -blocks argument, the disk partition will be filled with as many blocks as it can contain.

On the Window NT platform, you must mark the cache device as unwritable. Use the disk utility to delete the device or partition you want to use, then re-create it without formatting it. This will cause the system to consider the device unavailable. To create the device in Windows NT, use the command shown in Example 7-5. The raw device path is defined as \\.\d: for d:

Example 7-5 Using Windows NTC:\>htcformat \\.\d:Are you sure you want to format \\.\d:? (y/n): yFormatting device \\.\d:, block size 8192, 524120 blocks ... Done

On the AIX platform, we first created a volume group. Then we defined a logical volume called lvwte. To create the device in AIX, use the command shown in Example 7-6. The raw device path is defined as /dev/rlvwte for /dev/lvwte.

Example 7-6 On AIX# htcformat /dev/rlvwteAre you sure you want to format /dev/rlvwte? (y/n): yFormatting device /dev/rlvwte, block size 8192, 20480 blocks ... Done

� For a file:

The -blocksize argument is required because it determines the file size. The default block size is 8192 bytes and the -blocks argument must be at least 2049 bytes.

Example 7-7 Using Windows NTC:\>htcformat -file D:\cache\cache.wte -blocks 2049Are you sure you want to format cache.wte? (y/n): yFormatting device cache.wte, block size 8192, 2049 blocks ... Done

Example 7-8 Using the AIX platform# htcformat -file /cache/cache.wte -blocks 2049Are you sure you want to format cache.wte? (y/n): yFormatting device cache.wte, block size 8192, 2049 blocks ... Done

Example 7-9 Using the Linux platform[root@rhlinux1 wte]# htcformat -file /cache/cache.wte -blocks 2049Are you sure you want to format cache.wte? (y/n): yFormatting device cache.wte, block size 8192, 2049 blocks ... Done

Note: On the Linux platform, caching directly to a storage device is not supported. Caches on Linux can be stored in cache file, or in RAM.


After formatting the cache devices, we can configure the cache. Select Cache Settings from Cache Configuration in the navigation pane to retrieve the Cache Settings form, as shown in the following figure:

Figure 7-8 Cache Settings window

In the Device Name field, enter the directory or file where the cache file is located. We input with the raw logical volume created on AIX.


The cache access log file is used for logging hits on the proxy cache. This is only valid if the WTE server is running as a proxy. We stored the cache access log file in the /wte/logs/server file.

Again, note that permissions are important when changing the default directory where the log files will reside. The WTE server will write to that directory as the user ID/group ID specified in the ibmproxy.conf configuration file (nobody/nobody by default). If you have created a new directory for the logs, you must ensure that the WTE server’s user ID can write to that directory.

For the configuration changes to be written in the configuration files, click Submit. This directive requires you to stop and start the server, as noted in Table 7-3 on page 295.

Garbage collectionOnce caching is enabled, disk space and file maintenance are required. WTE provides a storage reclamation process for disk space and file management. This process should be enabled so that you can prevent the cache of your proxy server from growing beyond the maximum size that you set.

Garbage collection is a process that deletes all expired files as defined by the Cache Expiration Settings form. Expired files should be the cached files that are no longer used.

The WTE cache expiration is set by clicking Cache Configuration -> Cache Expiry Settings -> Cache Expiration Settings. The WTE Cache Expiration Settings form is shown in Figure 7-9.

We accepted the default settings:

� Expire a cached HTTP file which has been unused for 2 days

� Expire a cached FTP file which has been unused for 3 days

� Expire a cached Gopher file which has been unused for 12 hours

� Set FTP Default Expiration time to 1 day

� Set Gopher Default Expiration time to 2 days

� Enable cached file expiration checking

� Disable caching of files due to expire within 10 minutes

Note: The cache file is not deleted when you reinstall WTE. It can be accessed and used again when WTE is reinstalled.


Figure 7-9 Cache expiration settings

By default, the garbage collection process is enabled, and is performed every time the cached volume reaches the value specified in the High Water Mark field. The process stops when it reaches the value specified in the Low Water Mark field, shown in Figure 7-10.


Figure 7-10 Garbage collections settings

Garbage collection provides three algorithms for choosing how files are removed from the cache. They are:

� bandwidth: the algorithm that optimizes network bandwidth

� responsetime: the algorithm that optimizes user response time

� blend: the algorithm that blends the two and gives you a balance of network bandwidth and user response time


When you are tuning the cache to minimize network bandwidth, larger files are given a lower priority for deletion. They are less likely to be removed during the storage reclamation. When you are tuning the cache to minimize response time, larger files are given a higher priority for deletion and, therefore, are more likely to be removed during garbage collection.

The default value of the cache algorithm field is bandwidth. We chose to select blend because it gives you a balance of network bandwidth and user response time.

To access the garbage collection settings form, click the Cache Configuration folder in the navigation frame. Then, select Garbage Collection Settings, as shown in Figure 7-10.

Select Submit to write the configuration changes in the configuration file. For the changes to take effect, you must restart the WTE server and click Restart at the top right of the pane to restart the ibmproxy daemon, which will reread the ibmproxy.conf file.

7.1.5 Configuration of the ibmproxy.conf fileWe have performed basic customization using the Configuration and Administration forms; the following example displays the directives that were changed on the ibmproxy.conf file.


Example 7-10 All configured directives # Specify the protocols that this proxy server will forward:Proxy http:*Proxy ftp:*Proxy gopher:*...# PureProxy directive:PureProxy on...# Caching directive:Caching ON...# Logging directives CacheAccessLog /wte/logs/cacheProxyAccessLog /wte/logs/proxy ...# CacheDev directive: CacheDev /dev/rlvwte...# CacheMemory directive:CacheMemory 16392 K...# CacheDefaultExpiry directive:CacheDefaultExpiry http:* 0 daysCacheDefaultExpiry ftp:* 1 dayCacheDefaultExpiry gopher:* 2 days ...# CacheUnused directive:CacheUnused http:* 2 daysCacheUnused ftp:* 3 daysCacheUnused gopher:* 12 hours ...# CacheTimeMargin directive:CacheTimeMargin 10 minutes...# Gc (Garbage Collection) directive:Gc on...# CacheAlgorithm directive:cacheAlgorithm blend...# GcHighWater directive:GcHighWater 90...# GcLowWater directive: GcLowWater 60


Note that the directives in the ibmproxy.conf file are not arranged in the same way as in the Configuration and Administration forms.

7.2 TestsTo test if our proxy server is working as expected, we must customize our browser and access a URL.

7.2.1 Configuring the browserWe configured Netscape Communicator 4.7 and Microsoft Internet Explorer 5 on the client machine (client IP address 9.24.105.82). Refer to Figure 7-1 on page 289 to see this configuration.

Netscape browserTo configure the Netscape browser to use our proxy server, click Edit -> Preferences and open the Advanced option. Choose Proxies and Manual proxy configuration and click View.... You will see the following window:

Figure 7-11 Netscape browser configuration


We defined our AIX proxy server, rs600034, on port 80. Click OK to confirm the customizations.

Microsoft Internet Explorer browserTo configure the Microsoft Internet Explorer browser, click Tools -> Internet Options... and open the Connections tabs. Click the LAN Settings... button. You will see the following window:

Figure 7-12 Microsoft Internet Explore browser configuration

We defined our AIX proxy server, rs600034 on port 80. Click OK to confirm the customizations.

7.2.2 Testing the accessWe performed the following steps to test our proxy configuration.

Client machineWe entered the URL w3.ibm.com in the browser on our client machine (IP address 9.24.105.82). The following window was displayed.


Figure 7-13 Testing access through the proxy server


Proxy serverIn the proxy access log (refer to Table 7-4 on page 299 to see where this log is defined), we see that the client machine sent multiple URL requests. In the Configuration and Administration forms, open the Server Activity monitor and choose Proxy Access Statistics. You will see the following window:

Figure 7-14 The proxy access statistics information

You can view information about which URLs were requested and which were satisfied from the cache. The first column contains the IP address of the machine that requested the URL.


Chapter 8. Advanced features

In this chapter, we cover details and scenarios describing some of the advanced features of Web Traffic Express. Included are some of the new features in this version of the Web Traffic Express:

� LDAP authentication (see 8.1.3, “User authentication using LDAP” on page 326)

� CyberPatrol filtering (see 8.1.4, “Blocking URLs using the CyberPatrol plug-in” on page 336)

� Dynamic caching (see 8.2.4, “Caching of dynamic content” on page 359)

8


8.1 Security enhancementsIn this section, we discuss features to improve the security of your environment. We show you how to restrict the access to WTE and how to hide the information that comes from Web browsers so that it is not passed on to Web servers. The following features are discussed in this section:

� Modifying HTTP header data - 8.1.1, “Handling header information” on page 314

� Enable user authentication - 8.1.2, “Basic user authentication” on page 320� LDAP for user authentication - 8.1.3, “User authentication using LDAP” on

page 326� CyberPatrol plug-in - 8.1.4, “Blocking URLs using the CyberPatrol plug-in” on

page 336�

8.1.1 Handling header informationWhen you send a request using any browser, the Web client includes HTML headers that provide additional information about the browser. Header generation occurs automatically when a request is sent. Web Traffic Express can hide or change some information from this HTML header to improve the security of your environment.

In this scenario, we configure WTE to modify some of the header information passed on to the Web server.

Headers contain information about the User-Agent, Client-IP and Referer. A header example is shown below:

Example 8-1 Sample HTML headerUser-Agent: Mozilla 4.7/WinNTClient-IP: 9.24.105.82Referer: http://www.itso.com/index.html

Descriptions of the fields in this header are:

� User-Agent

The value of this field provides browser and operating system information.

� Client-IP

The value of this field provides the IP address of the client requesting the URL.

� Referer


The value of this field provides the destination server with the URL of the link referring to this page.

For this scenario and test, we used the following machines:

Table 8-1 Description of the machines

The machines are on the same network and they are all part of the itso.ral.ibm.com domain.

Figure 8-1 Environment for HTML header information testing

Hostname IPAddress Operating System Service

m238p4yl 9.24.105.82 Win NT Server 4.0 Web Client

rs600034 9.24.105.61 AIX 4.3.3 Caching Proxy Server

rs60008 9.24.105.59 AIX 4.3.3 Web Server

WTEProxy Server

Content ServerIBM HTTP Server

WEB Clients

Internet

9.24.105.82M238P4YL

9.24.105.61RS600034

9.24.105.59RS60008


The following example is an IP packet log captured by iptrace on the Web Server (9.24.105.59). It shows the header information of a typical HTTP request.

Example 8-2 iptrace: header information # iptrace -a -p 80 -d 9.24.105.59 -b trace1# ipreport -Nnsr trace1 > trace1.log# pg trace1.logTCP: th_win=16060, th_sum=f954, th_urp=0TCP: 00000000 47455420 2f204854 54502f31 2e310d0a |GET / HTTP/1.1..|TCP: 00000010 486f7374 3a20392e 32342e31 30352e35 |Host: 9.24.105.5|TCP: 00000020 390d0a43 6f6e6e65 6374696f 6e3a204b |9..Connection: K|TCP: 00000030 6565702d 416c6976 652c2054 450d0a50 |eep-Alive, TE..P|TCP: 00000040 7261676d 613a206e 6f2d6361 6368650d |ragma: no-cache.|TCP: 00000050 0a416363 6570743a 20696d61 67652f67 |.Accept: image/g|TCP: 00000060 69662c20 696d6167 652f782d 78626974 |if, image/x-xbit|TCP: 00000070 6d61702c 20696d61 67652f6a 7065672c |map, image/jpeg,|TCP: 00000080 20696d61 67652f70 6a706567 2c20696d | image/pjpeg, im|TCP: 00000090 6167652f 706e672c 202a2f2a 0d0a4163 |age/png, */*..Ac|TCP: 000000a0 63657074 2d456e63 6f64696e 673a2067 |cept-Encoding: g|TCP: 000000b0 7a69700d 0a416363 6570742d 4c616e67 |zip..Accept-Lang|TCP: 000000c0 75616765 3a20656e 0d0a4163 63657074 |uage: en..Accept|TCP: 000000d0 2d436861 72736574 3a206973 6f2d3838 |-Charset: iso-88|TCP: 000000e0 35392d31 2c2a2c75 74662d38 0d0a5445 |59-1,*,utf-8..TE|TCP: 000000f0 3a206368 756e6b65 640d0a56 69613a20 |: chunked..Via: |TCP: 00000100 48545450 2f312e30 20727336 30303033 |HTTP/1.0 rs60003|TCP: 00000110 342e6974 736f2e72 616c2e69 626d2e63 |4.itso.ral.ibm.c|TCP: 00000120 6f6d2028 49424d2d 50524f58 592d5754 |om (IBM-PROXY-WT|TCP: 00000130 452d5553 290d0a55 7365722d 4167656e |E-US)..User-Agen|TCP: 00000140 743a204d 6f7a696c 6c612f34 2e37205b |t: Mozilla/4.7 [|TCP: 00000150 656e5d20 2857696e 4e543b20 49290d0a |en] (WinNT; I)..|

Configuring header optionsHeader options can be configured using the Configuration and Administration forms or by editing the WTE proxy configuration file ibmproxy.conf. We show how to configure the header using the Configuration and Administration forms.

To access the header configuration form, click the Proxy Configuration folder in the navigation frame. Then, select Privacy Settings, as shown in Figure 8-2.


Figure 8-2 Privacy settings

In the Privacy Settings form:

� For security reasons, we did not select the Forward client’s IP address to destination server box. This directive specifies whether the WTE proxy server should forward the requesting client’s IP address to the destination server or not. If the box is checked, a header field will be generated:

Client-IP: IP_address_of_clientX

� To improve client anonymity, we entered IBM_WTE_Proxy_Server in the User-agent string field. The value entered will overwrite the value sent by the client, which contained browser and operating system information.


� To notify users of any errors, we provided the e-mail address of the WTE administrator, [email protected], so that users can contact the WTE administrator if they encounter any errors or problems. The value entered should become the value of the From: header field.

� Click the Submit button to make the changes to the WTE configuration file, ibmproxy.conf.

� Click the Restart icon to refresh the WTE server.

Once again, we collected an iptrace after the we completed the WTE header information changes. The results are listed in the example below.

Example 8-3 Header information changed# iptrace -a -p 80 -d 9.24.105.59 -b trace2# ipreport -Nnsr trace2 > trace2.log# pg trace2.logTCP: 00000000 47455420 2f204854 54502f31 2e310d0a |GET / HTTP/1.1..|TCP: 00000010 486f7374 3a20392e 32342e31 30352e35 |Host: 9.24.105.5|TCP: 00000020 390d0a43 6f6e6e65 6374696f 6e3a204b |9..Connection: K|TCP: 00000030 6565702d 416c6976 652c2054 450d0a50 |eep-Alive, TE..P|TCP: 00000040 7261676d 613a206e 6f2d6361 6368650d |ragma: no-cache.|TCP: 00000050 0a416363 6570743a 20696d61 67652f67 |.Accept: image/g|TCP: 00000060 69662c20 696d6167 652f782d 78626974 |if, image/x-xbit|TCP: 00000070 6d61702c 20696d61 67652f6a 7065672c |map, image/jpeg,|TCP: 00000080 20696d61 67652f70 6a706567 2c20696d | image/pjpeg, im|TCP: 00000090 6167652f 706e672c 202a2f2a 0d0a4163 |age/png, */*..Ac|TCP: 000000a0 63657074 2d456e63 6f64696e 673a2067 |cept-Encoding: g|TCP: 000000b0 7a69700d 0a416363 6570742d 4c616e67 |zip..Accept-Lang|TCP: 000000c0 75616765 3a20656e 0d0a4163 63657074 |uage: en..Accept|TCP: 000000d0 2d436861 72736574 3a206973 6f2d3838 |-Charset: iso-88|TCP: 000000e0 35392d31 2c2a2c75 74662d38 0d0a5445 |59-1,*,utf-8..TE|TCP: 000000f0 3a206368 756e6b65 640d0a56 69613a20 |: chunked..Via: |TCP: 00000100 48545450 2f312e30 20727336 30303033 |HTTP/1.0 rs60003|TCP: 00000110 342e6974 736f2e72 616c2e69 626d2e63 |4.itso.ral.ibm.c|TCP: 00000120 6f6d2028 49424d2d 50524f58 592d5754 |om (IBM-PROXY-WT|TCP: 00000130 452d5553 290d0a46 726f6d3a 20777465 |E-US)..From: wte|TCP: 00000140 61646d69 6e406962 6d2e636f 6d0d0a55 |[email protected]|TCP: 00000150 7365722d 4167656e 743a2049 424d5f57 |ser-Agent: IBM_W|TCP: 00000160 54455f50 726f7879 5f536572 7665720d |TE_Proxy_Server.|

When we compare this result with Example 8-2 on page 316, we can see that:

� The From: header field is [email protected]

� The User-Agent: field no longer shows the browser and operating system information of the client, but the string IBM_WTE_Proxy_Server.

You can also selectively block client header information using the Proxy Header Filtering form. This information is blocked on the proxy server machine and the


Web server does not receive it.

To access this form, click the Proxy Configuration folder in the navigation frame. Then, select Proxy Header Filtering, as shown in the Figure 8-3:

Figure 8-3 Proxy header filtering window

In Figure 8-3, we blocked User-Agent information. Click Submit to write the changes to the WTE configuration file ibmproxy.conf. Click the Restart icon to refresh the WTE server.


We collected an iptrace after we blocked the User-Agent information. You can see the result in the example below:

Example 8-4 Header information without User-Agent# iptrace -a -p 80-d 9.24.105.59 -b trace3# ipreport -Nnsr trace3 > trace3.log# pg trace3.logTCP: th_off=5, flags<PUSH | ACK>TCP: th_win=16060, th_sum=2bf3, th_urp=0TCP: 00000000 47455420 2f204854 54502f31 2e310d0a |GET / HTTP/1.1..|TCP: 00000010 486f7374 3a20392e 32342e31 30352e35 |Host: 9.24.105.5|TCP: 00000020 390d0a43 6f6e6e65 6374696f 6e3a204b |9..Connection: K|TCP: 00000030 6565702d 416c6976 652c2054 450d0a41 |eep-Alive, TE..A|TCP: 00000040 63636570 743a2069 6d616765 2f676966 |ccept: image/gif|TCP: 00000050 2c20696d 6167652f 782d7862 69746d61 |, image/x-xbitma|TCP: 00000060 702c2069 6d616765 2f6a7065 672c2069 |p, image/jpeg, i|TCP: 00000070 6d616765 2f706a70 65672c20 696d6167 |mage/pjpeg, imag|TCP: 00000080 652f706e 672c202a 2f2a0d0a 41636365 |e/png, */*..Acce|TCP: 00000090 70742d45 6e636f64 696e673a 20677a69 |pt-Encoding: gzi|TCP: 000000a0 700d0a41 63636570 742d4c61 6e677561 |p..Accept-Langua|TCP: 000000b0 67653a20 656e0d0a 41636365 70742d43 |ge: en..Accept-C|TCP: 000000c0 68617273 65743a20 69736f2d 38383539 |harset: iso-8859|TCP: 000000d0 2d312c2a 2c757466 2d380d0a 54453a20 |-1,*,utf-8..TE: |TCP: 000000e0 6368756e 6b65640d 0a566961 3a204854 |chunked..Via: HT|TCP: 000000f0 54502f31 2e302072 73363030 3033342e |TP/1.0 rs600034.|TCP: 00000100 6974736f 2e72616c 2e69626d 2e636f6d |itso.ral.ibm.com|TCP: 00000110 20284942 4d2d5052 4f58592d 5754452d | (IBM-PROXY-WTE-|TCP: 00000120 5553290d 0a46726f 6d3a2077 74656164 |US)..From: wtead|TCP: 00000130 6d696e40 69626d2e 636f6d0d 0a0d0a |[email protected].... |

When we compare this result with Example 8-3 on page 318, we can see that the User-Agent field does not appear.

8.1.2 Basic user authenticationIn this section, we show how to configure WTE to enable user authentication. Only authorized users can access files and forms on the proxy server, as covered in 7.1.3, “Web Traffic Express Administration settings” on page 291. A user is required to be authenticated with a user ID and password.

To configure this feature, follow these steps:

� Create a user and password file. To access the basic user authentication form, click the Server Configuration folder in the navigation frame. Then, select User Administration; with this option, you can add a user, delete a user, verify if the user already exists or change the password. In the following window, we will add a new user.

Figure 8-4 Add user

After filling in the required fields, click Submit to add the user. The following example shows the commands used to create the files required for the Add User form shown above.

Example 8-5 Commands used to create Add User form files# cd /opt/IBMWTE/usr/internet/server_root/protect# cat wteusers.groupwteusers: jonh# cat wteusers.passwdjonh:WED2Ye1pOaqek:Jonh Doe

� Define a new protection setup. A protection setup contains details about what to do with a request that matches a request template. To access the


document protection form, click the Server Configuration folder in the navigation frame. Then, select Document Protection, as shown below in Figure 8-5.

Figure 8-5 Document protection: part 1

The URL request template field defines the name of the proxy server resource that you want to protect. We entered http:*. So, in our case, all client requests starting with http: will cause the WTE server to prompt for a user ID and password.


The Named protection field defines the name of a new protection setup. We entered PROT-PROXY. We added this new protection after the PROT-ADMIN protection setup.

When you click Submit, the following window is displayed.

Note: If you want to protect the access to all the WTE server’s functions, then the only directive to use would be:

Protect * PROT-PROXY.


Figure 8-6 Document protection: part 2

This window has more directives associated with protection setup. These directives are used to specify the permissions (read, write, delete) a user or group has when manipulating the proxy server configuration and files.


� The Server Identifier field specifies the name that identifies the protection setup to requesters. When the proxy server sends a requester a prompt for a user ID and password, it will also include the name you specify as the value of the Server Identifier subdirective. Most browsers display this name in the prompt. In this way, the requester is capable of understanding which proxy server sent the prompt, and can decide which user ID and password to send back.

� The PasswdFile and GroupFile fields specify the path and name of the password file and group file to be used. In our case, we defined this subdirective as /opt/IBMWTE/usr/internet/server_root/protect/wteusers.passwd and /opt/IBMWTE/usr/internet/server_root/protect/wteusers.group. A password file contains a list of user IDs and passwords. Each user ID has one valid password defined for it. Note that the user ID in the password file does not have any relation to the addresses or host name machines of the requester, nor with the users defined in the underlying operating system. A password file is created in the proxy server machine itself. To create and maintain password files, you can access the Configuration and Administration forms, as we showed above, or you can use the htadm command, as shown in 7.1.3, “Web Traffic Express Administration settings” on page 291.

� The other fields allow you to specify how one or more of the HTTP methods are to be accessed. You can specify different access levels for GET and POST methods. We inputted All to GET and POST, so any user in the password file specified by the PasswdFile subdirective could issue a GET or POST request. You can also specify a mask. If you specify All@96.*.*.* in the GET field, then WTE will allow access to all users in the password file, but only if the request comes in from an IP address starting with 96. Any other request will be rejected.

Click Submit to write the configuration changes to the configuration file. For the changes to take effect, you must restart the WTE server.

With the protection setup, we received a prompt from the proxy server requesting a valid user ID and password as defined in the password file (see the following figure).


Figure 8-7 Request for User ID and password for authentication

Configuration of the ibmproxy.conf fileIn the following example, we show the directives that were included in the ibmproxy.conf file for the configuration described above.

Example 8-6 Directives included for the above configuration# Protection directive: Protection PROT-PROXY { GroupFile /opt/IBMWTE/usr/internet/server_root/protect/wteusers.group PasswdFile /opt/IBMWTE/usr/internet/server_root/protect/wteusers.passwd PostMask All PutMask All GetMask All AuthType Basic ServerID ProxyServer}#Protect /admin-bin/* PROT-ADMINProtect /reports/* PROT-ADMINProtect /Usage* PROT-ADMINProtect http:* PROT-PROXY

8.1.3 User authentication using LDAPWeb Traffic Express provides a plug-in that supports the use of an LDAP database as a WTE registry for the storing of user authentication information. This plug-in enables the authentication of clients using an LDAP directory and user defined policy information.


The following LDAP directory servers are currently supported:

� IBM SecureWay Directory 3.2.1� Netscape Directory Server 4.12� Novel NDS eDirectory 8.5 (not on AIX)

The Configuration and Administration forms are not available for editing the WTE LDAP configuration directives. A text editor must be used to edit the LDAP configuration and policy file, pac.conf.

The machines used in our scenario for testing this plug-in are described in the following table.

Table 8-2 Description of the machines

The machines are on the same network and they belong to the same itso.ral.ibm.com domain.

Figure 8-8 Environment for LDAP Authentication Testing

Hostname IPAddress Operating System Service

m238p4xl 9.24.105.83 Win NT Server 4.0 IBM SecureWay Directory - LDAP Server 3.2.1

rs600034 9.24.105.61 AIX 4.3.3 Caching Proxy Server and Netscape Directory SDK C4.11

m238p4yl 9.24.105.82 AIX 4.3.3 Web Client

9.24.105.61RS600034

WTEProxy Server

WEB Client

9.24.105.82M238P4YL

Web ContentServer

9.24.105.83M238P4XL

LDAP Server

UserAuthentication

Internet


Table 8-3 Required LDAP plugin libraries

On an AIX platform, symbolic links are created from /usr/lib to the /opt/IBMWTE/usr/internet/lib/plugins/pac WTE libraries. All Netscape libraries must be copied to /usr/lib.

Configuring LDAP authenticationWeb Traffic Express will authenticate the users using the LDAP database. Therefore, we must add clients in our IBM SecuryWay Directory - LDAP Server. To test our scenario we defined:

� Country: c=us

� Organization: o=ibm

� Organization Unit: ou=ITSO

� Common Name: cn=Ana Mostardinha and cn=Cristiane Ferreira

� Username: uid=analums and uid=crismari

We input two distinguishable names:

� dn=Ana Mostardinha,ou=ITSO,o=IBM,c=us

� dn=Cristiane Ferreira,ou=ITSO,o=IBM,c=us

Note: As of the writing of this redbook, you must install the Netscape Directory SDK for C4.11 and WTE in the same machine to run this plug-in. Four Netscape directory libraries are required to use the LDAP plug-in. You can download them from http://www.iplanet.com/downloads/developer/

Platform Path WTE files Netscape libraries

AIX /opt/IBMWTE/usr/internet/lib/plugins/pac libpacwte.a.solibpacman.a

libldapssl41.solibplc3.solibplds3.solibnspr3.so

Windows c:\program files\IBM\WTE\bin\plugins\pac libpacwte.dlllibpacman.dll

libldapssl41.dlllibplc3.dlllibplds3.dlllibnspr3.dll

Linux /usr/lib libpacwte.solibpacman.so


http://www.iplanet.com/downloads/developer/

When using LDAP as a WTE registry, all authentication is performed by the LDAP Server. You do not access the Configuration and Administration forms using the logon and password defined in the installation process of Web Traffic Express.

To prevent users from accessing the Configuration and Administration forms, we recommend disabling the WTE Configuration and Administration forms. Edit the ibmproxy.conf file and comment the following lines:

Example 8-7 Disabling the Configuration and Administration forms# ===================================================================== ### Mapping rules## ===================================================================== #...# Pass /admin-bin/webexec/*.style /opt/IBMWTE/usr/internet/server_root/admin-bin/webexec/*.style# Pass /admin-bin/webexec/*.NLS /opt/IBMWTE/usr/internet/server_root/admin-bin/webexec/*.NLS# Pass /admin-bin/webexec/*.props /opt/IBMWTE/usr/internet/server_root/admin-bin/webexec/*.props# Pass /admin-bin/webexec/*.class /opt/IBMWTE/usr/internet/server_root/admin-bin/webexec/*.class# Pass /admin-bin/webexec/*.gif /opt/IBMWTE/usr/internet/server_root/admin-bin/webexec/*.gif# Pass /admin-bin/webexec/*.jar /opt/IBMWTE/usr/internet/server_root/admin-bin/webexec/*.jar# Pass /admin-bin/webexec/*.html /opt/IBMWTE/usr/internet/server_root/admin-bin/webexec/*.html # Pass /admin-bin/webexec/images/* /opt/IBMWTE/usr/internet/server_root/admin-bin/webexec/images*# Pass /admin-bin/webexec/* /opt/IBMWTE/usr/internet/server_root/admin-bin/webexec/*...# Exec /admin-bin/* /opt/IBMWTE/usr/internet/server_root/admin-bin/* ...# ===================================================================== ### User authentication and document protection## ===================================================================== #...# Protect /admin-bin/* PROT-ADMIN# Protect /reports/* PROT-ADMIN# Protect /Usage* PROT-ADMIN


The LDAP plug-in configuration and policy information are defined in the pac.conf file. This file is located in the same directory as the ibmproxy.conf file, as described in Table 7-2 on page 295.

The following example describes the attributes in the pac.conf file and their possible values:

Example 8-8 Terminology of the pac.conf file[PAC_MAN_SERVER]hostname: fully qualified hostname of proxy server.port: free port number to be used LDAP plugin daemon (pacd).default_policy: [grant/deny], policy for users not covered by policies below.conn_type: [ssl], to allow aal connection between pacd and LDAP Server. If you do not want an ssl connection, comment this option.

[LDAP_SERVER]hostname: fully qualified hostname of the LDAP directory server.port: port used by LDAP directory serverssl_port: ssl port used by LDAP directory serveradmin_dn: an distinguished name with read access to the entire LDAP directory server.search_base: the root of the LDAP directorysearch_key: username field as defined in the LDAP directory.

[PACWTE_PLUGIN]hostname_check: [true/false] allow DNS lookups. Must have DNS lookup turned on for ibmproxy to work.

[POLICY]id: id to which this policy pertains.type: [0,1,2,3] LDAP entry which is a group(0), is a not group (1), IP address (2) and hostname (3).grant: [true/false] means to get access, otherwise means deny for this id.propagate: [true/false], means the access rights (grant or deny) will be propagated to its descendants or members. If the id is a group, the value of propagate will default to TRUE.stop_entry: [entry/NULL] the access right propagation will be stopped on this entry and its sub-tree of this entry, stop_entry may be a set of the entries. If the id is a group stop_entry=NULL.Excepton_entry: [entry/NULL] The access right will not be propagated on this entry,be propagated on its sub-tree or members of this entry. exception_entry may be a list of the entries. exception_entry might be applied to all type of IDs.

Multiple policies are supported.

If no policies exists, there is no restriction for the access permissions.

[cache]


cred_cache_enable: [true/false]cred_cache_min_size: [100-64000] initial number of cred cache entriescred_cache_max_size: [100-64000] cache can grow to this ceilingcred_cache_expiration: [300-86400] secondspolicy_cache_enabled: [true/false]policy_cache_min_size: [100-64000] initial number of policy cache entriespolicy_cache_max_size: [100-64000] cache can grow to this ceilingpolicy_cache_expiration: [300-86400] seconds

The cache will automatically double upon reaching 75% full

We edited the pac.conf file on AIX and configured it for our scenario.

Example 8-9 Configured pac.conf file[PAC_MAN_SERVER]hostname:rs600034.itso.ral.ibm.comport:2222default_policy:deny

[LDAP_SERVER]hostname:m238p4xl.itso.ral.ibm.comport:389admin_dn:cn=rootsearch_base:o=ibm,c=ussearch_key:uid

[PACWTE_PLUGIN]hostname_check:false

[POLICY]type:1id:cn=admin,ou=ITSO,o=ibm,c=usgrant:truepropagate:true

[POLICY]type:1id:cn=Ana Mostardinha,ou=ITSO,o=ibm,c=usgrant:truepropagate:true

[POLICY]type:1id:cn=Cristiana Ferreira,ou=ITSO,o=ibm,c=usgrant:falsepropagate:true

[CACHE]


cred_cache_enabled:FALSEcred_cache_min_size:100cred_cache_max_size:64000cred_cache_expiration:86400policy_cache_enabled:FALSEpolicy_cache_min_size:100policy_cache_max_size:64000policy_cache_expiration:86400

On the LDAP_SERVER stanza, the admin_dn is an administration user with read access of the LDAP database. In our environment, it is cn=root. The password for this user is stored in the pac_ldap.cred file. You must create this file and add to it the password admin_dn. You must also change the permission for this file to nobody. This file is located in <WTE_home>/pac/creds. Initially, this file contains a text password. However, when the LDAP plugin daemon pacd starts, it will encrypt this password. We created this file and added the password to it.

After updating or changing the pac.conf file, you must restart the LDAP plugin daemon pacd. On an AIX and Linux platform, run the pacd_restart.sh command. On a Windows platform, run pacd_restart.bat. These scripts restart the daemon without interrupting WTE.

To start the pacd daemon, enter the following command.

pacd -h <WTE_home> -C directory_of_pac.conf

where WTE_home is /opt/IBMWTE/usr/internet/server_root on an AIX or Linux platform and c:\Program files\IBM\WTE on a Windows platform.

The WTE LDAP plugin includes three routines:

1. pacwte_auth_init()-- starts and initializes LDAP plugin daemon2. pacwte_auth_policy()-- authenticates and checks the policies3. pacwte_shutdown()-- kills the LDAP plugin daemon.

There are four directives:

1. ServerInit 2. Authentication 3. Authorization4. ServerTerm

The directives use the plugin routines and must be added in the API directives section of the ibmproxy.conf file. Edit the file, find, uncomment and change, if necessary, the following lines.

� For an AIX platform:

Example 8-10 LDAP plugin on AIX and directive syntax# ===== LDAP Plugin =====# ServerInit path_of_shared_library:pacwte_auth_init path_of_conf_policy_fileServerInit /opt/IBMWTE/usr/internet/lib/plugins/pac/libpacwte.a.so:pacwte_auth_init /etc/pac.conf

# Authorization </URL> path_of_shared_libraries:pacwte_auth_policy# Authorization http://* /opt/IBMWTE/usr/internet/lib/plugins/pac/libpacwte.a .so:pacwte_auth_policy

# Authentication BASIC path_of_shared_library:pacwte_auth_policyAuthentication BASIC /opt/IBMWTE/usr/internet/lib/plugins/pac/libpacwte.a.so:pacwte_auth_policy

# ServerTerm path_of_shared_library:pacwte_shutdownServerTerm /opt/IBMWTE/usr/internet/lib/plugins/pac/libpacwte.a.so:pacwte_shutdown

Note that we do not uncomment the Authorization directive on the AIX platform, because our scenario shows only user authentication.

� For a Windows platform:

Example 8-11 LDAP plugin on Windows and directive syntax# ===== LDAP Plugin =====#ServerInit path_of_shared_library:pacwte_auth_init path_of_conf_policy_fileServerInit C:\Progra~ 1\IBM\WTE\bin\plugins\pac\pacwte.dll:pacwte_auth_init C:\Progra~ 1\IBM\WTE\pac.conf

# Authorization </URL> path_of_shared_libraries:pacwte_auth_policy# Authorization http://* C:\Progra~1\IBM\WTE\bin\plugins\pac\pacwte.dll:pacwt e_auth_policy

# Authentication BASIC path_of_shared_library:pacwte_auth_policyAuthentication BASIC C:\Progra~1\IBM\WTE\bin\plugins\pac\pacwte.dll:pacwt e_auth_policy

# ServerTerm path_of_shared_library:pacwte_shutdownServerTerm C:\Progra~ 1\IBM\WTE\bin\plugins\pac\pacwte.dll:pacwte_shutdown

� For a Linux platform:

Example 8-12 LDAP plugin on Linux and directive syntax# ===== LDAP Plugin =====

# ServerInit path_wte_libraries:pacwte_auth_init path_of_conf_police_file# ServerInit /usr/lib/libpacwte.so:pacwte_auth_init /etc/pac.conf

# Authorization </URL> path_of_shared_libraries:pacwte_auth_policy# Authorization http://* /usr/lib/libpacwte.so:pacwte_auth_policy

# Authentication BASIC path_of_shared_library:pacwte_auth_policyAuthentication BASIC /usr/lib/libpacwte.so:pacwte_auth_policy

# ServerTerm path_of_shared_library:pacwte_shutdownServerTerm /usr/lib/libpacwte.so:pacwte_shutdown

To authorize users to access the requested URL through the defined proxy server, you must define what type of protection to use. For our scenario, all HTTP requests will be authenticated. Edit the ibmproxy.conf file, find the Protection directive and add the lines which we have highlighted in bold in the following example:

Example 8-13 Editing the ibmproxy.conf file# ===================================================================== ### User authentication and document protection## ===================================================================== ##Protection PROT-PROXY-LDAP { PostMask All GetMask All AuthType Basic ServerID LDAP_Authentication}Protection PROT-ADMIN { ServerId Private_Authorization AuthType Basic GetMask All@(*) PutMask All@(*) PostMask All@(*) Mask All@(*) PasswdFile /opt/IBMWTE/usr/internet/server_root/protect/webadmin.passwd}

Protect /admin-bin/* PROT-ADMINProtect /reports/* PROT-ADMINProtect /Usage* PROT-ADMINProtect http://* PROT-PROXY-LDAP

Note that there is no PasswdFile field and no GroupFile field because the authentication will be done by LDAP and not by WTE.

For the last step, we stopped and restarted the proxy server (described in Chapter 6, “Web Traffic Express installation” on page 247).

LoggingThere are four levels of log files that are controlled by the environment variable, PAC_DEBUG_LEVEL. All logs are generated on two sides:

� one side is the client side (Web Traffic Express Server)

� the other side is the server side (LDAP plugin - pacd).

Logs are requested as follows:

� The default log is the Error log file. This file is created with the name PacErr_side.date.

� The Audit log file is created as PacLog_side.date. To create this log, you must set PAC_DEBUG_LEVEL=2.

� The Socket Package log file is created as PacPac_side.date. To create this log, you must set PAC_DEBUG_LEVEL=4.

� The Trace log file is created as PacTra_side.date. To create this log, you must set PAC_DEBUG_LEVEL=9.

side is defined as Client for the WTE logs or Server for the pacd logs. date is in the format mmmddyyyy. All logs are located in <WTE_server>/logs. Logging consumes significant resources and disk space. Set PAC_DEBUG_LEVEL=9 to receive the maximum amount of information.

After defining the log variable, we restarted the LDAP plugin daemon (pacd). On AIX and Linux, run pacd_restart.sh and on Windows run pacd_restart.bat.

Testing the configurationTo test our LDAP plugin configuration, we configured our browser to access the proxy server with host name RS600034, as described in 7.2.1, “Configuring the browser” on page 309. We made a request to the URL http://www.ibm.com. Because we defined in the ibmproxy.conf file that all http://* requests would be authenticated, the following window was displayed:

Figure 8-9 LDAP authentication window

We logged in as a user that is defined to the LDAP directory server and has access to our proxy server as defined in the pac.conf file. So the requested page was displayed. If you try to access the proxy server with an undefined user or a user that is defined in pac.conf with deny access, the following window will be displayed:

Figure 8-10 LDAP authorization failed.

8.1.4 Blocking URLs using the CyberPatrol plug-inThe CyberPatrol filtering plug-in allows Web content filtering based on the SurfControl database of acceptable and unacceptable Web sites. You can define which Web sites categories to permit or deny access to.


SurfControl maintains the database of Web sites the content of which can be subject to filtering. Two filtering methods are available. Choosing the CyberNOT method blocks access to Web sites that are listed in the negative categories. Choosing the CyberYES method allows access only to Web sites included in the positive categories. In addition, the plug-in allows you to specify additional Web sites to block or to allow.

Currently, there are sixteen categories to allow access and fourteen categories to block access. Below are all categories and their hexadecimal codes listed in this database:

Table 8-4 CyberPatrol categories

The precompiled encrypted URL lists are the heart of the CyberPatrol filter. You can find more information about filtering categories on the SurfControl Web site, http://www.surfcontrol.com.

CyberNOT CyberYES

Violence/Profanity - 0x0001 Games and Toys - 0x0001

Partial Nudity - 0x0002 Art, Books and Music - 0x0002

Full Nudity - 0x0004 Movies and TV- 0x0004

Sexual Acts - 0x0008 Outdoors and Sports- 0x0008

Gross Depictions -0x0010 Oceans and Space- 0x0010

Intolerance - 0x0020 Pets, Animals and Dinosaurs- 0x0020

Satanic/Cult -0x0040 Other Interesting Stuff- 0x0040

Drugs/Drug Culture - 0x0080 Vacation and Travel- 0x0080

Militant/Extremist - 0x0100 Puzzles and Hobbies- 0x0100

Sex Education - 0x0200 School Work- 0x0200

Questionable/Illegal and Gambling - 0x0400

Volunteer and Help- 0x00400

Alcohol and Tobacco - 0x0800 Kids- 0x0800

Sports and Entertainment - 0x1000 Teens- 0x1000

Search Engines - 0x8000 Reference Materials - 0x2000

Schools on the Net- 0x4000

Parents and Teacher- 0x8000


http://www.surfcontrol.com

The SurfControl database is delivered with the WTE product and installed automatically. The CyberLists, memory loadable databases, registration data and communication logs are stored in <WTE_server>/filter/cyberpatrol on the AIX platform and <WTE_server>\bin\plugins\filter\cyberpatrol on the Windows platform. The database files should be periodically updated to maintain a current list of Web sites that SurfControl has ranked. One year of free database updates is included with Web Traffic Express.

There are some considerations to remember when using this plug-in:

� Configuration overhead

� Some predictable performance impact on the proxy. However, all the domain names, IP addresses and directory names in the memory loadable databases are hashed, thus providing fast lookup, and the performance impact is not so severe.

A major benefit is that this feature can be used to block URLs that are inappropriate for your specific workplace environment. You have access control and a reduction in network traffic.

ConfigurationFirst, you must configure WTE to consult the CyberPatrol filtering plug-in. Edit the ibmproxy.conf file (this file is located as described on Table 7-2 on page 295). Find and uncomment the following directives:

� For an AIX platform:

Example 8-14 CyberPatrol API on AIX # ===== CyberPatrol Plugin =====ServerInit /opt/IBMWTE/usr/internet/lib/plugins/cyberpatrol/libcpfilter.o:cpfilterInitPreExit /opt/IBMWTE/usr/internet/lib/plugins/cyberpatrol/libcpfilter.o:cpfilterMain

� For a Windows platform:

Example 8-15 CyberPatrol API on Windows===== CyberPatrol Plugin =====ServerInit C:\Progra~1\IBM\WTE\bin\plugins\filter\cyberpatrol\cpfilter.dll:cpfilterInitPreExit C:\Progra~1\IBM\WTE\bin\plugins\filter\cyberpatrol\cpfilter.dll:cpfilterMain

� For a Linux platform:

Example 8-16 CyberPatrol API on Linux # ===== CyberPatrol Plugin =====ServerInit /usr/lib/libcpfilter.so:cpfilterInitPreExit /usr/lib/libcpfilter.so:cpfilterMain

Next, from the Configuration and Administration forms pane, click Plugin Configuration -> CyberPatrol Filtering. You will see the following window:

Figure 8-11 CyberPatrol filtering: part 1


We checked the Enable CyberPatrol Filtering option (the box is unselected by default). You can specify when this filtering will be running. By default, it runs all the time.

In the next configuration section, you choose what kind of filtering method to use. The CyberNOT setting denies access to specified Web sites, while the CyberYES setting allows access only to specified Web sites. We chose the CyberNot option and selected the Search Engine Queries box, as shown in Figure 8-12.



For the configuration changes to be written to the configuration files, click Submit.

The filter database must be updated based on our selection of categories to filter. Click the Update Filter Files button at the bottom of the configuration page to perform this function. This configuration-specific filter database is loaded with WTE when it starts. Each time you change the filtering method (CyberNOT or CyberYES) or select different categories, you must rebuild the proxy’s database.


When the process ends, you see the following window:


These directives require you to stop and start the server, as shown in Chapter 6, “Web Traffic Express installation” on page 247.

Note: If you did not create a separate filesystem to install WTE on the AIX platform, you may need to increase your root filesystem by 30 MB.


.

Testing the configurationTo test this feature, we made a request to the URL http://www.altavista.com in our browser which was configured as described in 7.2.1, “Configuring the browser” on page 309. This URL leads to a search engine home page and it will be denied access, as shown in Figure 8-14.

Figure 8-14 Denied access window

Note: You can use the command line to update the filter database. Issue the GenCyberDB command from the /usr/sbin directory for AIX, or the <install_path>\bin\plugins\filter\cyberpatrol directory for Windows.


http://www.altavista.com

Additional sites to filterYou can add other URLs or keywords to filter, but these work together with the type of filtering you have selected, so if you chose CyberNOT, then additional sites and keywords can be blocked. If you chose CyberYES, additional sites and keywords will also be allowed. The following window shows this configuration:

Figure 8-15 Adding to CyberPatrol filtering

We added the URL http://www.dilbert.com. This page will now be denied because we are using the CyberNOT option.

For the configuration changes to be written in the configuration files, click Submit. Do not forget to recreate the database by clicking the Update Filter Files button. Stop and start the server as shown in Chapter 6, “Web Traffic Express installation” on page 247.


http://www.dilbert.com

When trying to access this site, we received a window similar to the one shown in Figure 8-14.

Downloading the CyberPatrol databaseTo update the CyberPatrol database, you must register with SurfControl. There are two links on the Configuration and Administration forms for these functions. One is for registering and the other one is for downloading the database. Refer to the upper right part of Figure 8-11 on page 339 for the location of these links.

We used the Microsoft Internet Explorer browser to register and download the CyberPatrol database, because as of the writing of this redbook, there is a problem with the Netscape browser. The fix which will allows you to use the Netscape browser is being built and you can download it from the Web site http://www.ibm.com/software/webservers/edgeserver/.

To access the registration forms, we clicked the link here in the text: “If you have not registered with SurfControl to receive updated CyberLists databases, click here to register” in Plugin Configuration -> CyberPatrol Filtering in the Configuration and Administration forms (refer to Figure 8-11 on page 339). The following window was displayed:


http://www.ibm.com/software/webservers/edgeserver/

Figure 8-16 CyberPatrol proxy registration form

We filled out our information and clicked Submit. When the registration process finished, we were shown the following window:


Figure 8-17 Registration forms: successful registration window

One file was created called registration.data which contained all our information.

We then updated the CyberPatrol database. We clicked the link here in the text: “If you already have subscribed for database updates, click here to download updated CyberLists from SurfControl” in Plugin Configuration -> CyberPatrol Filtering in the Configuration and Administration forms. The following window was shown:


Figure 8-18 CyberPatrol download window

We clicked the Download button. After this process finished, we were shown the following window:


Figure 8-19 CyberList download window

You can automatize this process if you have sucessfully registered yourself as described above. To download the database, use the command cpreg -d. After the download has completed, it is necessary to regenerate the memory database. Run the GenCyberDB command. Below we show an example for each platform:


� On AIX platforms, you can set the crontab file:

Example 8-17 Updating the crontab file on an AIX platform# crontab -eadd the following two lines:00 01 * * * 6 (cd /opt/IBMWTE/usr/internet/server_root/filter/cyberpatrol/; ./cpreg -d) >/dev/null 2>&100 02 * * 6 (cd /usr/sbin/; ./GenCyberDB) > /dev/null 2>&1

Every Saturday at 01:00am the download CyberList will be done, and at 02:00 the GenCyberDB command will run to generate the memory loadable database.

� On Windows platforms, you can set Schedule Tasks:

Example 8-18 Configuring Schedule Tasks on a Windows platformTo schedule the CyberList download:From the START menu choose Settings-> Control Panel -> Scheduled Tasks -> Add Scheduled Task.The Scheduled Task Wizard window will be displayed. Click the Next button to continue the process. Click Browse on the next window.Find C:\Program Files\IBMWTE\bin\plugins\filter\cyberpatrol\cpreg and click Open. Select the “weekly” radio button then click Next. Select 01:00am and Saturday. On the last window, you must select “Open advanced properties for this task when I click Finish” option. Click the Finish button. Another panel will be displayed. In the “Run” box, add “-d” at the end. The entry will look like this:“C:\Program Files\IBMWTE\bin\plugins\filter\cyberpatrol\ cpreg.exe“ -d. Click OK button.To schedule the update Filer Files:Perform the same steps as for the download with the following changes:Change the time to 2:00.Change the binary is GenCyberDBDo not add the “-d” option the end of command.

� On Linux platforms, you can set the crontab file:

Example 8-19 Updating the crontab file on a Linux platform# crontab -u root -eadd the following two lines:00 01 * * * 6 (cd /opt/IBMWTE/usr/internet/server_root/filter/cyberpatrol/; ./cpreg -d) >/dev/null 2>&100 02 * * 6 (cd /usr/sbin/; ./GenCyberDB) > /dev/null 2>&1

Every Saturday at 01:00am the download CyberList will be done, and at 02:00 the GenCyberDB command will run to generate the memory loadable database.


LoggingOne file, dnload.log, is created when you send the registration form. This file contains all logs of the registration and download steps. This file is stored in /opt/IBMWTE/ usr/internet/server_root/filter/cyberpatrol/ on AIX and Linux platforms and in C:\Program Files\IBM\WTE\bin\plugins\filter\cyberpatrol\ on Windows platforms.

There is also a log for the CyberPatrol Filtering Engine. The environment variable, FILTER_DEBUG_ON is used to control this log. This variable switches the verbose mode on and off. The log is written to the stderror.Mmmddyyyy file.

Configuration of the ibmproxy.conf file

In Example 8-20, we show the directives that were updated in the ibmproxy.conf file to configure the CyberPatrol plug-in.

Example 8-20 All configured CyberPatrol directives# ===== CyberPatrol Plugin =====ServerInit /opt/IBMWTE/usr/internet/lib/plugins/cyberpatrol/libcpfilter.o:cpfilterInitPreExit /opt/IBMWTE/usr/internet/lib/plugins/cyberpatrol/libcpfilter.o:cpfilterMain ## Cyber Patrol Filter directives:FilterCyberPatrolOnOff onFilterActivePeriodOneFrom 00:00FilterActivePeriodOneUntil 24:00FilterActivePeriodTwoFrom 00:00FilterActivePeriodTwoUntil 00:00FilterActiveWeekDays Sunday Monday Tuesday Wednesday Thursday Friday SaturdayFilterCyberPatrolList CyberNOTFilterCyberPatrolRuleMask 0x8000FilterAdditionalURL www.dilbert.com

8.2 Cache content managementMost network managers that use a caching function to minimize network bandwidth costs want to get the best hit rate possible. That is, they want to increase the probability that user requests can be satisfied using content within the cache versus using a network request.


There are several decisions an administrator needs to make:

� Which documents are kept in the cache?

� How many documents can be cached?

� How long are they considered current?

� When is the cache refreshed?

In this section, we show how to configure those features using the Configuration and Administration forms.

8.2.1 Controlling which documents are cachedYou can specify which documents should be cached, how long they should be cached, and which documents should never be cached. To define which documents should and should not be cached, choose Cache Configuration -> Cache filters. You will see the following window:


Figure 8-20 Caching filters


8.2.2 Cache freshnessIt is important that the WTE administrator ensure that cached documents are consistent with the original documents located at the originating Web server. For each document that has been cached, WTE computes a time at which the document will expire:

� For HTTP documents, the header of the document generated by the Web server contains the expiration information.

� For FTP documents, WTE generates its own Last-Modified: header information to compute expiration times.

The Web server can indicate the expiration time in several ways when entering header information in the HTTP response. The permitted header information is in the following order of preference:

� The Web server can specify how long the document is valid after it has been received.

� The Web server can specify the exact time at which the document should be considered expired.

� The Web server can indicate the time when a document was last modified. In this case, the caching and filtering proxy server performs a sequence of operations based on a class of parameters specified in the configuration file and calculates how long the document will be valid.

When a document is found in the cache, but has expired, WTE issues a special request to the Web server. This request is called if-modified-since. The Web server sends back the document to the caching and filtering proxy server only if the document has been modified since it was last received by the proxy. Otherwise, the Web server only sends a message indicating that the document has not been modified, and does not send the entire file. At this point, the caching and filtering proxy server can serve the page to the client.

The settings that determine expiration time for cached files are controlled by the Cache Expiry Settings in the Configuration and Administration forms.

Choose Cache Configuration -> Cache Expiry Settings. You will see the following options:

� URL Expiration: Allows you to set a default expiration time for cached files based on theirs URLs.

� Cache Expiration Settings: Define how long WTE keeps unused cached files in the cache. Different values can be set for HTTP, FTP, and Gopher files.

� Last modified Factor: Calculates an expiration date for cached files with no expiration dates in their headers.


� Time Limit for cached files: Defines how long WTE keeps cached files in the cache. Time limits are set based on request templates, and you can specify files that should be discarded as well as files that should be revalidated when the time limit has been reached.

8.2.3 Automatic cache refreshingTypically, the most common proxy servers cache a particular page only after a user requests it. WTE, in addition to this default caching, has a cache agent that provides automatic caching and gives more control to the administrator. The cache agent can retrieve specified URLs even before they are effectively requested and refresh the cache automatically. The cache refresh takes place when the proxy server activity is low (by default, every night at midnight, local time) and all the retrieved pages are ready in the cache to provide faster service, even if it is the first time that a user requests them.

Automatic cache refreshing has two sources it can use to refresh the cache:

� It can load specific URLs defined by the administrator. In this way, the administrator can specify a certain set of pages that must be loaded by the cache agent when it starts.

� It can load the most popular URLs from the previous day’s activity. To obtain these, the cache agent checks the cache access log, sorts it by frequency of requests and then picks the most popular pages. It can refresh the top number of pages as specified by the administrator.

The cache agent can use both sources of input.

Optionally, the cache agent can follow a specified level of HTML links on the pages it is loading and cache all of those linked pages. This operation is also known as delving. It is not necessary that the linked pages reside on the same host as the source page; the cache agent can retrieve them even if they reside on other hosts.

The cache agent offers a very useful service: caching is performed even before cached pages are effectively requested, so that the average response time is minimized. Moreover, the cache is built before user activity increases, typically at night. However, turning on the cache agent forces the caching and filtering proxy server machine to be busy even during hours of low activity. Moreover, configuring the cache agent to perform delving requires giving more control to the caching and filtering proxy server administrator. For example, disabling delving from high-level pages, such as Web indexes or search sites, is recommended; otherwise, multiple requests for large numbers of pages will be generated.


How to set up automatic cache refreshingThe following figures demonstrate how to set up a cache refresh agent using the Configuration and Administration forms. Follow these steps:

� You can specify the URLs to load. Choose Cache Configuration -> Cache Preload. The window shown below is displayed:

Figure 8-21 Cache Preload window

By default, the cache daily agent is enabled to run everyday at 3:00 AM. You specify which URLs to load with the option cache in the Cache status field. You can also exclude URLs from being preloaded with the option ignore in the same field.

We used the URL http://www.uol.com for this test. Click Submit to update the proxy configuration file. This directive requires you to stop and start the server, as shown in Table 7-3 on page 295.


� To configure other cache agent options and the number of popular URLs to load, choose Cache Configuration -> Cache refresh. The following window is displayed:

Figure 8-22 Cache Refresh window


To configure delving, select the Load linked pages field. You use a menu to configure how to apply to delve. To delve for all pages, choose always. For no pages, choose never. For administrator-specified pages only, choose admin. To delve for links within documents served only from the cache, choose topn. We used the default configuration, so we did not need to restart the server for these settings.

The cache agent loads and then refreshes the cache in the following order:

� Specific pages the administrator has specified

� Popular pages from the cache access log

� Additional pages, by following links from the pages that have already been loaded; these are retrieved if the maximum number of pages has not been reached.

Starting cache refreshing manuallyWhen the cache refresh is enabled, the cache refresh agent is automatically run at the specified time. However, you can run the cache agent manually at any time.

In AIX, the cache refresh agent executable is cacheagt. If you input this command on the screen, you receive a small description on how to use it.

Example 8-21 Cache agent syntax# ./cacheagt --Usage: cacheagt [ options ... ] where options include: -r <proxy config file> -vv <verbose mode>

The manual cache refresh agent commands by platform are listed in the following table.

Table 8-5 Cache agent commands

Platform command

AIX 4.3.3 /usr/sbin/cacheagt

Windows c:\program files\ibm\wte\bin\cacheagt.exe

Linux /usr/sbin/cacheagt

8.2.4 Caching of dynamic contentThe JavaServer Pages (JSP) and servlet caching function enables Web Traffic Express to cache JSP and servlet results generated from a WebSphere Application Server. WTE pulls the JSP results from the application server’s dynamic cache, which is configured with a WTE caching proxy plug-in module. The main advantages WTE offers by caching dynamically generated content are:

� Reducing the load on the content hosts

� Reducing the load on the application servers

� Speeding the delivery of requested resources to end users

The ability to dynamically cache JavaServer Pages and servlets was introduced in WebSphere Application Server V3.5 with PTF3. WAS uses an in-memory cache to improve performance. The dynamic cache function of WAS is enabled if the following file is present within WAS:

/WebSphere/AppServer/properties/dynacache.xml

A specific servlet is enabled for caching by adding an entry for it in:

/WebSphere/AppServer/properties/servletcache.xml

Web Traffic Express communicates with WebSphere Application Server using an external cache adapter (ECA) plug-in provided with WTE. The following figure illustrates an overview of the dynamic caching support included in WTE and its relationship with the WAS dynacache.

Figure 8-23 WTE dynamic caching with WAS

dynacache

WTE ExternalCache Adapter

WTE WAS

cache

Web

Server

JSPsServlets

Check for cachecontrol header

Invalidate cache entries

InitialRequest

Response

Client


Currently, the external cache interface exports only fully composed public pages (cannot use cookies in the session information). Private pages are not cached by the proxy.

The following table lists a matrix of WebSphere Application Server and Web Traffic Express dynacache compatibility.

Table 8-6 WAS and WTE dynacache support

Configuring WTE and WAS for dynamic cachingThe Configuration and Administration forms are not available for configuring this feature. A text editor must be used to edit the ibmproxy.conf, dynacache.xml and servletcache.xml files.

We used the following configuration for our scenario:

Note: Dynacache functionality is supported in WebSphere Application Server 3.5 with PTF3 applied.

WAS version V3.5.3 V3.5.4 and V4.0

WTE V3.6 (WSES V1.0.3) Dynacache support NO dynacache support

WTE V3.6.01 Dynacache support NO dynacache support

WTE V3.6.0.1 plus fix NO dynacache support Dynacache support


Figure 8-24 Dynamic caching scenario

We followed these steps to configure WTE dynamic cache support.

� First, we created a Web application called ITSO in the WebSphere Application Server. We included two example servlets:

– CalcServlet - calculates the timestamp and calls TimeServlet

– TimeServlet - calculates the timestamp

CalcServlet is defined as cacheable, while TimeServlet is not. Using these servlets, we can have two different timestamps displayed at the same moment. The following window shows our WebSphere Application Server configuration.

9.24.105.61RS600034

WTE ProxyServer

WEBClients

9.24.105.82M238P4YL

9.24.105.39M23KK904

WASServer


Figure 8-25 WebSphere Application Server configuration

The source code for TimeServlet.java and CalcServlet.java can be found in Appendix A, “Java source code” on page 475

� The dynamic caching function of Web Traffic Express includes two JavaBean components: WteAdapter.class and WteMetaDataGeneratorImpl.class, located in <WTE_root>/classes/com/ibm/wte. You must copy these class files into <WAS_root>/properties. We used FTP to send these .class files to the WAS server. The example below shows the steps to follow:

Example 8-22 Transferring necessary filesc:> hostnameM23KK904c:> cd \WebSphere\AppServer\propertiesc:\WebSphere\AppServer\properties> ftp rs600034Connected to rs600034.itso.ral.ibm.com.

220 rs600034 FTP server (Version 4.1 Thu Apr 6 14:11:28 CDT 2000) ready.Name (rs600034:root): root331 Password required for root.Password:230 User root logged in.ftp> cd /opt/IBMWTE/usr/internet/classes/com/ibm/wte250 CWD command successful. ftp> bin200 Type set to I.ftp> mget *.classmget WteAdapter.class? y200 PORT command successful.150 Opening data connection for WteAdapter.class (6550 bytes).226 Transfer complete.6550 bytes received in 0.001107 seconds (5778 Kbytes/s)local: WteAdapter.class remote: WteAdapter.classmget WteMetaDataGeneratorImpl.class? y200 PORT command successful.150 Opening data connection for WteMetaDataGeneratorImpl.class (1682 bytes).226 Transfer complete.1682 bytes received in 0.000357 seconds (4601 Kbytes/s)local: WteMetaDataGeneratorImpl.class remote: WteMetaDataGeneratorImpl.classftp> quit221 Goodbye.

� The WebSphere Application Server communicates with WTE using an external cache adapter (ECA) supplied with WTE (WteAdapter.class file FTPed to the WAS in the previous step). On the WebSphere Application Server we must edit dynacache.xml and add an externalCacheGroup entry to enable the WTE proxy’s ECA. WAS reads the dynacache.xml file at startup.

WAS provides an example file named dynacache.sample.xml to help identify how to add plug-in entries. The following table describes where this sample file is located.

Table 8-7 Sample dynacache.xml file location

We copied this file to a new file called dynacache.xml. We then edit this file and add the WTE externalCacheGroup plug-in entry.

Example 8-23 Updated dynacache file<?xml version="1.0"?>

<cacheUnit>

Platform path

AIX \usr\WebSphere\AppServer\properties

Windows c:\websphere\properties

<cache size="1000" priority="1" /> <externalCacheGroups> <group id="IBM-WTE-ITSO" > <member address="rs600034.itso.ral.ibm.com"

adapterBeanName="com.ibm.wte.WteAdapter" /> </group> </externalCacheGroups></cacheUnit>

� To enable WAS dynamic caching of servlets, you must have entries for them in the servletcache.xml file on the WebSphere Application Server. In addition, for Web Traffic Express to cache a servlet’s results, the servlet must be marked as externally cacheable in the servletcache.xml file.

The servletcache.xml file is located in the same directory as the dynacache.xml file described in Table 8-7 on page 363. There is an example file called dynacache.sample.xml to help configure this entry. We copied this file to a new file called serveltcache.xml, edited it, and input the servlet entries. The following example lists our file:

Example 8-24 serveltcache.xml file

<?xml version="1.0" ?> <!DOCTYPE servletCache (View Source for full doctype...)> <servletCache>

<servlet> <metagenerator class="com.ibm.wte.WteMetaDataGeneratorImpl" /> <externalcache id="IBM-WTE-ITSO" /> <path uri="/webapp/ITSO/CalcServlet" /> <timeout seconds="100" /> </servlet></servletCache>

� Edit the WTE ibmproxy.conf file (this file is located as described in Table 7-2 on page 295). Find and uncomment the following directives which enable Web Traffic Express to receive invalidate messages sent by the WebSphere Application Server:

For an AIX platform:

Example 8-25 JSP caching function on AIX# ===== JSP Plugin =====Service /WES_External_Adapter /opt/IBMWTE/usr/internet/lib/plugins/dynacache/ libdyna_plugin.o:exec_dynacmd

Note: One application server can be configured to serve multiple proxy servers simply by adding additional group id statements.

For a Windows platform:

Example 8-26 JSP caching function on Windows# ===== JSP Plugin =====Service /WES_External_Adapter C:\Progra~1\IBM\WTE\bin\plugins\dynacache\ dyna_plugin.dll:exec_dynacmd

For a Linux platform:

Example 8-27 JSP caching function on Linux# ===== JSP Plugin =====Service /WES_External_Adapter /usr/lib/libdyna_plugin.so:exec_dynacmd

For a Solaris platform:

Example 8-28 JSP caching function on Solaris# ===== JSP Plugin =====Service /WES_External_Adapter /opt/IBMWTE/usr/internet/lib/plugins/dynacache/libdyna_plugin.so:exec_dynacmd

� Add a directive for each external cache manager in the ibmproxy.conf file. The syntax is as follows:

ExternalCacheManager <ExternalCacheManager ID> <MaxExpiryTime>

The ExternalCacheManager ID must match the ID set in the externalCacheGroups: group id attribute in the dynacache.xml config file in the WAS (refer to Example 8-23 on page 363). The MaxExpiryTime is the default expiration time set for resources cached on behalf of the external cache manager. If the external cache manager fails to invalidate a resource within the specified time, the resource will expire automatically in the specified time. We edited the ibmproxy.conf file and added the following line:

ExternalCacheManager IBM-WTE-ITSO 20 minutes

WTE will classify all the cached entries as expired in 20 minutes. WTE caches only contents from a WAS that has an ExternalCacheManager matching that which is specified in the ibmproxy.conf file.

� The WebSphere Application Server sends an invalidate message to all Web Traffic Express servers defined in the externalCacheGroup entry in the dynacache.xml file. When the JSP or servlet results are invalidated, WTE clears this invalid entry from its cache. Entries in the dynamic cache can be invalidated when:

– The dynamic cache garbage collector removes an entry during cache congestion

– The timeout set in the servlet entry expires

– An external agent or application invokes the dynamic cache API to invalidate cache entries.

� Stop the WTE proxy server and restart it again, as described in Chapter 6, “Web Traffic Express installation” on page 247.

The following considerations should be kept in mind when using this feature:

� Dynacache will export only fully cacheable pages.

� Pages must be public.

� Invalidate messages delivery is not reliable.

� No secure connection for invalidates.

Testing the configurationTo test our configuration, we customized the Web client’s browser (9.24.105.82) to access the Web Traffic Express proxy server (RS600034), as shown in 7.2.1, “Configuring the browser” on page 309. We made some requests to http://9.24.105.39/webapp/ITSO/CalcServlet. We receive the following window in response:

Figure 8-26 Timestamp window


The CalcServlet was cached while TimeServlet was reexecuted.

8.3 SOCKS client supportFlexible-client SOCKS allows the caching and filtering proxy server to reside behind a firewall or SOCKS server without sharing the same physical machine with this firewall or SOCKS server. With this feature, the WTE server does not have to forward all the requests to the SOCKS server. On the contrary, requests for Web content located in the intranet can be routed directly to the destination content server. This way, workload on the SOCKS server is dramatically reduced, and the end user will experience a better response time. The following two figures illustrate the benefit of using a flexible-client SOCKS WTE proxy server.

Figure 8-27 Client intranet access without flexible-client SOCKS support

ProxyServer

SOCKS/Firewall

WebServer

InternetIntranet

WEBClient


Figure 8-28 Client intranet with flexible-client SOCKS support

The flexible-client SOCKS enhancement implemented in WTE improves security by allowing a firewall server to be isolated from the proxy server. It reduces the load on the firewall, since the internal requests are handled by the proxy server. Moreover, the administrator can easily specify which IP addresses or domains should be contacted directly by the proxy server and which ones should be contacted through the SOCKS server.

If your system does not already have a SOCKS configuration file, a default SOCKS configuration file will be installed with Web Traffic Express.

Table 8-8 Location of the SOCKS configuration file

8.3.1 How to set up flexible-client SOCKSWe will use the following configuration for our scenario:

Note: WTE is a SOCKS4 client and does not include a SOCKS server. A SOCKS server is included in the Tivoli SecureWay Firewall.


Windows NT 4.0 c:\program files\ibm\wte\socks.cnf

AIX 4.3.3 /etc/socks.conf (the file in this directory is a link to /opt/IBMWTE/usr/internet/etc/en_US/etc/socks.conf.exp)

Linux Red Hat 6.2 /etc/socks.conf

ProxyServer

SOCKS/Firewall

WebServer

InternetIntranetWEB

Client


Figure 8-29 Flexible-client SOCKS scenario

We will detail the steps to set up a flexible-client SOCKS environment for our network configuration, using the Configuration and Administration forms:

� Configure the direct (intranet) connections. Use direct connections for requests that should not pass through the SOCKS server. Choose Proxy

9.24.105.61RS600034

WTE ProxyServer

9.24.105.59RS60008

WEBClients

9.24.105.82M238P4YL

WebServer

SOCKS/FirewallIntranet

Traffic

Internettraffic

WebServer

socks.itso.ral.ibm.com

InternetIntranetIntranet


Configuration -> SOCKS Configuration -> Direct Connections. We see the window shown in Figure 8-30:

Figure 8-30 Direct connections configuration

We input the IP address 9.24.105.0, Subnet Mask 255.255.255.0; we set Port to Equal to and entered Port Number 80. Therefore, all HTTP (port 80) traffic within this network will have direct connections. Click Submit to update the SOCKS configuration file.

� Configure SOCKS (Internet) connections. Use SOCKS connections for requests that should pass through the SOCKS server. Choose Proxy


Configuration -> SOCKS Configuration -> SOCKS Connections. We see the window shown in Figure 8-31.

Figure 8-31 SOCKS connections configuration


We input the IP address 0.0.0.0, Subnet Mask 0.0.0.0, we set Port to All ports and we entered the SOCKS server host name socks.itso.ral.ibm.com. Therefore, traffic requests for all the other networks will pass through the SOCKS server. Click Submit to update the SOCKS configuration file. Click the Restart icon to restart the WTE server.

8.3.2 Testing the configurationTo test if our environment is configured correctly, we customized the Web client’s browser (9.24.105.82) to access the Web Traffic Express proxy server (RS600034), as described in 7.2.1, “Configuring the browser” on page 309. We performed an intranet access request and an Internet access request.

� For intranet access, we entered the URL http://RS60008/WSsamples/index into the Web client’s browser. It returned the WAS index home page. The following is a log captured by the tcpdump command on the Web Traffic Express proxy server machine, showing the communication between machines:

Example 8-29 Intranet access# tcpdump -n tcptcpdump: listening on en0 11:18:59.788124229 9.24.105.82.3077 > 9.24.105.61.80: P608128506:608128867(361) ack 3979850168 win 7334 (DF)11:18:59.789552182 9.24.105.61.32957 > 9.24.105.59.80: S4235327931:4235327931(0) win 16384 <mss 1460>11:18:59.790066792 9.24.105.59.80 > 9.24.105.61.32957: S2814790083:2814790083(0) ack 4235327932 win 16060 <mss 1460>11:18:59.790150186 9.24.105.61.32957 > 9.24.105.59.80: . ack 1 win 1606011:18:59.790298159 9.24.105.61.32957 > 9.24.105.59.80: P 1:402(401) ack 1 win1606011:18:59.792816445 9.24.105.59.80 > 9.24.105.61.32957: P 1:1299(1298) ack 402 win 1606011:18:59.793818765 9.24.105.61.80 > 9.24.105.82.3077: P 1:1389(1388) ack 361 win 16060

The captured trace data shows that all traffic from the Web client browser (9.24.105.82) goes through the WTE proxy server (9.24.105.61) and is answered by the intranet Web Server(9.24.105.59). This verifies that the 9.24.105.0 network’s traffic was not sent to the SOCKS server.

� For Internet access, we entered the URL http://www.cnn.com on the Web client’s browser. It returned the CNN main page. The following is a log captured by tcpdump on the Web Traffic Express proxy server.

Example 8-30 Internet access# tcpdump -n tcp13:33:53.208493801 9.24.105.82.3198 > 9.24.105.61.80: P616538424:616538864(440) ack 4260219018 win 7370 (DF)13:33:53.209958470 9.24.105.61.33215 > 9.14.3.72.1080: S1966035796:1966035796(0) win 16384 <mss 1460> (DF)13:33:53.281441233 9.14.3.72.1080 > 9.24.105.61.33215: S2404853300:2404853300(0) ack 1966035797 win 16060 <mss 1460>13:33:53.281529391 9.24.105.61.33215 > 9.14.3.72.1080: . ack 1 win 16060 (DF)13:33:53.281630783 9.24.105.61.33215 > 9.14.3.72.1080: P 1:9(8) ack 1 win 16060(DF)13:33:53.385213417 9.24.105.61.80 > 9.24.105.82.3198: . ack 440 win 1606013:33:53.496049862 9.14.3.72.1080 > 9.24.105.61.33215: . ack 9 win 1606013:33:53.496122573 9.24.105.61.33215 > 9.14.3.72.1080: P 9:13(4) ack 1 win16060 (DF)

From data captured, we see that the request destined for the Internet Web server is sent by the WTE proxy server (9.24.105.61) to the SOCKS server (9.14.3.72).

8.4 Proxy auto-configurationWeb Traffic Express supports automatic proxy configuration, a technology that matches a feature implemented in Netscape Navigator 2.0 or later and Microsoft Internet Explorer Version 4.0 or later. Automatic proxy configuration implies that the client browser does not need to be configured to point to a specific proxy or SOCKS server, as long as it points to a proxy automatic configuration (PAC) file.

The advantage of this feature is that it allows the server administrator to update only this file if there is any change to the proxy or SOCKS server configuration. Client browsers need not be reconfigured. This feature can also be used to reroute requests when servers are down, to balance workload or to send specific URLs to specific proxies.

Automatic proxy configuration is a browser function that enables more dynamic server selection. A PAC file is a JavaScript file that consists of functions called by the client browser before each URL retrieval. The function will return values indicating whether a proxy server, a SOCKS server, or a direct connection will be used to service the request. This file can also redirect the request if the initial connection to be used is down.

8.4.1 How to write a PAC fileThe proxy automatic configuration file must be named xxxx.pac and defines the FindProxyForURL function:

Example 8-31 PAC file syntaxfunction FindProxyForURL(url, host){...}

We will define a proxy automatic configuration file for the following scenario (the machines are described on Table 7-1 on page 288).

Figure 8-32 Autoproxy scenario

The following factors are defined in our scenario:

Note: The PAC file is reloaded when a browser is restarted.

WTE ProxyServers

9.24.105.61RS600034

9.24.105.83M238P4XL

WEBClients

9.24.105.39M23KK904

9.24.105.82M238P4YL

FirewallFirewall WebServer

Internet

9.24.105.63RHLINUX1

Intranet


� The client browser with IP address 9.24.105.39 (M23KK904) must access the intranet/Internet through the proxy server with IP address 9.24.105.83 (M238P4XL) as first option.

� The client browser with IP address 9.24.105.82 (M238P4YL) must access the Intranet/Internet through the proxy server with IP address 9.24.105.61 (RS600034) as first option.

� The proxy server with IP address 9.24.105.63 contains the autoproxy file configuration and will be the backup proxy server.

The script shown in the following example will instruct the browsers to route requests as described above.

Example 8-32 File autoproxy.pacfunction FindProxyForURL(url, host){hostip=myIpAddress(); if (isInNet(hostip, "9.24.105.39","255.255.255.0")) { return "PROXY 9.24.105.83:80; " + "PROXY 9.24.105.63:80; " + "DIRECT"; } else if (isInNet(hostip,"9.24.105.82", "255.255.255.0")) { return "PROXY 9.24.105.61:80; " + "PROXY 9.24.105.63:80; " + "DIRECT"; } else { return "PROXY 9.24.105.63:80; " + "DIRECT"; } }

In our customization, if the primary proxy fails, the client browser will automatically point to the secondary proxy server. This is indicated in the PAC file as the proxy server with IP address 9.24.105.63. If both the primary and secondary proxies are unavailable, the browser will attempt to connect using a direct connection to the Internet.

For those client IP addresses not defined in the PAC file, they will access the Internet through the proxy server with IP address 9.24.105.63.


8.4.2 How to set up an automatic proxyNow we will describe how to configure this option using the Configuration and Administration forms. The Configuration and Administration forms’ support for automatic proxy configuration is limited. Choose Proxy Configuration -> Proxy Auto-Configuration. The following window will be displayed:

Figure 8-33 Proxy Auto-Configuration window

We selected the Create new file radio button and entered a PAC file name, autoproxy.pac. Click Submit to create the file. The following window will be displayed:


Figure 8-34 Proxy Auto-Configuration window, next step

In this window, we defined the primary and secondary proxy servers as 9.24.105.83 and 9.24.105.63. We checked the Route DIRECT option for the browser to attempt a direct connection if both proxies servers are down. Click Submit to write the configuration changes to the autoproxy.pac file. After processing the form, Web Traffic Express displays the PAC file name in the Proxy Auto-Configuration form, as shown in Figure 8-35.


Figure 8-35 Proxy Auto-Configuration window, next step


The autoproxy file created using the steps listed above is shown in the following example:

Example 8-33 Autoproxy.pac file generated by Configuration and Administration forms//Created by WTE remote config-DO NOT EDIT//PRI PROXY 9.24.105.83 80 END//SEC PROXY 9.24.105.63 80 END//DIR yes ENDfunction FindProxyForURL(url, host) { return "PROXY 9.24.105.83:80; " + "PROXY 9.24.105.63:80; " + "DIRECT"; }

The file generated by the Configuration and Administration forms does not contain all the necessary entries to allow us to configure our scenario. We had to edit the autoproxy.pac file to add the entries as shown in Example 8-32 on page 375.

The locations of the proxy automatic configuration file are listed in Table 8-9.

Table 8-9 Location of the PAC file

8.4.3 TestsTo test if our autoproxy environment is working as planned, we must customize our client browser and access a URL.

Configuring the browserWe configured Netscape Communicator 4.7 and Microsoft Internet Explorer 5 on the client machines with IP addresses 9.24.105.82 and 9.24.15.39.

Netscape browserTo configure the Netscape browser, click Edit -> Preferences and open the Advanced option. Choose Proxies, click the Automatic Proxy Configuration, and enter the automatic proxy configuration URL. You will see the following window:


Windows c:\Program Files\IBM\WTE\HTML\pacfiles

AIX 4.3.3 /opt/IBMWTE/usr/internet/server_root/pub/pacfiles

Linux Red Hat 6.2 /opt/IBMWTE/usr/internet/server_root/pub/pacfiles


Figure 8-36 Netscape browser preference settings

We entered the URL http://9.24.105.63/pacfiles/autoproxy.pac. This URL is where the PAC file is located, as defined in our scenario. We then clicked the Reload button to activate the changes.


Microsoft Internet Explorer browserTo configure the Microsoft Internet Explorer browser, click Tools -> Internet Options... and open the Connections tab. Click the LAN Settings... button. You will see the following window:

Figure 8-37 Microsoft Internet Explorer browser LAN settings

We entered the URL http://9.24.105.63/pacfiles/autoproxy.pac. This URL is where the PAC file is located, as defined in our scenario. Click OK to activate the changes

Testing the accessWe followed the steps below to test our configuration.

Client machinesWe entered the URL http://www.cnn.com in the browser for the client machines with IP addresses of 9.24.105.82 and 9.24.105.39.


� The request from the client machine with IP address 9.24.105.82 was taken by proxy server 9.24.105.61- RS600034. To verify this, we viewed the access log on this proxy server. In the Configuration and Administration forms, open Server Activity Monitor -> Proxy Access Statistics. You see the following window:

Figure 8-38 The Proxy Access Statistics information window

You can see information about which URLs were requested on the WTE proxy server RS600034. The first column contains the IP address of the machine that requested the URL. In this case, the request was from the client with IP address 9.24.105.82.

� The request of the client machine with IP address 9.24.105.39 was taken by the proxy server, 9.24.105.83 - M238P4XL. To verify this, we viewed the access log on this proxy server. In the Configuration and Administration forms, open Server Activity Monitor -> Proxy Access Statistics. You see the following window:


Figure 8-39 The Proxy Access Statistics information window, next step

You can see information about which URLs were requested on the WTE proxy server M238P4XL. The first column contains the IP address of the machine that requested the URL. In this case, the request was from the client with IP address 9.24.105.39.

Next, we disabled the proxy server with IP address 9.24.105.83 - M238P4XL. We then entered the URL http://www.cnn.com in the browser for the client machine with IP address 9.24.105.39.

� The request from this machine was taken by the secondary proxy server with IP address 9.24.105.63 - RHLINUX1, as defined in the PAC file. We can confirm this by opening Server Activity Monitor -> Proxy Access Statistics


in the Configuration and Administration forms on this proxy server. You see the following window:

Figure 8-40 Process Access Statistics window, final step

You can see that the first column contains the IP address of our Web client (24.105.39). It is defined to access the WTE proxy server RS600034. However, this WTE server is disabled. The client requirement was automatically sent to the secondary WTE proxy server, RHLINUX1.

8.5 Proxy chainingProxy chaining is a function that enables you to chain proxies together and to identify domains to which requests should be passed without going through a proxy.

Note: Proxy chaining is supported in a forward proxy environment only.


A client sends its request first to the local WTE server. If the local WTE server does not find the file in its cache, it sends a proxy request to another proxy at a higher level. This WTE server, in turn, sends the request to a higher level proxy in the chain, and so forth. The last proxy in the chain will direct the request to the Internet and return the requested content back through the chain to the local WTE server, which returns it to the client. All the proxies in the chain have the option of caching Web files.

These are the advantages to using proxy chaining:

� Proxies at a lower level, which are closer to the client that originated the request, benefit from the caches of the higher level proxies.

� Proxy chaining reduces the load on the highest level proxy (typically, the proxy closest to the firewall) and ultimately on the content server, since lower level proxies may already have the document cached.

� The larger the number of users, the higher the probability that a proxy server already has a Web document in its cache. Considering that high-level proxies serve a larger number of clients, many requests that cannot be honored by a low-level proxy can be resolved by higher level proxies, since other groups of clients may have already requested the same files.

These are the disadvantages:

� A high-level proxy should have a larger cache, since it has to honor the requests of a large number of users.

� Proxy chaining greatly increases response time for requests, especially for noncacheable files or for those cacheable files that have not yet been cached by any proxies in the hierarchy.

� The risk of failure in a chain increases with each additional node.

WTE allows the creation of proxy chains based on the protocol of the request to be forwarded (HTTP, FTP or Gopher). In other words, a proxy server can be configured to send all incoming requests with a particular protocol to a higher level proxy server in the chain. At the same time, you can specify the domains to which requests should be passed without going through a proxy.

Note: The more proxies you chain together, the greater the latency you will introduce. It is recommended that you chain together no more than two proxies.


8.5.1 How to set up proxy chainingWe will use the following configuration for our test scenario:

Figure 8-41 Proxy chaining scenario

We will set up our proxy chaining scenario using the Configuration and Administration forms.

In the WTE Proxy Server with IP address 9.24.105.61 (RS600034), choose Proxy Configuration -> Proxy Chaining and Non-Proxy Domains. You will see the window shown in Figure 8-42:

9.24.105.61RS600034

WTEProxy

Servers

9.24.105.82M238P4YL

Web ContentServer

9.24.105.59RS60008

WebServer

FirewallFirewall

socks.itso.ral.ibm.com

WEBClients

9.24.105.39M23KK904

9.24.105.63RHLINUX1

Intranet Internet


Figure 8-42 Proxy Chaining and Non-Proxy Domains window

We configured this proxy server, RS600034, to forward all HTTP requests that are not in its own cache to the second-level proxy, RHLINUX1. We also defined a non-proxy domain configuration, RS6008.itso.ral.ibm.com. This is the Web Server which we would not need a proxy to access.


8.5.2 Testing the configurationTo test if our environment is working as expected, we customized the two Web clients’ browsers as shown in 7.2.1, “Configuring the browser” on page 309:

� 9.24.105.82 to access the Web Traffic Express proxy server RS600034

� 9.24.105.39 to access the Web Traffic Express proxy server RSLINUX1

The first access we entered was a URL in the browser on Web client 9.24.105.39. We made a request to URL http://www.rural.com.br. Since this client is configured to use the RHLINUX1 proxy server, the request is taken by this proxy server. This proxy server does not find the file in its own cache file, so the request is sent to the SOCKS server (9.14.3.72) and returns the results to the proxy server RHLINUX1. This proxy server caches this request, and also sends it to the Web client’s browser. The following examples show the cache access log and a log captured by tcpdump.

No cache access was performed on RHLINUX1. As can be seen in Example 8-34, the proxy cache is empty.

Example 8-34 RHLINUX1 empty proxy cache.

[root@rhlinux1 /]# hostname rhlinux1[root@rhlinux1 /]# cd /opt/IBMWTE/usr/internet/server_root/logs/[root@rhlinux1 logs]# tail -f cache.Mar202001

The log captured by tcpdump shows the request being sent to SOCKS server.

Example 8-35 tcpdump log file on RHLINUX115:34:56.029598 eth0 < 9.24.105.39.1285 > 9.24.105.63.www: P 1:291(290) ack 1 win 8760 (DF)15:34:56.029651 eth0 > 9.24.105.63.www > 9.24.105.39.1285:. 1:1(0) ack 291 win31830 (DF)15:34:56.209796 eth0 > 9.24.105.63.3640 > 9.14.3.72.socks: S 1406934733:1406934733(0) win 32120 <mss 1460,sackOK,timestamp 7663818 0,nop,wscale 0> (DF)15:34:59.208525 eth0 > 9.24.105.63.3640 > 9.14.3.72.socks: S 1406934733:1406934733(0) win 32120 <mss 1460,sackOK,timestamp 7664118 0,nop,wscale 0> (DF)15:35:05.208520 eth0 > 9.24.105.63.3640 > 9.14.3.72.socks: S 1406934733:1406934733(0) win 32120 <mss 1460,sackOK,timestamp 7664718 0,nop,wscale 0> (DF)15:35:17.208518 eth0 > 9.24.105.63.3640 > 9.14.3.72.socks: S 1406934733:1406934733(0) win 32120 <mss 1460,sackOK,timestamp 7665918 0,nop,wscale 0> (DF)15:35:17.254737 eth0 < 9.14.3.72.socks > 9.24.105.63.3640: R 0:0(0) ack 1406934734 win 015:35:17.257990 eth0 > 9.24.105.63.www > 9.24.105.39.1285: P 1:1461(1460) ack 291 win 32120 (DF)15:35:17.258178 eth0 > 9.24.105.63.www > 9.24.105.39.1285: FP 1461:2033(572) ack 291 win 32120 (DF)

For the second access, we entered the same URL on the other Web client browser, 9.24.105.82. Since this client is configured to use the RS600034 proxy server, the request is taken by this proxy server. This proxy server does not find the file in its own cache, so it sends this request to the next highest level proxy server in the chain, which is the proxy server RHLINUX1. The proxy server RHLINUX1 looks up its own cache and finds the Web page there. It then sends this Web page back to proxy server RS600034. The proxy server RS600034 caches the Web page and sends it to the Web client’s browser. The following examples show the cache access log of the two proxy servers and a log captured by tcpdump.

As can be seen in Example 8-36, the proxy cache on RS600034 is empty.

Example 8-36 RS600034 empty proxy cache

# hostname rs600034# cd /wte/logs# tail -f cache.Mar202001

The log captured by tcpdump on the RS600034 shows the request being sent to another proxy server, RHLINUX1.

Example 8-37 tcpdump log file on RS600034# tcpdump -n tcp17:39:14.659175488 9.24.105.82.1533 > 9.24.105.61.80: P 1:425(424) ack 1 win 8760 (DF)17:39:14.669557379 9.24.105.61.37871 > 9.24.105.63.80: S 2311065038:2311065038(0) win 16384 <mss 1460>17:39:14.669693755 9.24.105.63.80 > 9.24.105.61.37871: S 1129326489:1129326489(0) ack 2311065039 win 32120 <mss 1460> (DF)17:39:14.669771519 9.24.105.61.37871 > 9.24.105.63.80:. ack 1 win 1606017:39:14.669921225 9.24.105.61.37871 > 9.24.105.63.80: P 1:496(495) ack 1 win 1606017:39:14.670173814 9.24.105.63.80 > 9.24.105.61.37871:. ack 496 win 31625 (DF)17:39:14.690600987 9.24.105.82.1534 > 9.24.105.61.80: S 26876359:26876359(0) win 8192 <mss 1460> (DF)17:39:14.690701416 9.24.105.61.80 > 9.24.105.82.1534: S 1816308422:1816308422(0) ack 26876360 win 16060 <mss 1460>

There was access to the cache in the proxy server RHLINUX1 for proxy server 9.24.105.61 (RS600034) as can be seen in Example 8-38.

Example 8-38 RHLINUX1 proxy cache

[root@rhlinux1 /]# hostname rhlinux1[root@rhlinux1 /]# cd /opt/IBMWTE/usr/internet/server_root/logs/[root@rhlinux1 logs]# tail -f cache.Mar2020019.24.105.61 - - [20/Mar/2001:18:03:23 +0500] “GET http://www.rural.com.br/ HTTP/1.0” 200 18319.24.105.61 - - [20/Mar/2001:18:03:24 +0500] “GET http://www.rural.com.br/index.swf HTTP/1.0” 200 24674

The third example demonstrates accessing a non-proxy domain. This means that we specify a URL in the client browser that is in the domain to which the WTE server should directly connect, and only used when you are doing proxy chaining. This directive does not apply when the proxy goes through a SOCKS server; use socks.conf for that purpose. In our scenario, we defined the Web Server RS60008 as a non-proxy domain (refer to Figure 8-42 on page 387). We requested a Web page from this Web server, http://rs60008, from the Web client’s browser, 9.24.105.82. The request is sent to the proxy server. The proxy server does not find the file in its own cache file. Instead of forwarding the request to the second-level proxy, it sends it to Web Server RS60008 directly. The Web server returns the results to the proxy server RS600034. The proxy server on RS600034 sends the response to the Web client. So a proxy chaining environment supports a low-level proxy to forward requests to the Web server directly instead of using a higher-level proxy.

8.5.3 Configuration of the ibmproxy.conf fileThe following example shows the directives that were updated in the ibmproxy.conf file on RS60034 to configure the proxy chaining.

Example 8-39 The directives for proxy chaining on RS60034http_proxy http://rhlinux1.itso.ral.ibm.com/no_proxy rs60008.itso.ral.ibm.com

8.6 Logging and performanceIn this section, we show you how to use the Server Activity Monitor of Web Traffic Express. This tool monitors statistics and access log entries, and uses the statistics to assist the WTE administrator in the tuning of WTE for performance improvements. We also describe how the data collected from the Server Activity Monitor can be used to tune WTE for better performance.


8.6.1 Activity MonitorThe Server Activity Monitor enables you to display:

� Activity statistics

� Network statistics

� Access statistics

� Proxy access statistics

� Cache statistics

� Cache refresh summary

Note: The server creates new log files every day at midnight. If the server is not running at midnight, new logs are created when it is first started the next day.


Activity statisticsThe Activity Statistics page provides information on the number of threads (connections) used and available, server response time, throughput, number of requests processed and also number of errors. To view the Activity Statistics page, click Server Activity Statistics -> Activity Statistics in the Configuration and Administration forms (see Figure 8-43). Click Refresh to update the information.

Figure 8-43 Activity Statistics window


Network statisticsThe Network Statistics page provides information about the network on which the proxy is running, such as data rate, bytes sent and received, and bandwidth. To view the Network Statistics window, click Server Activity Monitor -> Network Statistics in the Configuration and Administration forms (see Figure 8-44). Click Refresh to update the information.

Figure 8-44 Network Statistics window


Access statisticsThe Access statistics page provides information found in the access log files, including users accessing your proxy, IP addresses, user IDs (for protected pages), date and time of access, and the HTTP method used (GET, PUT, POST, DELETE). To view the Access Statistics window, click Server Activity Monitor -> Access Statistics in the Configuration and Administration forms, as shown in Figure 8-45. Click Refresh to update the information.

Figure 8-45 Access Statistics window


Proxy access statisticsThe Proxy Access Statistics page provides information on the proxy activity, such as which URLs were requested and if they were retrieved from the cache (the blue lines). Following the URLs are the return codes given to the client, and the file size in bytes. To view the Proxy Access Statistics window, click Server Activity Monitor -> Proxy Access Statistics in the Configuration and Administration forms, as shown in Figure 8-46. Click Refresh to update the information.

Figure 8-46 Proxy Access Statistics window


Cache statisticsThe Cache Statistics page provides the status of the proxy cache, such as whether the cache is currently operational. For example, if the cache is currently operational, the Cache Statistics page will display the message:

The proxy cache is in normal operation

To view the Cache Statistics page, click Server Activity Monitor -> Cache Statistics in the Configuration and Administration forms, as shown in Figure 8-47. Click Refresh to update the information.

Figure 8-47 Cache Statistics window


Cache refresh summaryThe Cache Refresh Summary provides information about:

� Number of URLs specified in the configuration file that were refreshed

� Number of pages refreshed during the previous night’s cache access log

� Number of URLs that the cache agent refreshed

� Number of URLs remaining in the queue after reaching the maximum allowed number of URLs

The cache agent must have run at least once to display any information.

To view the Cache Refresh Summary page, click Server Activity Monitor -> Cache Refresh Summary.

8.6.2 Configuration for performance improvementsThis section describes the WTE configuration settings you can modify to tune your server.

Understanding performance settingsEach time your server receives a request from a client, it uses a thread to perform the requested action. If no threads are available, the server holds the request until more threads become available. The MaxActiveThreads directive in the WTE configuration file specifies the maximum number of active threads. Clients will experience slower response times or even failures if no active threads are available. The failure would be indicated by an error message such as:

Error 400 - Proxy ErrorUnable to connect to remote host or host not responding

Note: To run the cache agent, the hostname directive must be configured as described in “Configuration host name” on page 296


If the number of active connections is very low, you can lower the amount of virtual memory used by lowering the MaxActiveThreads setting. To change the active threads started by WTE, click Server Configuration -> System Management -> Performance. Click Submit to update the information after making any changes. The Performance window is shown in the following figure:

Figure 8-48 Performance window


The following factors may impact the server response time:

� Network speed� Traffic on the local area network (LAN)� Number of clients requesting pages from your server� Number of threads set on your server� Other applications running on your server� System resources

Change the values of the Persistent Connections Timeout and Maximum Requests fields to specify the characteristics of a persistent connection. A persistent connection allows the server to accept multiple requests and to send responses over the same TCP/IP connection.

By increasing the number of maximum requests, overall throughput will also be increased, because the server will not have to establish a separate TCP/IP connection for each request and response. Also, the TCP/IP connection is used more efficiently because of its characteristics. However, keep in mind that persistent connections require network bandwidth as well as a dedicated server thread.

The Persist timeout specifies the amount of time that the server will wait between client requests before cancelling a persistent connection.


To view the active threads started by WTE, click Server Activity Statistics -> Activity Statistics. Click Refresh to update the information. The following figure shows the Activity Statistics window of the WTE:

Figure 8-49 Activity Statistics window

Timeouts: closing connections automaticallyTimeouts are used to control the server processing time. The default values are appropriate for most requests and need not be changed unless necessary.


To change the timeout values of WTE, click Server Configuration folder -> System Management -> Timeouts. The Timeouts window is shown in the following figure:

Figure 8-50 Timeouts window

Notice that:

� The Input Timeout field is used to set the time allowed for a client to send a request after making a connection to the server. A client first connects to the server and then sends a request. If the client does not send a request within the amount of time specified in this field, the server drops the connection. Take note that if you are using persistent connections, the persistent connection timeout specifies the time to wait for the client to send another request.


� The Read Timeout field is used to specify the time allowed without network activity before the connection is cancelled.

� The Output Timeout field is used to set the maximum time allowed for your server to send output to a client. The time limit applies to requests for local files and to requests for which the server is acting as a proxy. However, it does not apply to requests that start a local CGI program. If the server does not send the complete response within the amount of time specified in this field, the server drops the connection.

� The Script Timeout field is used to set the time allowed for a CGI program started by the server to finish. When the time runs out, the server ends the program.

� The Persist Timeout field specifies the amount of time the server will wait between client requests before cancelling a persistent connection.


Configuring for proxy performanceClick Proxy Configuration -> Proxy Performance (see the following figure):

Figure 8-51 Proxy Performance window

Fill in the Proxy Performance form by clicking the following fields:

� Run as a pure proxy

Select this box if you want to increase proxy performance by strictly running a proxy server. The alternative would be to run WTE as a content server too. If you want to use caching, leave the box unselected.


� Allow persistent connections

Use this setting to allow clients to force the server to keep an open connection with them. This decreases the portion of the client's log time spent requesting documents from the proxy. However, maintaining persistent connections requires network bandwidth as well as a dedicated server thread. Do not allow persistent connections if your setup limits the number of available threads.

� Use SOCKS configuration file

Select this box if you want the proxy to look at the SOCKS configuration file to decide whether or not to connect through the SOCKS server, as described in “How to set up flexible-client SOCKS” on page 368.

� Send HTTP/1.0 to downstream servers

Select this if you have downstream servers which do not correctly handle requests from HTTP/1.1 clients.

� Run as a transparent proxy

Select this box if you want your WTE server to run as a transparent proxy.

To Identify how FTP URL paths should be resolved, select one of the following:

� absolute paths: this option resolves to fully-specified paths with respect to the root directory.

� relative paths: this option resolves with respect to the home directory.

After you have completed your selections, click Submit to make the changes to the configuration file and restart the WTE server.

8.6.3 WTE loggingWeb Traffic Express can be customized to provide access and error logs. The logs are specified by the following directives:

� AccessLog - used for logging local document requests.

� ErrorLog - used for logging any errors.

� ProxyAccessLog - used for logging proxy requests.

� CacheAccessLog - used for logging hits on the proxy cache (only valid if the WTE server is running as a proxy).

� EventLog - used for logging all events of the proxy server.

The directory where the logs are installed by default changes by platform, as shown in Table 7-4 on page 299.


The default log file names are shown in the following table:

Table 8-10 Default log file names

These logs are stored on disk at midnight every day; the server closes the current log files and creates new log files for the next day. If the server is not running at midnight, it starts a new log file the first time you start it on a given day. This feature cannot be changed.

When creating the file, the server uses the file name you specify and appends a date suffix. The date suffix is in the format Mmmddyyyy, where:

� Mmm are the first three letters of the month.

� dd is the day of the month.

� yyyy is the year.

Logs File name

AccessLog local

ErrorLog error

ProxyAccessLog proxy

CacheAccessLog cache


The path and name of these log files can be customized using the Log Files form. Click Server Configuration -> Logging -> Log Files, as shown below:

Figure 8-52 Log Files window

We selected the Log information to Syslog box as we also want send the log information to the underlying operating system. On AIX and Linux, the syslog file is /etc/syslog.conf. We configured it via the following steps:

� Add the following lines to the /etc/syslog.conf file:

user.err /wte/logs/wte.log # For error informationuser.info /wte/logs/wte.log # For access information


� Restart the syslogd daemon with the following command:

refresh -s syslogd

We changed the path of the log files to the file system /wte, which we created earlier. The default path is /usr/lpp/internet/server_root. So all log paths were changed to /wte/logs. Press Submit to update the configuration file. You must stop and start the WTE server (see Table 7-3, “Directives not refreshed upon restart” on page 295).

Log archivingLog archiving is recommended as a means to maintain WTE server log files. The Log Archiving form supports two log archiving methods:

� Compress

� Purge

You can also choose to have no log archiving by using the option None.

On our AIX platform, we set the Log Archiving method to Compress. In particular, we configured our WTE server to compress logs older than 30 days and delete them only after 90 days. The following command can be used to tar, compress and remove the log files:

tar -cf /tmp/log%%DATE%%.tar %%LOGFILES%% ; compress /tmp/log%%DATE%%.tar ; rm %%LOGFILES%%

The command must be entered on one line. This will tar all the logs from the specified date range, compress the resulting tar file, and delete the original logs.

Additional sample compress commands are included in ibmproxy.conf in the Compress Directives.


This configuration can be issued in the Log Archiving window, accessible by clicking Server Configuration -> Logging -> Log Archiving, as shown below:

Figure 8-53 Log Archiving window

Access log exclusionsAccess log exclusions allow you to control what is logged. You can exclude data based on specific directories and/or files, user agents, host names or IP addresses, methods, MIME types, and return codes. Click Server Configuration -> Logging -> Access Log Exclusions. As this is a long form, we will show it in sections.


Let us suppose that we do not want to log the requests to the /Docs and /admin-bin directories, which are the documentation and forms directories, respectively, as well as the /tunetips.html file. In this case, we will add the entries as shown in the form section below:

Figure 8-54 Access Log Exclusions window- Excluded Directories/Files section

We do not add any entry in the user agent section. In this section, we can define which user agent we do not want to log.


The following window shows that we can configure our WTE server to not log requests coming from the machine with IP address 9.179.105.82. Notice that the wildcard character (*) is accepted.

Figure 8-55 Access Log Exclusions window- Excluded Host section


Since we do not want to log requests for files of the image/gif type, we only select the image/gif box, as shown in the last section of the Access Log Exclusions form, shown below:

Figure 8-56 Access Log Exclusions window- last section


Part 4 Combining Network Dispatcher and Web Traffic Express

Part 4


Chapter 9. Multiple Web Traffic Express proxy servers

In this chapter, we discuss using several Web Traffic Express proxy servers to process high traffic demand on a network, and two main ways to do this.

We describe the configuration of a scenario using Network Dispatcher to load balance the back-end Web Traffic Express proxy servers.

9


9.1 Using Autoproxy or Network DispatcherIn today’s networking environment, there is great demand for fast Internet access. Using a proxy cache will improve the response time for your Internet users, but what can you do if a single proxy server is not enough?

In 8.4, “Proxy auto-configuration” on page 373, we discussed the configuration of a feature called Autoproxy. This feature allows you to work with multiple Web Traffic Express proxy servers and to define which client IP addresses will use each proxy server. This is a fixed load balancing, since it does not analyze any data from the servers in order to actually balance the load equally among them.

For a small- or medium-sized network, this would be a good choice, since the configuration is simple, and you will work with only one product (no need for extra machines).

For large networks or for heavy traffic networks, we recommend using Network Dispatcher to load balance proxy servers.

Network Dispatcher can balance the load among all proxy servers, and using the WTE advisor it can analyze the response time for each server and use this information when calculating the server weight.

You can work with Network Dispatcher and Web Traffic Express on the same machine (except on a Windows system). This is a collocation scenario. However, we recommend you use a separate machine for Network Dispatcher, for better performance.

In the next sections, we describe a simple scenario showing Network Dispatcher load balancing multiple Web Traffic Express proxy servers.


9.2 Scenario descriptionIn our scenario, we set up two Web Traffic Express proxy servers and one Network Dispatcher, which will load balance all browser requests to these two proxy servers (see Figure 9-1).

Figure 9-1 Network Dispatcher load balancing Web Traffic Express proxy servers

This scenario works like the scenario described in 4.1, “Load balancing” on page 70, except that the Dispatcher will be load balancing proxy servers instead of Web servers.

All browsers and clients must be configured to use the cluster IP address (9.24.105.89) as the proxy address. They will send all HTTP requests to the Dispatcher, which will load balance to one of the proxy servers.

You can also use Internet Caching Protocol (ICP) or Remote Cache Access (RCA) to improve your environment by allowing each WTE server to check the contents or exchange information about other WTE servers’ cache. Refer to the product’s readme file for more information on ICP, and refer to Web Traffic Express User’s Guide Version 1.0, GC09-4567-00 for more information on RCA.

9.24.105.63rhlinux1

9.24.105.61rs600034 Web Traffic Express

proxy servers

NetworkDispatcher

Cluster:9.24.105.89

wses39.24.105.62rs600037

Chapter 9. Multiple Web Traffic Express proxy servers 417

9.3 ConfigurationThe Web Traffic Express proxy servers were configured as described in Chapter 7, “Web Traffic Express basic configuration” on page 287.

Network Dispatcher was configured as in the scenario we described in 4.1, “Load balancing” on page 70. We followed the steps listed below:

� Start the Executor

� Add the 9.24.105.89 cluster, using the local Dispatcher as primary host and configuring this address on the local network adapter (see Figure 9-2).

Figure 9-2 Adding the WTE cluster

� Add port 80. In our scenario, both proxy servers are listening to port 80. Make sure that all your proxy servers are listening to the same port. If it is not port 80, create the corresponding port in this step.

� Add the servers 9.24.105.61 and 9.24.105.63.

� Start the Manager.

� Start the WTE advisor, as shown in Figure 9-3. In our previous scenarios, we used the HTTP advisor. However, this time we are not load balancing Web servers, so we need to start an advisor specific to the service that the Dispatcher will load balance.


Figure 9-3 Starting the WTE advisor

We also set the stickytime so the Dispatcher would send all requests from a client to the same proxy server where their previous requests were sent. We recommend setting the stickytime in this scenario so the cached pages from a certain user will be kept on the same proxy server.

To set the stickytime, click Port:80 in the left pane of the window, then click Configuration Settings in the right pane. Locate the field Sticky time (seconds), fill in the value you want (we used 300 seconds), and click Update Configuration at the bottom of the window to activate the changes (see Figure 9-4).


Figure 9-4 Setting the stickytime

The configuration on the Dispatcher is ready. The next step is to set up the proxy servers to respond to the cluster IP address (see “Configuration of the back-end servers” on page 95).

On the proxy server 9.24.105.61, which is an AIX machine, we ran the following command:


On the proxy server 9.24.105.63, which is a Linux machine, we ran the following command:

# ifconfig lo:1 9.24.105.89 netmask 255.255.255.0


After running the commands above, the advisors began receiving responses from both proxy servers, as shown in Figure 9-5.

Figure 9-5 Advisors’ output for the WTE cluster

Any request sent to the cluster address will now be redirected to the proxy servers. The last thing to do is to configure the Web browsers to send the requests to the Dispatcher IP address (see 7.2.1, “Configuring the browser” on page 309).


We performed our tests using Netscape Communicator V4.7; the proxy configuration is shown below:

Figure 9-6 Browser configuration

9.3.1 Configuration fileIf you prefer to use the command line interface to configure the scenario described above, you can use the commands shown in Example 9-1.

Example 9-1 Configuration filendcontrol executor startndcontrol executor set nfa 9.24.105.62






ndcontrol advisor start wte 80 wte_80.log


9.4 TestsFirst, we did a test from only one browser, to make sure that the sticky option is working as expected. We configured this browser to use the Dispatcher as a proxy server, then we requested a few URLs. Figure 9-7 shows the monitor output while we did this test. Note that all requests are forwarded to only one of the proxy servers, 9.24.105.61. This confirms that while we used only one client IP address, the Dispatcher kept all requests forwarded to the same proxy server.

Figure 9-7 Monitor output: requests from only one client


After that, we tested using several clients to increase the traffic to the Dispatcher. This time, since we were using several IP addresses, there was load balancing among all proxy servers (see Figure 9-8).

Figure 9-8 Monitor output: using several browsers


Chapter 10. Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy

In this chapter, we discuss the combination of a wildcard cluster and Web Traffic Express to create a transparent proxy scenario.

10


10.1 IntroductionIf Web Traffic Express is configured to operate as a transparent proxy, the client software is totally unaware of the existence of the intermediate proxy server. Normally, if a client browser uses a proxy server, then the Web browser must be configured to specify the address and port of the proxy server. This is no longer necessary with transparent proxy, because the client is unaware that an intermediate proxy is in the network.

To use transparent proxy, the router is programmed to redirect requests to the WTE transparent proxy. WTE then intercepts all HTTP requests on port 80 that are targeted at some server on the Internet. The request is parsed and processed, and may be satisfied from the transparent proxy's cache. The router must be capable of dealing with the requests for other services that will also come from the clients.

This router may be a dedicated router, or it can be a machine running Network Dispatcher. We recommend you study carefully your environment to determine which one is the best solution for you.

10.2 Scenario descriptionIn the WTE scenarios which we showed in the previous chapters, all clients were configured to send the HTTP requests to the proxy server. The other packets were sent to the default router, which will forward them to the next router or to the proper destination. An example of this scenario is shown in Figure 10-1.

Figure 10-1 Basic WTE scenario

Web browsers

9.24.105.61rs600034

Default route

R

9.24.105.1Router

HTTP requests

Web Traffic Express


The impact of this scenario is that you need to configure all clients to use WTE. If you want to make it transparent for the user, the router needs to redirect all packets sent to port 80 to WTE. In this scenario, we include an extra machine running Network Dispatcher (ND) to perform the task of redirecting the packets based on the service being requested.

In our scenario, we used two machines: the first one is running Network Dispatcher (using only the Dispatcher component) and the second one is running Web Traffic Express. The Dispatcher works as a router, receiving all packets from the clients, no matter what the final destination of those packets may be. You can also install both WTE and ND on the same machine.

All packets sent to port 80 that are received by the Dispatcher will be forwarded to WTE, which is configured as a transparent proxy. The packets sent to ports other than 80 will be forwarded to a router; this is the default router of the local network (see Figure 10-2).

Figure 10-2 Transparent proxy scenario

In this scenario, the only configuration needed for the clients is the default route, which must be set to the Dispatcher IP address.

The following table shows more information on the machines we used for this scenario.

9.24.105.62rs600037Network

Dispatcher Web browsers

9.24.105.61rs600034

Web TrafficExpress

Default route

Port 80

R

All ports except 80

9.24.105.1Router

Chapter 10. Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy 427

Table 10-1 Machines used in this scenario


10.3 ConfigurationAs we mentioned previously, the only Network Dispatcher component we need for this scenario is the Dispatcher. Refer to Chapter 3, “Network Dispatcher installation” on page 27 for information about installing Dispatcher.

WTE needs to be installed as a transparent proxy. Refer to Chapter 6, “Web Traffic Express installation” on page 247 for information.

If you did not install WTE as a transparent proxy, you can still set it after the installation. Open the Configuration and Administration forms, click Proxy Configuration->Proxy Performance, and make sure that the Run as a transparent proxy checkbox is selected as shown in Figure 10-3. If it is not, you must select it, click Submit at the bottom of the window, and restart WTE.


Service


Dispatcher


Caching proxy

m238p4yl 9.24.105.82 Windows NT 4.0Service Pack 5Netscape Communicator V4.7

Web client


Figure 10-3 Selecting transparent proxy

Refer to Chapter 7, “Web Traffic Express basic configuration” on page 287 for more information about configuring WTE. As we mentioned before, the only requirement for WTE in this scenario is that it must run as a transparent proxy. You can configure it further according to your needs.

You can test WTE by configuring a browser to use it as an HTTP proxy. Make sure that WTE is working properly before configuring Network Dispatcher.


After you have tested WTE and confirmed that it is working properly, follow the steps below to configure a wildcard cluster in Network Dispatcher:

� Using the GUI, connect to the Dispatcher machine.

� Start the Executor.

� Add the cluster 0.0.0.0 (using the local Dispatcher as primary host). See Figure 10-4.

Figure 10-4 Adding the wildcard cluster

� Add port 80 to the wildcard cluster (see Figure 10-5). Port 80 is the port that WTE is listening to. In this scenario, we do not recommend that you use a different port.

Figure 10-5 Adding port 80 to the wildcard cluster

� Add a server to port 80 using the IP address of the WTE proxy server. If you installed both WTE and ND on the same machine, you must use the NFA of


the Dispatcher machine (see Figure 10-6). This is the only server we need to add to this cluster. In this case, the Dispatcher will not perform any load balancing.

Figure 10-6 Adding the server to the wildcard cluster

� The next step is to add a wildcard port to the wildcard cluster. Add port 0 to the wildcard cluster as shown in Figure 10-7. The wildcard port will handle all packets received by the Dispatcher, except for the packets sent to port 80.

Figure 10-7 Adding port 0 to the wildcard cluster


� Add a server to port 0, using the router IP address. The Dispatcher will forward to this server all packets sent to ports other than 80. We used the IP address of the default router of the local network (see Figure 10-8).

Figure 10-8 Adding the router IP address to port 0

� Start the Manager.

� After you start the Manager, you can also start the WTE advisor, as shown in Figure 10-9.

Figure 10-9 Starting the WTE advisor


The clients do not need any configuration, but their default route must be the Dispatcher IP address (see Figure 10-10).

Figure 10-10 Default route on the client machine


Make sure that the browser on the client machine is set to use a direct connection to the network. Figure 10-11 shows the configuration on a Netscape Communicator V4.7 browser.

Figure 10-11 Browser configuration


10.4 Network Dispatcher configuration fileIf you want to use the command line interface to configure the Dispatcher, use the commands shown in Example 10-1.

Example 10-1 Configuration file for the wildcard clusterndcontrol executor startndcontrol executor set nfa 9.24.105.60







ndcontrol advisor start wte 80 wte_80.log


10.5 TestsTo test this scenario, we configured a client machine with the default router as the Dispatcher IP address, and the browser set to use a direct connection to the network, as we described in the previous sections.

From that browser, we requested the URL http://www.yahoo.com. As shown in Figure 10-12, we received the home page.

Figure 10-12 Browser showing the requested home page


But how can we make sure that the request was sent to the Dispatcher, and then forwarded to WTE? To verify this, we opened the Dispatcher monitor on port 80, and requested several home pages again; we confirmed that the Dispatcher was receiving the requests and forwarding them to WTE (see Figure 10-13).

Figure 10-13 Server monitor showing the packets sent to WTE

We also looked at the proxy access log on WTE to make sure that our request had been properly received. We used the Configuration and Administration forms and clicked Server Activity Monitor->Proxy Access Statistics, as shown in Figure 10-14. You can see the request to http://www.yahoo.com in the statistics. This confirms that the Dispatcher forwarded the packet properly to WTE, and that WTE was able to fulfill the request.


Figure 10-14 Proxy Access Statistics window

These tests confirmed that port 80 is working properly. Now, we need to test the exceptional situation of connections to other ports that should be redirected to the router, and not to WTE. In that case, the router should forward the packet according to its routing table.

To perform this test, we initiated a Telnet session to a remote machine on a separate network. Telnet uses port 23, so when the Dispatcher receives the packets to this connection, it should forward them to the router (the server that was added to the wildcard port).


From the same client machine, we ran the following command:

# telnet 9.179.160.114

The connection was successfully established, as shown in Figure 10-15:

Figure 10-15 Telnet to a remote machine

To make sure that this connection was sent to the Dispatcher and then to the router, we ran iptrace at the Dispatcher level. Iptrace is a tool shipped with AIX to collect packets from the network. In order to use iptrace, you need to install the fileset bos.net.tcp.server.

Two SYN packets are shown in Example 10-2. Actually, they represent the same packet. The first one is being received by the Ethernet adapter on the Dispatcher machine; the second one is the same packet, but this time it is being sent to another machine.

Example 10-2 Iptrace of the Telnet connectionETH: ====( 60 bytes received on interface en0 )==== 20:24:59.575788775ETH: [ 00:06:29:b0:94:63 -> 00:04:ac:97:53:c8 ] type 800 (IP)IP: < SRC = 9.24.105.82 > IP: < DST = 9.179.160.114 > IP: ip_v=4, ip_hl=20, ip_tos=0, ip_len=44, ip_id=48058, ip_off=0 DFIP: ip_ttl=128, ip_sum=2282, ip_p = 6 (TCP)TCP: <source port=2708, destination port=23(telnet) >TCP: th_seq=1207f47, th_ack=0

TCP: th_off=6, flags<SYN>TCP: th_win=8192, th_sum=d084, th_urp=0TCP: mss 1460

ETH: ====( 60 bytes transmitted on interface en0 )==== 20:24:59.575812788ETH: [ 00:04:ac:97:53:c8 -> 10:00:5a:99:d4:f1 ] type 800 (IP)IP: < SRC = 9.24.105.82 > IP: < DST = 9.179.160.114 > IP: ip_v=4, ip_hl=20, ip_tos=0, ip_len=44, ip_id=48058, ip_off=0 DFIP: ip_ttl=128, ip_sum=2282, ip_p = 6 (TCP)TCP: <source port=2708, destination port=23(telnet) >TCP: th_seq=1207f47, th_ack=0TCP: th_off=6, flags<SYN>TCP: th_win=8192, th_sum=d084, th_urp=0TCP: mss 1460

This packet was sent from 9.24.105.82 (the client machine that started the Telnet connection) with destination 9.179.160.114. The port is 23, which is the Telnet port.

Since the source and destination IP addresses do not change while the packet goes through routers (and they must not!), the only way to know which machines redirected this packet is to look at the MAC addresses that appear in each packet.

In the second line of each packet on the iptrace, you can see the MAC address of the machines that sent and received the packet. The first one was sent from 00:06:29:b0:94:63 to 00:04:ac:97:53:c8. The second one was sent from 00:04:ac:97:53:c8 to 10:00:5a:99:d4:f1.

You can determine the local MAC addresses by running the command netstat -in (see Example 10-3).

Example 10-3 Output of the netstat -in commandName Mtu Network Address Ipkts Ierrs Opkts Oerrs Colllo0 16896 link#1 1111 0 1115 0 0lo0 16896 127 127.0.0.1 1111 0 1115 0 0lo0 16896 ::1 1111 0 1115 0 0en0 1500 link#2 0.4.ac.97.53.c8 50229 0 85850 0 0en0 1500 9.24.105 9.24.105.62 50229 0 85850 0 0

This shows that the MAC address of the local Ethernet adapter is 00:04:ac:97:53:c8. An easy way to determine the MAC addresses of the other machines is to look at the ARP table of the Dispatcher. The entries in the ARP table are deleted frequently, so the MAC addresses are not kept there for a long time. We looked at the ARP table right after our test, since we knew that all IP addresses would be there.

To view the ARP table, you can use the command arp -an (see Example 10-4).

Example 10-4 Output of the arp -an command? (9.24.105.1) at 10:0:5a:99:d4:f1 [ethernet]? (9.24.105.82) at 0:6:29:b0:94:63 [ethernet]? (9.24.105.60) at 0:4:ac:17:34:68 [ethernet]? (9.24.105.61) at 0:4:ac:97:52:97 [ethernet]

We came up with the following table after comparing the iptrace and the output of netstat -in and arp -an.

Table 10-2 Results of the test

The results of this analysis show that the packet was sent to the Dispatcher, which then redirected the packet to router 9.24.105.1.

Finally, you do not need to change anything in the routing table of the Dispatcher to make this scenario work. We used the Dispatcher machine with only the basic routes (the ones that are automatically added when you configure the network interface). You can see the routing table of the Dispatcher in Example 10-5.

Example 10-5 Routing table of the Dispatcher# netstat -rnRouting tablesDestination Gateway Flags Refs Use If PMTU Exp Groups

Route Tree for Protocol Family 2 (Internet):9.24.105/24 9.24.105.62 U 2 4858 en0 - -127/8 127.0.0.1 U 2 102 lo0 - -

Route Tree for Protocol Family 24 (Internet v6):::1 ::1 UH 0 0 lo0 16896 -

Packet Source MAC Destination MAC Sent from Received by

1 00:06:29:b0:94:63 00:04:ac:97:53:c8 9.24.105.82 9.24.105.62

2 00:04:ac:97:53:c8 10:00:5a:99:d4:f1 9.24.105.62 9.24.105.1


If we try to ping the 9.179.160.114 machine, it cannot reach the destination, as shown in Example 10-6.

Example 10-6 Ping failed# ping 9.179.160.114PING 9.179.160.114: (9.179.160.114): 56 data bytes0821-069 ping: sendto: Cannot reach the destination network.ping: wrote 9.179.160.114 64 chars, ret=-10821-069 ping: sendto: Cannot reach the destination network.ping: wrote 9.179.160.114 64 chars, ret=-1

We set the configuration this way just to make sure that Network Dispatcher was handling all packets, and not the TCP stack of the machine.

Configure the machine for your environment with the routes you need, and the traffic from the other machines will be handled by the Dispatcher’s clusters.


Chapter 11. Content Based Routing (CBR)

In this chapter, we discuss the Content Based Routing (CBR) component of Network Dispatcher.

11


11.1 IntroductionIn this chapter, we will document scenarios using Content Based Routing (CBR). See IBM Network Dispatcher User’s Guide Version 3.0 for Multiplatforms, GC31-8496-05 for more information on how CBR works.

Content Based Routing can be configured with WTE for HTTP servers. In addition, it can be configured as a CBR proxy (without WTE) for IMAP and POP3 servers. However, CBR cannot be configured for both on the same Network Dispatcher machine.

11.1.1 Installation of the CBR functionRefer to Chapter 3, “Network Dispatcher installation” on page 27 for information on how to install any Network Dispatcher component.

When you reach the point where you must choose which component to install, select the following three items to install CBR on your machine:

� Content Based Routing Runtime

� Content Based Routing Administration

� Content Based Routing License


The following figure shows these three items being selected:

Figure 11-1 Installing the CBR components

If you want to use CBR for HTTP, you must also install Web Traffic Express. Refer to Chapter 6, “Web Traffic Express installation” on page 247 for more information.

11.2 CBR for HTTPCBR works with WTE to proxy client requests to specified servers. CBR, along with WTE, filters Web page content using specified rules.

11.2.1 CBR scenarioIn this scenario, we demonstrate how to use CBR for HTTP by configuring a basic environment.

Chapter 11. Content Based Routing (CBR) 445

WTE configuration overviewThe WTE component of WebSphere Edge Server for Multiplatforms has the functionality to work with the CBR component of Network Dispatcher. The steps required to start the WTE server are different on UNIX systems than on Windows systems. However, on all these platforms, WTE can be started with all of the default values without making any modifications to the configuration file.

Refer to Chapter 7, “Web Traffic Express basic configuration” on page 287 for more information about configuring WTE.

WTE configuration file CBR modificationsWhen WTE is installed, the WTE configuration file, ibmproxy.conf, contains many lines of preconfigured WTE directives. It includes four lines to configure CBR, but these lines are commented out. Search for the directives shown in Example 11-1 and uncomment them. Note that the ServerInit directive is written using only one line. The next line is PreExit. Make sure that the path is correct on the predefined directives.

Example 11-1 Modifications to the ibmproxy.conf file# ===== CBR Plug-in =====ServerInit C:\Progra~1\IBM\nd\cbr\lib\libndcbr.dll:ndServerInit CBR_CLIENT_KEYS_DIRECTORY=C:\Progra~1\IBM\nd\admin\keys\cbr,CBR_SERVER_KEYS_DIRECTORY=C:\Progra~1\IBM\nd\cbr\key,END_INSTALL_PATH=C:\Progra~1\IBM\nd,C:\Progra~1\IBM\nd\cbr\lib;C:\Progra~1\IBM\nd\cbr\lib\ibmcbr.jar;C:\Progra~1\IBM\nd\admin\lib\ChartRuntime.jar,11099,C:\Progra~1\IBM\nd\cbr\logs\,C:\Progra~1\IBM\nd\cbr\configurations\

PreExit C:\Progra~1\IBM\nd\cbr\lib\libndcbr.dll:ndPreExit

PostExit C:\Progra~1\IBM\nd\cbr\lib\libndcbr.dll:ndPostExit

ServerTerm C:\Progra~1\IBM\nd\cbr\lib\libndcbr.dll:ndServerTerm

Add these lines to the WTE configuration file and save the file.

Note: If you are using Windows NT, you must complete the configuration by adding the following to the Path system environment variable:

C:\Program Files\nd\cbr\lib;C:\Program Files\Ibm\Java13\jre\bin\classic

In this case, we had to reboot the machine after changing the Path variable, so that the changes could take effect for Control Panel.

Perform the proper changes depending on the operating system you are using. Now you can restart WTE.


CBR configurationOnce the ibmproxy.conf file has been modified with the four lines to enable CBR, and the WTE proxy server has been restarted, then CBR configuration can begin.

In this scenario, we defined two clusters, A and B, each with two servers. Cluster A had two servers, an AIX server and a Linux server, and the two servers in cluster B were Windows NT systems.

Network environmentThe following figure is a graphical representation of our scenario environment:

Figure 11-2 CBR scenario

Cluster, port, server and rule configurationOnce the CBR-enabled proxy server was running, we used the Network Dispatcher GUI to configure our CBR cluster.

To start the GUI, we clicked Start->Programs->IBM WebSphere->Edge Server->IBM Network Dispatcher. We right-clicked the Content Based Routing component and we selected Connect to Host from the pop-up menu.

Web servers

Cluster A9.24.105.87

9.24.105.82m238p4yl

Cluster B9.24.105.88

9.24.105.20rs600036

CBR/WTE

9.24.105.39m23kk904

9.24.105.95m23ff426

9.24.105.63rhlinux1


If, at the point where you try to connect to the host to set the configuration either through the Network Dispatcher GUI or with the cbrcontrol commands, you receive a key related error message, then it is possible that there is an error in the four lines that were added to your ibmproxy.conf file.

In our scenario, we then started the Executor, added our clusters, and to each cluster added a port and two servers (we followed the same procedure to add clusters and servers as we did with the Dispatcher).

Figure 11-3 Basic cluster configuration

We will now show you details of how to add two rules to the 9.24.105.87 cluster. To configure the content rules that CBR uses to determine among which servers to load balance the request, we right-clicked Port:80 in the navigation portion of the GUI and selected Add Rule... from the pop-up menu.


Of course, you should plan the logic that you want CBR to follow before you start adding rules to your configuration.

The network interface card of the CBR machine must be aliased to all the cluster addresses used in the configuration (see Figure 11-4). However, the loopback interface on the TCP server machines does not need to be aliased; for those who are familiar with this kind of aliasing operations in a Dispatcher scenario, it may seem unusual that similar aliasing is not done with CBR.

Figure 11-4 Aliases on the network interface

The reason for this is that when the packet is sent from the client machine, its destination IP address is the IP address of one of the clusters. When the packet arrives at the CBR machine, the packet is accepted because the network interface card on the CBR machine has been aliased to that cluster IP address. WTE receives the request and offers CBR the opportunity to examine its clusters and rules for a match. If a match is found, URL name translation is performed.

If and when the proxy server sends the request to a clustered TCP server (the selection of which was done by CBR), WTE proxies a new request to the TCP server with the modified URL and a destination IP address that is the IP address of the TCP server machine. When the TCP server replies, its response goes


back to the CBR server, and is cached by WTE, if caching is enabled and if the WTE caching algorithms require caching of that particular page. For this reason, there is no need to alias the cluster IP address to the loopback interface on the TCP server machine.

Contrarily, Dispatcher keeps the cluster IP address as the destination address of the packet and identifies the TCP servers through their Media Access Control (MAC) addresses. For this to work, the TCP servers need to have the cluster IP address aliased on their loopback interface. An advantage of this is that the server’s response can flow directly to the client, without any need for it to be proxied by the Dispatcher. This would not be a good solution with CBR, however, because it would not allow caching.

Another positive factor is that a CBR cluster does not have the same restriction as a Dispatcher cluster concerning the location of the TCP servers relative to the Dispatcher or CBR server. With CBR, because WTE proxies the request to the servers, there is no requirement for the servers to be located on the same LAN as the CBR machine.

Our next step was to create a content rule to direct all requests to cluster 9.24.105.87, which contained the string products.html in the URL to server 9.24.105.63. To accomplish this, we right-clicked Port:80->Add Rule->Content, and in the Add Rule dialog window, we filled in the fields as follows:

Figure 11-5 Adding the first content rule


� We assigned the name productpage to this rule.

� We did not put a value in the Priority field for the rule. Priorities establish the order in which rules will be reviewed. This parameter accepts integer values. If you do not specify the priority of the first rule you add, CBR will set it by default to 1. When a subsequent rule is added, by default its priority is calculated to be 10 plus the current lowest priority of any existing rule. For example, assume you have an existing rule whose priority is 30. You add a new rule and set its priority at 25 (which is a higher priority than 30). Then you add a third rule without setting a priority. The priority of the third rule is calculated to be 30 + 10 = 40, and so on.

� The Affinity type field contains one of two possible values: Client IP or Cookie. Client IP affinity as used in the Dispatcher component can also be used with CBR. The cookie affinity feature applies only to the CBR component and provides a way to make clients sticky to a particular server. This function is enabled by setting the stickytime of a rule to a positive number, and setting the affinity to Cookie.

We left the default value of Client IP in the Affinity type field. In the next rule, we demonstrate setting cookie affinity and stickytime.

� The Pattern field is used to define the pattern of characters that CBR will match against each client request.

The pattern must not contain any spaces and can make use of the special characters listed in the following table:

Table 11-1 Special characters allowed in the Pattern field

Character Function

* Matches 0 to x of any character

( Used for logic grouping

) Used for logic grouping

& Logical AND

| Logical OR

! Logical NOT


The reserved keywords shown in the following table must always be followed by the equal (=) sign:

Table 11-2 Keywords followed by the equal (=) sign

In our case, we made use of the URL reserved word and the wildcard (*) character. We specified the pattern as:

url=http://*/products.html

This rule specifies that any URL that is a request for the page products.html will be a match.

Other examples of valid rule patterns are:

– url=http://*/*.gif

– client=9.32.*

– (path=index/*.gif&protocol=http)|(client=9.1.2.3)

– !(path=*.jpeg)

The last item in the dialog window shown in Figure 11-5 on page 450 is a scrollable list of server addresses you may choose from. In this case, we chose the server with the IP address 9.24.105.63, meaning that we wanted this server to serve the client requests containing the string products.html in the URL.

We then clicked OK in the dialog window shown in Figure 11-5 on page 450.

The next rule was added to send requests to both servers in the 9.24.105.88 cluster for client requests for the page purchase.html. In this case, however, we specified an affinity type of Cookie and a stickytime of 30 seconds. This means that when a client request to the cluster is made and the URL matches the rule (that is, contains the string purchase.html), CBR would load balance the request to the best server of the two; then, all subsequent HTTP requests from that client to this cluster address would also go to that server for a period of 30 seconds. We called this rule purchase. The dialog window in this case appeared as follows:

Keyword Value

client Client IP address

url URL in request

protocol Protocol section of URL

path Path section of URL

refer Referred URL (quality of service)

user User ID section of URL


Figure 11-6 Adding the second rule

We then clicked OK in the window above. The GUI reported that the configuration was updated with the added rules, as shown in the following figure:


Figure 11-7 Configuration showing both rules added

CBR Manager and AdvisorsAs with the Dispatcher, starting the Manager and using the Advisors is optional. The Manager can be activated with the command:

cbrcontrol manager start

The typical advisor that is used with CBR is the HTTP advisor, which can be launched by entering the following command:

cbrcontrol advisor start http port

where port is the port configured for the cluster.


Saving the configurationIn our scenario, once we finished configuring CBR, we saved the configuration to a file. To do this from the Network Dispatcher GUI, we right-clicked Host and selected Save Configuration File As....

In the resulting pop-up window, we were prompted to enter the name of the configuration file in which we wanted to save this information. We entered the file name default.cfg in the Save Configuration pop-up window. By default, this file is placed in the directory <install path>/<component>/configurations, where <install path> varies depending on the operating system, and the <component> in this case is cbr.

The configuration file is saved in ASCII format and contains the list of commands that would be necessary to reconfigure your environment. Following is the content of the default.cfg file:

Example 11-2 CBR configuration filecbrcontrol cluster add 9.24.105.88

cbrcontrol port add 9.24.105.88:80

cbrcontrol server add 9.24.105.88:80:9.24.105.39


cbrcontrol rule add 9.24.105.88:80:purchase type content pattern url=http://*/purchase.html priority 1 cbrcontrol rule useserver 9.24.105.88:80:purchase 9.24.105.39cbrcontrol rule useserver 9.24.105.88:80:purchase 9.24.105.95cbrcontrol rule set 9.24.105.88:80:purchase stickytime 30cbrcontrol rule set 9.24.105.88:80:purchase affinity cookie

cbrcontrol cluster add 9.24.105.87

cbrcontrol port add 9.24.105.87:80



cbrcontrol rule add 9.24.105.87:80:productpage type content pattern url=http://*/products.html priority 1 cbrcontrol rule useserver 9.24.105.87:80:productpage 9.24.105.63

The saved configuration file is interesting because it contains each of the cbrcontrol commands that could also be entered from the command line to configure the same environment. If you choose to configure CBR with commands, you can save your configured environment with the command:

cbrcontrol file save configurationfilename

You can then subsequently reload the configuration file with the command:

cbrcontrol file load configurationfilename

Scenario resultsWe placed simple HTML files in the document root directories on each of our Web servers. On the Web server with IP address 9.24.105.63, we placed a file named products.html and another called purchase.html. On the Web server with IP address 9.24.105.20 we placed a different file, also called products.html. Recall that we had defined a rule specifying that client requests containing the string products.html in the URL be load balanced between both of these servers. Each of the files uniquely identified the server on which it was located, as well as the name of the page.

In a CBR scenario, the client machine does not need any special configuration. In particular, client browsers must not be set to redirect all the requests to the CBR machine. For example, in a real-life situation, it would not be appropriate to require end-users to reconfigure their Web browsers before accessing a Web site that uses CBR. The use of CBR in a Web site is completely transparent to end-users.


Client IP affinity demonstrationFor our first request, with our browser we requested the products.html page from our cluster IP address and received the following page in response:

Figure 11-8 Accessing the cluster

To verify that our productpage rule was matched by this request, we used the cbrcontrol persistent session command rule rep.

The command line version of this is:

cbrcontrol rule report cluster:port:rule


As with other cbrcontrol commands, short forms of the keywords can be used. In order to specify that you would like the report to include all clusters, ports and rules, use only the two colon (::) delimiters in the command line. The command and its output are shown in Example 11-3.

Example 11-3 statistics for the rulesC:\> cbrcontrolcbrcontrol>>rule rep ::

---------------------------------------------Cluster Address: 9.24.105.88Port:80---------------------------------------------

Rule Report: ------------ Rule name ...................... purchase Priority ....................... 1 Times Fired .................... 0 Number of Servers .............. 2

---------------------------------------------Cluster Address: 9.24.105.87Port:80---------------------------------------------

Rule Report: ------------ Rule name ...................... productpage Priority ....................... 1 Times Fired .................... 11 Number of Servers .............. 1

Since the stickytime is set to 0 seconds, Web requests from the same client should not stick to the same Web server, and subsequent requests should normally load balance between the two Web servers defined in cluster 9.24.105.87. If the stickytime were set to a positive number of seconds, CBR would choose one of the two servers the first time a client request arrives that matches the productpage rule, and then would redirect all the client requests coming from the same IP address to the same server until the stickytime expires.

In this case, however, we forced the CBR server to redirect all the requests to the Web server with IP address 9.24.105.63, because this was the only server available in the productpage rule.


Cookie Affinity demonstrationThe next test we performed demonstrated the use of Cookie affinity.

Web cookies are simple pieces of information passed between the client Web browser and the Web server during an HTTP transaction. Cookies do not contain any information about the client that the server does not already have and they cannot do anything on the client machine that the client itself cannot already do, provided the browser is within the specifications. Cookies were introduced as an answer to a fundamental problem of the Web's underlying HTTP 1.0 protocol: the lack of a state, or a persistent connection.

The first time a client accesses a Web site that serves cookies, the chosen server sends a cookie to the client browser, identifying the server and giving some information about which URLs the cookie is good for. The next time the client visits one of those URLs, the browser includes the cookie in its request. As long as the client’s future requests contain the cookie, and each request arrives within the stickytime interval, the client will maintain affinity with the initial server.

Prior to making a request for the purchase page that will trigger the 30 second affinity rule, we examined the contents of the local cookie file (cookies.txt) in the browser directory on our client machine; it was empty:

Example 11-4 Cookies before the testC:\Program Files\Netscape\Users\default>type cookies.txt# Netscape HTTP Cookie File# http://www.netscape.com/newsref/std/cookie_spec.html# This is a generated file! Do not edit.

yp.yahoo.com TRUE / FALSE 1262383331 YP.us v=2&m=city&d=%01Durham%0 0%01-78.838402%015%01cwww.aol.com FALSE FALSE 986502134 myCookie nm_b

Recall that we had enabled a 30 second cookie affinity on our purchase rule by setting the stickytime to 30, and setting the affinity to Cookie when we created the rule. This could also have been set with the command:

cbrcontrol rule set

Once a server is selected by CBR to respond to our request, subsequent requests were also responded to by the same server.

In the 30 second period, we made three requests for http://9.24.105.88/purchase.html. Each request was responded to by the same server:


Figure 11-9 Accessing using Cookie affinity

We set the browser to warn us before setting a cookie, so it showed the following window before accessing the home page:

Figure 11-10 Information about the cookie


We verified that the cookie was set on our browser machine by examining the cookie.txt file:

Example 11-5 Cookies after the testC:\Program Files\Netscape\Users\default>type cookies.txt# Netscape HTTP Cookie File# http://www.netscape.com/newsref/std/cookie_spec.html# This is a generated file! Do not edit.

yp.yahoo.com TRUE / FALSE 1262383331 YP.us v=2&m=city&d=%01Durham%0 0%01-78.838402%015%01cwww.aol.com FALSE FALSE 986502134 myCookie nm_b9.24.105.88 FALSE / FALSE 986511773 IBMCBR -54-29-1007656-5932-42-6048-10045+986504573541!

Even though the contents of the cookie are not understandable by us, they are meaningful to the Web server and browser.

After the 30 second stickytime expired, we saw that the CBR server was allowed to choose the other Web server to satisfy the request from the client.

11.2.2 WTE CacheByIncomingUrl directiveA WTE directive has been added for use with CBR in the ibmproxy.conf configuration file. This directive, CacheByIncomingUrl, specifies whether to use the incoming URL or the outgoing URL as the basis for generating cache file names. The values of this directive can be on or off.

If CacheByIncomingUrl is set to on, the incoming URL will be used to generate the cache file name. In other words, when this directive is set to on, WTE keeps the original URL and uses it to cache the page that it gets back from the Network Dispatcher-changed URL.

On the other hand, if off is specified, CBR rule matching and load balancing will be done on the incoming URL and the resulting URL will be used to generate the cache name. In this case, WTE uses the Network Dispatcher-changed URL to cache the page.

The CacheByIncomingUrl directive’s default value is off in the ibmproxy.conf file.

Notice that CacheByIncomingUrl is a hidden directive of WTE. In other words, it is possible to alter its value only by manually editing the configuration file ibmproxy.conf; there is no way to change the value of this directive through the Configuration and Administration forms.


Notice also that a Web page is cached only if WTE decides it should be. WTE decides this by looking at the Expires tag. If there is none, it estimates an expiration time via the Last-Modified parameter. You should keep this in mind if you are testing CBR caching with sample pages you have just created and which do not contain Expires header information. In this case, recently created pages would not be cached, since they would be interpreted by WTE as frequently changed pages, thereby resulting in an apparent failure of the CBR caching function.

11.3 CBR proxy for IMAP and POP3CBR for IMAP or POP3 proxies the client requests to the appropriate server based on user ID and password. You can configure a cluster of IMAP or POP3 servers as though they were one server, thus allowing support for extremely large numbers of mailboxes from a single site.

The content in question is only the IMAP or POP3 login sequence. The CBR proxy is placed in front of a cluster of either IMAP or POP3 servers (not both). The proxy directs the login request to a server that supports the user name and password supplied by the client.

11.3.1 ScenarioWe used a scenario with one machine running Network Dispatcher, where we created the CBR cluster, and two back end servers running POP3. On these servers, we created individual users (the user created in one POP3 server does not exist on the other POP3 server).

Note: If any individual mailbox is actually installed on more than one of these clustered mailbox servers, then the first one to reply will be used.


Figure 11-11 shows the environment we used.

Figure 11-11 CBR for POP3 scenario

11.3.2 ConfigurationThe configuration of CBR for IMAP and POP3 is very similar to the configuration of Dispatcher and CBR for HTTP. You need to add a cluster, ports and servers.

If you prefer to use the wizard, you can run it by right-clicking Content Based Routing in the left pane of the GUI window and selecting Start Configuration Wizard, as shown in Figure 11-12.

9.24.105.62rs600037

POP3servers

9.24.105.83m238p4yl

NetworkDispatcher

Cluster:9.24.105.87

wses1

9.24.105.60rs600010


Figure 11-12 Starting the CBR configuration wizard

The configuration wizard will guide you through the steps necessary to add the clusters, ports and server.

In this section, we describe the configuration steps using the GUI.

First, you need to start cbrserver. Run the following command (this is only needed for CBR for IMAP or POP3):

# cbrserver


Then, you can connect to the host where CBR is running using the GUI. Once you are connected, you can add the cluster by right-clicking Executor in the left pane of the GUI window, as shown in Figure 11-13.

Important: You need to automatically start cbrserver after a reboot if you are using this in your production environment. Refer to “Starting Network Dispatcher after a reboot” on page 95 for more information.

For Windows systems, since there is no service created for cbrserver, you can:

� create a service and start it automatically, or

� add it to the Startup program group

For more information, refer to Windows NT or Windows 2000 documentation.


Figure 11-13 Adding a cluster

A pop-up window will be displayed, as shown in Figure 11-14. We typed in the IP address of our POP3 cluster, 9.24.105.87.

Figure 11-14 Entering the cluster address


The next step is to add a port to this cluster. Right-click Cluster, then select Add Port.... The pop-up window shown in Figure 11-15 will be displayed. Type in the port number you are using and select the protocol (either POP3 or IMAP). Since we are creating a POP3 cluster, we typed in the port number 110 and selected the protocol POP3.

Figure 11-15 Adding the POP3 port

Now you need to add the servers that belong to this cluster. Right click Port:110, then select Add Server.... The pop-up window shown in Figure 11-16 will be displayed. Type in the IP address of the POP3 server you want to add to this cluster.

Figure 11-16 Adding a server

Add all servers that you want to be part of this cluster. We added two servers: 9.24.105.60 and 9.24.105.62. The complete configuration is shown in Figure 11-17.


Figure 11-17 CBR for POP3 configuration

If you want, you can also start the Manager and Advisors for this environment.


11.3.3 Configuration fileThe configuration file generated by the GUI after saving the configuration is shown in Figure 11-6. You can use these commands to create your configuration using the command line interface.

Example 11-6 CBR for POP3 configuration filecbrcontrol cluster add 9.24.105.87cbrcontrol port add 9.24.105.87:110 protocol POP3cbrcontrol server add 9.24.105.87:110:9.24.105.62cbrcontrol server add 9.24.105.87:110:9.24.105.60

11.3.4 TestsWe recommend that you test your POP3 servers to make sure they are working properly before you start testing the CBR cluster configuration.

We created a user cris in the machine 9.24.105.62. We checked to see if this user had messages in the mailbox (see Example 11-7). When you check for messages in the user’s mailbox, make sure you do not remove them from the mailbox.

Example 11-7 Checking the user cris’s mailbox# cd /var/spool/mail# ls -ltotal 5-rw-rw---- 1 cris mail 1217 Apr 06 09:52 cris-rw-rw---- 1 root mail 535 Mar 01 16:31 root# su - cris[YOU HAVE NEW MAIL]$ mailMail [5.2 UCB] [AIX 4.1] Type ? for help."/var/spool/mail/cris": 2messages 2 unread>U 2 root Fri Apr 6 09:25 16/371 "Test 1" U 3 root Fri Apr 6 09:52 16/365 "Test 2"? qHeld 2messages in /var/spool/mail/cris


Then we set up Netscape Mail to use the CBR cluster as a POP3 server, as shown in Figure 11-18.

Figure 11-18 Netscape Mail configuration for user cris

Using Netscape Mail, we tried to download the messages from the POP3 server. The client opened a connection to the cluster address; CBR communicated with the POP3 that had the user cris on it, and sent back the messages to the client (see Figure 11-19).


Figure 11-19 Netscape Mail showing the messages for user cris

As Figure 11-19 shows, the messages were successfully downloaded from the POP3 server using the CBR cluster.


Part 5 Appendix

Part 5


Appendix A. Java source code

This appendix lists source code for the two Java applications used to demonstrate the dynamic caching feature of Web Traffic Express.

A


TimeServlet Java source codeExample 11-8 TimeServlet.java

import java.io.*;import javax.servlet.*;import javax.servlet.http.*;

public class TimeServlet extends HttpServlet { public void service(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException { java.util.Date d = new java.util.Date(); PrintWriter out = res.getWriter(); out.println(" "); out.println(d.getHours()+":"+d.getMinutes()+":"+d.getSeconds()); out.println(" This timestamp was calculated by TimeServlet, which is not cacheable, and should change "); }}

CalcServlet Java source codeExample 11-9 CalcServlet.java

import java.io.*;import javax.servlet.*;import javax.servlet.http.*;

public class CalcServlet extends HttpServlet { public void service(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException { try {

ServletContext servletContext = getServletContext(); java.util.Date d = new java.util.Date(); PrintWriter out = res.getWriter(); out.println(d.getHours()+":"+d.getMinutes()+":"+d.getSeconds()+"\n--Performing Calculation... "); out.println("--This timestamp was calculated by CalcServlet, which is cacheable, and shouldn't change "); servletContext.getRequestDispatcher("/servlet/TimeServlet").include(req,res); int arg1 = Integer.parseInt(req.getParameter("arg1")); int arg2 = Integer.parseInt(req.getParameter("arg2")); //we'll wait for a little while to pretend this calculation takes some effort, //and make the caching more noticable. synchronized (this) { try {

wait(300); } catch (Exception e) {e.printStackTrace();} } out.println(d.getHours()+":"+d.getMinutes()+":"+d.getSeconds()+"\n--answer = "+arg1*arg2+" "); } catch (Exception e) { e.printStackTrace(); } }}

Appendix A. Java source code 477

Related publications

The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this redbook.

IBM RedbooksFor information on ordering these publications, see “How to get IBM Redbooks” on page 480.

� IBM WebSphere Performance Pack: Load Balancing with IBM SecureWay Network Dispatcher, SG24-5858-00

� IBM WebSphere Performance Pack: Caching and Filtering with IBM Web Traffic Express, SG24-5859-00

Other resourcesThese publications are also relevant as further information sources:

� WebSphere Edge Server for Multiplatforms Getting Started Version 1.0, SC09-4566

� WebSphere Edge Server for Multiplatforms Web Traffic Express User’s Guide, GC09-4567

� WebSphere Edge Server for Multiplatforms WTE Programming Guide, GC09-4568

� WebSphere Edge Server for Multiplatforms WTE Programming Guide, GC09-4568

� IBM Network Dispatcher User’s Guide Version 3.0 for Multiplatforms, GC31-8496

Referenced Web sitesThese Web sites are also relevant as further information sources:

� http://www-105.ibm.com/developerworks/tools.nsf/dw/java-devkits-byname?OpenDocument&Count=100 developerWorks: Java™ technology : Tools and

products - Developer kits




� http://www.ibm.com/developerworks/java/jdk/index.html developerWorks: Java technology: IBM Developer Kits

� http://www.ibm.com/software/webservers/edgeserver/library.html WebSphere Edge Server for Multiplatforms README Version 1.0.3

� http://www.ibm.com/software/webservers/edgeserver/doc/wte_v36/readme.html WebSphere Edge Server for Multiplatforms WTE V3.6 Readme Version

1.0.3

� http://www.ibm.com/software/webservers/edgeserver/doc/ndv36/readme.txtREADME for IBM Network Dispatcher V3.6

� http://www.ibm.com/software/webservers/edgeserver/library.html WebSphere Edge Server for Multiplatforms Version 1.0.3 Documentation

Supplement

� http://www.ibm.com/software/webservers/edgeserver/library.html Network Dispatcher, V3.6: Scalability, Availability and Load-balancing for

TCP/IP Applications

� http://www.ibm.com/software/webservers/edgeserver/efix-wte.html Caching Proxy component - WebSphere Edge Server 1.0 corrective

service

� http://www.ibm.com/software/webservers/edgeserver/efix-nd.html Load Balancing component - WebSphere Edge Server 1.0.3 corrective

service

� http://www.ibm.com/software/webservers/edgeserver/library.html Websphere Edge Services architecture

� http://www.iplanet.com/downloads/developer/ iPlanet Downloads Developer Downloads

� http://www.surfcontrol.com SurfControl - Internet Filtering, Monitoring, Blocking, and Access Control Software

� http://www.ibm.com/developerworks/linux/ developerWorks: Linux

How to get IBM RedbooksSearch for additional Redbooks or redpieces, view, download, or order hardcopy from the Redbooks Web site:

ibm.com/redbooks

Also download additional materials (code samples or diskette/CD-ROM images) from this Redbooks site.

480 WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher480 WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher


http://www.ibm.com/redbooks/

http://www.ibm.com/redbooks/

http://www.ibm.com/software/webservers/edgeserver/library.html




http://www.ibm.com/software/webservers/edgeserver/doc/wte_v36/readme.html

http://www.ibm.com/software/webservers/edgeserver/doc/wte_v36/readme.html

http://www.ibm.com/software/webservers/edgeserver/doc/ndv36/readme.txt

http://www.ibm.com/software/webservers/edgeserver/efix-wte.html

http://www.ibm.com/software/webservers/edgeserver/efix-nd.html

http://www.iplanet.com/downloads/developer/

http://www.surfcontrol.com



Redpieces are Redbooks in progress; not all Redbooks become redpieces and sometimes just a few chapters will be published this way. The intent is to get the information out much quicker than the formal publishing process allows.

IBM Redbooks collectionsRedbooks are also available on CD-ROMs. Click the CD-ROMs button on the Redbooks Web site for information about all the CD-ROMs offered, as well as updates and formats.

Related publications 481

482 WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher482 WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Special notices

References in this publication to IBM products, programs or services do not imply that IBM intends to make these available in all countries in which IBM operates. Any reference to an IBM product, program, or service is not intended to state or imply that only IBM's product, program, or service may be used. Any functionally equivalent program that does not infringe any of IBM's intellectual property rights may be used instead of the IBM product, program or service.

Information in this book was developed in conjunction with use of the equipment specified, and is limited in application to those specific hardware and software products and levels.

IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to the IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785.

Licensees of this program who wish to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information which has been exchanged, should contact IBM Corporation, Dept. 600A, Mail Drop 1329, Somers, NY 10589 USA.

Such information may be available, subject to appropriate terms and conditions, including in some cases, payment of a fee.

The information contained in this document has not been submitted to any formal IBM test and is distributed AS IS. The use of this information or the implementation of any of these techniques is a customer responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. While each item may have been reviewed by IBM for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Customers attempting to adapt these techniques to their own environments do so at their own risk.

Any pointers in this publication to external Web sites are provided for convenience only and do not in any manner serve as an endorsement of these Web sites.


The following terms are trademarks of other companies:

Tivoli, Manage. Anything. Anywhere.,The Power To Manage., Anything. Anywhere.,TME, NetView, Cross-Site, Tivoli Ready, Tivoli Certified, Planet Tivoli, and Tivoli Enterprise are trademarks or registered trademarks of Tivoli Systems Inc., an IBM company, in the United States, other countries, or both. In Denmark, Tivoli is a trademark licensed from Kjøbenhavns Sommer - Tivoli A/S.

C-bus is a trademark of Corollary, Inc. in the United States and/or other countries.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and/or other countries.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States and/or other countries.

PC Direct is a trademark of Ziff Communications Company in the United States and/or other countries and is used by IBM Corporation under license.

ActionMedia, LANDesk, MMX, Pentium and ProShare are trademarks of Intel Corporation in the United States and/or other countries.

UNIX is a registered trademark in the United States and other countries licensed exclusively through The Open Group.

SET, SET Secure Electronic Transaction, and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC.

Other company, product, and service names may be trademarks or service marks of others.


acronyms

ACL access control list

AFS Andrew File System

AIX advanced interactive executive

ATM Asynchronous Transfer Mode

API application programming interface

APPN Advanced Peer-to-Peer Network

ARP Address Resolution Protocol

CARP Cache Array Routing Protocol

CB Component Broker

CBR Content Based Routing

CGI Common Gateway Interface

CORBA Common Object Request Broker Architecture

DMZ demilitarized zone

DNS Domain Name System

ECA External Cache Adapter

EJB Enterprise JavaBeans

EJS Enterprise Java Services

FDDI Fiber Distributed Data Interface

FTP File Transfer Protocol

GC garbage collector

GEM Global Enterprise Manager

GRE Generic Routing Encapsulation

GUI graphical user interface

HACMP High Availability Cluster Multiprocessing

HP-UX Hewlett-Packard UNIX

Abbreviations and

© Copyright IBM Corp. 2001

HTML Hypertext Markup Language

HTTP Hypertext Transfer Protocol

IBM International Business Machines Corporation

ICP Internet Caching Protocol

IEEE Institute of Electrical and Electronics Engineers

IMAP Internet Mail Access Protocol

IP Internet Protocol

ISP Internet Service Provider

ISS Interactive Session Support

ITSO International Technical Support Organization

JDK Java Development Kit

JRE Java Runtime Environment

JSP JavaServer Pages

JVM Java Virtual Machine

LAN local area network

LDAP Lightweight Directory Access Protocol

LPAR Logical partition mode

MAC Media Access Control

Mbps megabits per second

MBps megabytes per second

MIB management information base

MVS multiple virtual storage

NAP network access points

NAT network address translation

NCSA National Center for Supercomputing Applications

ND Network Dispatcher

485

NFA Non-forwarding address

NNTP NetNews transfer protocol

OS/2 Operating System/2

OSA Open Systems Adapter

OSE open servlet engine

PAC proxy automatic configuration

PICS Platform for Internet Content Selection

POP points of presence

POP3 Post Office Protocol 3

RCA Remote Cache Access

RFC Request for Comment

RMI Remote Method Invocation

RPC remote procedure call

RSACI Recreational Software Advisory Council on the Internet

RTSP Real Time Streaming Protocol

SDA Server Directed Affinity

SSL Secure Sockets Layer

SMTP Simple Mail Transfer Protocol

SPARC Scalable processor architecture

TCP/IP Transmission Control Protocol/Internet Protocol

TEC Tivoli Enterprise Console

TOS Type of service

UDP User Datagram Protocol

URI Universal Resource Identifier

URL Universal Resource Locator, Uniform Resource Locator

WAN wide area network

WAND Wide Area Network Dispatcher

WTE Web Traffic Express

WLM workload manager

WPAD Web Proxy Auto Discovery

WWW World Wide Web

W3C World Wide Web Consortium

XML Extensible Markup Language


Index

Aaccess log exclusions 408, 411AccessLog 404–405active connections 93administration components 157advisor 80, 88–89, 93–94, 102, 122–123, 165–168, 172–173, 189, 199, 201, 232, 238, 241, 243, 416, 418, 421–422, 432, 435, 454, 468advisor fast-failure detection 10affinity 11, 231–232, 235, 451, 459affinity address mask 234affinity record 232affinity table 232–233ARP 92, 95, 441asynchronous I/O 248automatic cache refreshing 355–356automatic proxy configuration 373, 376, 379, 416autoproxy.pac 375–377, 379–381autorun 276, 285

Bback-end server 87–88, 90, 92, 95, 102, 107, 127, 129–131, 158, 167, 174, 180, 183, 198, 201, 231–232bandwidth rules 9

Ccache access log 304cache agent 355, 357–358, 397cache content management 351Cache expiration settings 305Cache Expiry Setting 354CacheAccessLog 404–405cacheagt 358CacheByIncomingUrl 461caching proxy component ix, 4–5caching proxy server 4–5, 22, 288, 292, 315, 327CalcServlet 361–362, 364, 366–367, 476CanMonitor 139capacity utilization 9CBR 6–7, 12, 30, 55, 57, 175–178, 235–237, 249, 263, 443–444, 446–452, 455–456, 458–459,

© Copyright IBM Corp. 2001

461–462CBR for HTTP 29, 56, 445CBR proxy for IMAP and POP3 29, 56, 462configuration file 455, 469

CBR proxy 444, 462cbrcontrol 448, 454–459, 469cbrkeys 157–158cbrserver 464–465Cell 133, 137–138, 143, 146cell 133, 146CGI 177, 236, 299, 402Client-IP 314, 317cluster 75, 77, 80, 84–85, 87, 89–90, 95, 98, 101, 206–213, 226, 231, 418, 431, 447–449, 463, 467cluster IP address 80, 90, 95, 99, 101, 107–108, 114, 120, 126, 164, 197, 200, 214, 417, 420, 449–450, 457collocation 9, 158, 160, 416Configuration and Administration forms 249, 258, 263, 270, 282, 288–292, 295, 297, 307, 309, 312, 316, 325, 327, 329, 345, 352, 354, 356, 360, 369, 376, 382, 384, 386, 392–393, 395–396, 428, 437, 461Connections per second on a port 183Content Based Routing. See CBRcookie 235–236, 459–461cookie affinity 7, 235–236, 451, 459–460cross port affinity 233–234CyberLists 338, 345, 347CyberNOT 337, 340–341, 344, 351CyberNot 340CyberPatrol 6, 14, 248CyberPatrol database 345, 347CyberPatrol filtering 336, 338–340, 342, 344–345, 347, 351CyberPatrol plug-in 336, 351CyberYES 337, 340–341, 344

Ddefault route 427, 433default router 426–427, 432, 436delving 355, 358directives 295–297, 300, 304, 307–309, 317,

487

323–324, 326–327, 332–334, 338, 342, 351, 356, 364–365, 390, 397, 404, 407, 446, 461

not refreshed 295disable listener 12dnload.log 351dynacache 13, 359–360, 366dynacache.xml 359–360, 363–365

location 363dynamic cache 359, 361, 365–366dynamic caching 13, 359–360, 362, 364

EECA 359, 363environment variable 52, 58ErrorLog 404–405EventLog 404Executor 83–84, 89, 109–110, 115–116, 120–122, 418, 422, 430, 435, 448, 465expiration time 354, 365external cache adapter. See ECAexternalCacheGroup 363–365ExternalCacheManager 365

FFILTER_DEBUG_ON 351FindProxyForURL 374firewall 7, 19–20, 202–203, 367–368, 385fixed weights 167–168, 170–172, 174flexible-client SOCKS 367–369forward proxy 249–250, 256, 262, 268, 275, 280, 384

Ggarbage collection 304–307

bandwidth 306–307blend 306–307responsetime 306

GenCyberDB 343, 349–350Generic Routing Encapsulation 10gigabit Ethernet 8GRE 10GroupFile 325–326, 335

HHACMP 7heartbeat 105, 110, 113, 115, 118–119, 122–126, 149, 220, 230

high availability 7, 16, 18, 104–110, 112–113, 115, 117–120, 123, 125–128, 158, 206, 208–209

backup machine 104–107, 110, 113–115, 117–119, 121, 123–127mutual high availability. See mutual high avail-abilityprimary machine 105, 107–108, 110, 112–116, 118–119, 121–126

High Availability Cluster Multi-Processing 7high availability scripts 114, 120, 210, 213–214, 219, 222

goActive 120–121, 213–214, 217–218goIdle 120goInOp 120–121, 213, 216, 219goStandby 120–121, 213, 215, 218

htadm 291–292, 325htcformat 249, 263, 275, 301–302HTML header 314

IIBM AIX Developer Kit, Java 2 Technology Edition 28IBM Cross Platform Technologies for Windows V2.0 40IBM Runtime Environment for Linux 55IBM SecureWay Directory 327IBM SecuryWay Directory 328ibmproxy 260–261, 284, 294ibmproxy.conf 289, 294–295, 299, 304

adding administrators 291basic configuration 288, 307, 309basic user authentication 326CBR 446–448, 461CyberPatrol 338, 351dynamic caching 360, 364–365garbage collection 307host name 296HTTP header options 316, 318–319LDAP authentication 329–330, 332, 334, 336location of 295logging 407PidFile 297proxy chaining 390

ICP 14, 417ifconfig 87, 91, 100–101, 200, 224, 420IMAP 444, 462–464, 467inittab 95InstallShield Wizard 31–32, 43, 58–59, 252, 260,


265, 271, 276, 284–285Interactive Session Support, See ISSInternet Caching Protocol. See ICPInternet Explorer 41, 309–310IP alias 85, 90–91, 95, 100, 108, 114, 120–121, 126, 164, 199–200, 209–210, 214–216, 224, 227, 231IP ranges 13iptrace 316, 439ISS 6, 11, 30, 93, 102, 127, 129–136, 146, 149

agent 11, 127, 130–131cell 7, 132–133, 137–138, 141, 143, 146, 149–150, 152daemon 133–134, 136, 149high availability 132, 138monitor 11, 127, 129–131, 136, 138, 146, 150

isscontrol 132, 150isskeys 157–158

JJava Runtime Environment 28, 40, 57, 249, 263JavaBean 362JavaServer Pages. See JSPJSP 13, 359, 365

LLDAP 13, 330

database 326, 328, 332directory 326, 330directory server 327, 330, 336plug-in 330plugin 328, 330, 332–333, 335–336

LDAP plugin 332–333, 335Lightweight Directory Access Protocol. See LDAPload balancing component ix, 4, 6, 18local area network 7log archiving 407–408log files 335, 391, 394, 405–407logging

Network Dispatcher 237Web Traffic Express 390

loopback adapter 96–99, 101, 200loopback interface 95, 215–216, 450

Mmailbox 462, 469Manager 88, 93, 132, 167–168, 432, 454, 468

manager proportions 89, 93–94, 122–123, 132MaxActiveThreads 295, 397–398MaxExpiryTime 365maximum requests 399memory caching 249, 262, 275mutual high availability 206, 208, 213, 220–221

backup machine 207, 210primary machine 208, 210, 212, 214

Nndadmin 38, 65, 71, 135ndconfig 92ndcontrol 94, 102–103, 113, 117, 124, 126, 165–166, 172, 223, 226, 234, 238–240, 422, 435ndkeys 157–158ndload 11ndloadstat 11ndserver 38, 65, 82, 95Netscape 249, 263, 295, 309Netscape Communicator 28, 41, 56, 309, 379, 422, 434Netscape Directory Server 327Netscape Mail 470Netscape Navigator 28, 41, 56, 373netstat 109Network Dispatcher

Advanced features 155Basic configuration 69

command line 89configuration wizard 72–73GUI 81

Installation 27Custom 29, 57Express 29, 56Guided 29, 56

Network Interface Card 28, 41, 55new connections 93NFA. See non-forwarding addressnon-advertising address 95non-forwarding address 89–90, 122–123, 161, 197, 430Novel NDS eDirectory 327

Oobserver 146, 148

Index 489

PPAC file. See proxy automatic configuration filepac.conf 327, 330–333, 336PAC_DEBUG_LEVEL 335pacd 330, 332, 335pacd_restart.bat 332, 335pacd_restart.sh 332, 335PasswdFile 325–326, 334–335persistent connection 399, 401–402, 404Persistent Connections Timeout 399PICS 6Platform for Internet Content Selection. See PICSPOP3 444, 462–464, 466–467, 469–471port 76, 87, 89, 102, 109, 115, 162, 177, 195–196, 199, 221, 231, 233–234, 251, 257, 269, 281, 289, 418, 426–427, 430–432, 437–438, 447, 454, 467port specific 93PreExit 446Privacy Settings 316–317private key 157–158process ID 261, 285protection setup 321, 323–325proxy 336, 375, 388proxy access log 299, 312, 437proxy automatic configuration file 373–377, 380–381, 383

location of 379proxy chaining 384–386Proxy Header Filtering 318–319Proxy server 312proxy server 5–6, 13, 16, 19–20, 251, 255, 267, 279, 291, 297, 299–300, 304, 308–310, 334–336, 355, 364, 366–368, 372–373, 375, 377, 382–385, 387–390, 403–404, 415–421, 423–424, 426, 430, 447, 449ProxyAccessLog 404–405public key 157–158pure proxy 300, 403

Qquad-port Ethernet 8quiesce enhancement 11

Rraw disk 301–303RCA 5, 21, 417reach target 110–111, 116, 220–221Recovery strategy 110, 113, 115, 118–119,

123–127, 221Auto 105, 110, 113, 115, 118–119, 122–126Manual 105, 110, 123–124, 127

Recreational Software Advisory Council on the In-ternet. See RSACIRedbooks Web Site 480

Contact us xiReferer 314registry 54remote administration 29, 56Remote Cache Access. See RCAremote configuration 156, 158Remote Method Invocation. See RMIresource type 141–143reverse proxy 249, 251, 262, 275RMI 156, 158round-robin 7route delete 99–100routing table 99–100, 165, 438, 441RSACI 6rule affinity override 235rules, how they are evaluated 179rules, types of 176

Always true 178, 184, 186–187, 235Capacity and bandwidth rules 178Capacity or bandwidth rules 184Client IP address 176Client port 177, 184Connections per second on a port 176Content of a request 177Time of day 176Total active connections for a port 177, 184Type of service 178, 184

rules-based load balancing 175Begin range 184End range 184Level to evaluate 185Level to share available bandwidth 185One or more server addresses 185, 187Priority 184, 187Rule name 184, 187TOS 185

SSDA 232SDA agent 232–233Secure Sockets Layer 12security policy 13


Selecting 45Server Activity Monitor 382–383, 390–391, 393–397

Access statistics 391, 394Activity statistics 391–392, 400Cache refresh summary 391, 397Cache statistics 391, 396Network statistics 391, 393Proxy access statistics 382–383, 391, 395

Server Directed Affinity. See SDAserver monitor 104Server weight field 171ServerInit 446servlet 359, 361, 364–365servletcache.xml 359–360, 364SMIT 31, 248, 261SOCKS configuration file 368, 370, 372, 404SOCKS server 367–370, 372–373, 388, 390, 404SSL 12SSL Tunneling 299startsrc 261stickymask 233–234stickytime 11, 231–236, 419–420, 451–452stopsrc 260Summary 48SurfControl 14, 337–338, 345, 347SurfControl database 336, 338syslog 406system metrics 93

Tthread 177, 392, 397–400, 404timeout 365, 399–402TimeServlet 361–362, 367, 476Tivoli SecureWay Firewall 368transparent proxy 249, 251, 262, 268, 275, 404, 425–429

Uuser administration 320user authentication

basic 320LDAP 326

User-Agent 314, 317–320

WWAS 13, 70, 359–365, 372

Web Traffic ExpressAdvanced features 313Basic configuration 287Installation 247

WebSphere Application Server. See WASWebSphere Edge Server for Multiplatforms 4–8WebSphere Performance Pack 4wide area network Dispatcher 191

local Dispatcher 194–195, 197, 199–201, 204–205remote Dispatcher 195–202, 204–206

wide area port number 195, 199wildcard cluster 425, 430–431, 435wildcard port 431, 438WTE registry 326, 329WteAdapter 362–364WteMetaDataGeneratorImpl 362–364

XX-Windows 31, 58

Index 491

(1.0” spine)0.875”<

->1.498”460 <

-> 788 pages

WebSphere Edge Server: W

orking with W

eb Traffic Express and Netw

ork Dispatcher

®

SG24-6172-00 ISBN 0738421812

INTERNATIONAL TECHNICALSUPPORTORGANIZATION

BUILDING TECHNICALINFORMATION BASED ONPRACTICAL EXPERIENCE

IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment.

For more information:ibm.com/redbooks

WebSphere Edge Server: Working with Web Traffic Express andNetwork DispatcherConfiguration of WebSphere Edge Server

Scenarios for Web Traffic Express and Network Dispatcher

Details of latest enhancements

As the Internet economy evolves, it is no longer enough to simply establish an online presence to sell your products and services. To compete in this fast-paced, global marketplace, you need to respond rapidly and intelligently to changing application and network demands while reducing the complexity of implementing new e-business solutions. Availability, scalability and performance are critical to ensuring that your e-business runs smoothly and cost efficiently.WebSphere Edge Server for Multiplatforms allows Internet service providers (ISPs) and enterprise customers to confidently host mission-critical Web sites for intranet and Internet e-business applications. WebSphere Edge Server brings together innovative caching, local- and wide-area load balancing and application-aware quality-of-service routing capabilities. The integration of these functions and their wide array of features can help reduce cost for the site owner while providing improved customer satisfaction.This redbook will help you install, tailor and configure the new features and enhancements included in WebSphere Edge Server for Multiplatforms. It is intended for networking personnel and information technology administrators who plan to use WebSphere Edge Server to provide Internet access to end users and distribute Web-accessible content more efficiently.

Back cover




websphere edge server: working with web traffic express and network dispatcher

Documents