intel technology journal q1, 2000 · intel technology journal q1, 2000 preface lin chao editor...

Intel Technology Journal Q1, 2000

Preface Lin Chao Editor Intel Technology Journal Intel's mission for 2000 is to "do a great job for our customers, employees, and stockholders by being the preeminent building block supplier to the worldwide Internet economy." Our corporate objective has undergone a significant change with this mission. While Intel will continue to grow our core processor and computing business, we also will focus on growing new Internet businesses and market segments. The buzz at Intel and throughout the industry seems to be the changes brought about by e-Business on traditional "brick and mortar" companies. An example of an e-Business is Intel's Web-based order management system, introduced in July 1998. In January 1998, Intel had no online customers. Orders were filled through phone, fax, and overnight parcel carriers. By June 1999, over 560 companies in 46 countries were using Intel's Web-based order management system to place orders, track deliveries, post inquiries, and get product and pricing updates. Today, this system produces nearly $1 billion in sales per month. By 2001, Intel projects that more than 90% of its orders will be handled over the Web. The shift to e-Business is a key driver of current business strategy and opens up many new opportunities for Intel. In this Q1'00 edition of the Intel Technology Journal, we will look at how Internet information technologies (IT) are enabling e-Businesses at Intel and beyond. The first paper describes Intel's own Internet connectivity architecture including the business factors that led to its creation and the technologies developed to build and maintain it. The second paper talks about factory indicators using near real-time supply chain performance indicators from a variety of manufacturing sources. These indicators provide the necessary information to run day-to-day operations and avoid or detect problems. The third paper describes corporate Intranets and how to manage information overload. The fourth paper looks at business-to-business (B2B) specifications for an industry-wide standards-based approach to e-Business known as RosettaNet*. The fifth paper looks at policy-based management of network services. It explores the dimensions of policy-based management, provides a pragmatic review of the technology, and discusses the deployment challenges and usage scenarios.

Copyright © Intel Corporation 2000. This publication was downloaded from http://www.intel.com/. Legal notices at http://www.intel.com/sites/corporate/tradmarx.htm

http://www.intel.com/technology/itj/chao_bio.htm

Applying Information Technology to Enable e-Business at Intel

By Doug Busch, Vice President and Director, IT

Intel Corp.

Intel is rapidly becoming dependent on information technology for every aspect of its operations. Our core business processes, like those in most large companies, have been enabled by information technology for many years, but a critical change is in progress. Many of our most critical business strategies are now based on taking advantage of Internet technologies to allow integration with our customers and suppliers, to enable electronic collaboration between our employees, and to dramatically increase the speed and efficiency of our business processes. Throughout Intel, groups are developing and executing strategies for improving business using these tools. Major initiatives are focusing on comprehensive business-to-business and business-to-consumer marketing and sales; cost improvements through supply chain integration and more responsive production planning; improved virtual factory operations; and simplified delivery of services to employees.

Worldwide, this dramatic transition is happening throughout the business world. The shift to electronic business, or e-Business, is the key driver of current business strategy. e-Business is the principal opportunity, and in the hands of competitors, the principal threat facing every business. Competitive forces are driving incredibly rapid adoption of e-Business: companies are expanding their computing infrastructure to participate in e-Business, and consumers are buying and upgrading their PC's and network

services to take advantage of the growing range of online businesses.

As a preeminent supplier of building blocks to the Internet economy, Intel's product and service strategies have been profoundly impacted by this transition. We are expanding our focus from clients and servers to encompass networking components and Internet services. Critical to Intel's success is the understanding, as a company, of not just how to design and manufacture these Internet building blocks, but of how to use them. Insight into the challenges of building an e-Business will enable us to deliver the best possible products and services to our customers.

Intel is recognized as a leader in the implementation of e-Business systems, conducting more than $1 billion of business-to-business commerce every month, and we intend to maintain that leadership through the aggressive yet pragmatic implementation of new e-Business technologies. The experience gained from building the e-Business infrastructure is one of Intel's most important assets. We deliver Internet connectivity to 65,000 employees worldwide, every day. We deliver information to our sales channel and customers in every location over the Internet. We are increasingly interacting with our suppliers over the Internet. In the process, we're learning first hand what it takes to build reliable, manageable systems; how to ensure they can change rapidly and scale up to meet bursts in demand; and how to integrate

Applying Information Technology to Enable e-Business at Intel 1

http://www.intel.com/pressroom/kits/bios/busch.htm


hundreds of companies into a virtual enterprise.

In this issue of the Intel Technology Journal you will find some of the lessons we have learned in the process of making Intel an e-Business. I hope these lessons are of value to you as you develop the products, services, and business strategies that will take you into the next millennium.

Copyright © Intel Corporation 2000. This publication was downloaded from http://www.intel.com/. Legal notices at http://www.intel.com/sites/corporate/tradmarx.htm

Applying Information Technology to Enable e-Business at Intel 2

http://www.intel.com/

http://www.intel.com/sites/corporate/tradmarx.htm

http://www.intel.com/sites/corporate/tradmarx.htm

Cockpit: Decision Support Tool for Factory Operations and Supply Chain Management 1

Cockpit: Decision Support Tool for Factory Operations andSupply Chain Management

Paul Calame, Assembly Test Manufacturing Information Systems, Intel CorporationRavi Nannapaneni, Assembly Test Manufacturing Information Systems, Intel Corporation

Scott Peterson, Assembly Test Manufacturing Information Systems, Intel CorporationJay Turpin, Assembly Test Manufacturing Information Systems, Intel CorporationJames Yu, Assembly Test Manufacturing Information Systems, Intel Corporation

Index words: supply chain, decision support systems, performance indicators, OLAP

ABSTRACTWe live in the Internet age, where businesses like Intelsell, buy, and manage internal operations in afundamentally different way than just a few short yearsago. Speed and agility have been added to the businessperformance equation: those that are fast and agileprosper, those that are not, don’t. Our customers demandthe ability to place and amend orders at will, requireminimum inventory stock and just-in-time delivery, anddesire order status visibility into the Intel supply line.These expectations drive internal demands for near real-time supply chain performance indicators from a varietyof sources, a potential problem alert capability, and alook-ahead and environmental scan capability to supportforward business planning.

This paper addresses one Intel program that focuses onsatisfying these information needs with the specificobjective of improving manufacturing operations andsupply line performance.

The new implementation is called Cockpit reflecting itsability to provide the pilot with the necessary informationto run day-to-day operations and avoid or detectproblems. Based on industry standard technologies (e.g.,OLAP‡), Cockpit is a new breed of Decision SupportSystems (DSS) and one that has earned the confidence ofIntel manufacturing professionals.

The Cockpit team shares the development process thatled to the production system: the initial challenges, thedevelopment approach, the technology choices, the

‡ OnLine Analytical Processing (OLAP).

enterprise-wide data partnerships, and the variousarchitectural details that make up the system.

INTRODUCTIONIt’s a brave new world in Intel manufacturing. Shrinkingproduct and technology lifecycles, pricing pressures,supply line and segment management requirements,increased manufacturing complexity, and the impact ofemerging e-Business are just some of the new challengesfor Intel’s back-end components manufacturing groups.Coupled with a geographically dispersed, 17 time zonenon-stop Virtual Factory◊ operation, the need forbreakthrough management strategies becomes imperative.The following questions are posed:

• How does manufacturing become more responsiveto customer delivery, flexibility, and cost needs?

• How do we transfer and ramp products faster withlow variability?

• How do we reduce cycle time to World Class, andstatic inventories to zero?

• How do we proactively and dynamically allocateproduction capacity?

In the Technology Manufacturing Group’s (TMG)Assembly Test Manufacturing (ATM) organization, thesechallenges are being addressed through a variety ofinitiatives such as supply chain management; betterforward planning; asset fungibility and utilizationimprovements; and people, team, and system resources.

◊ A Virtual Factory (VF) is a group of factories makingthe same product and managed as a unit.



To enable many of these initiatives, InformationTechnology needs to be exploited. Assembly TestManufacturing Information Systems (ATMis) is one ofthe programs chartered to deliver the necessary enablers.The ATMis mission is to facilitate manufacturingperformance breakthroughs by developing and deliveringthree key capabilities:

• Business operations and planning data to run factoryoperations, solve problems, and make decisionsfaster.

• Real-time and asynchronous collaboration toimprove virtual work team efficiencies, and tomitigate work-team burnout.

•• A virtual information and knowledge environment toenable intra- and inter-organization sharing andreuse.

In this paper, we discuss one of these tools developed byATMis, the ATM Cockpit. The goal of the Cockpit teamis to promote more effective use of ATM resources andassets as follows:

• Deliver management-level data and informationneeded to run VF manufacturing and supply lineoperations.

• Reduce time and overhead costs to obtain thisinformation.

• Provide tools and services in half the typicaldevelopment time at minimum new cost and Cost OfOwnership (COO).

The Cockpit was designed to support three ATMcustomer segments: the ATM/TMG executive team, theATM operations management team, and the VFTechnology management team. Its specific deliverablesare as follows:

• Put factory and supply-line historical status and look-ahead information from a variety of disparatedomains at the consumer’s fingertips.

• Provide a potential problem alert and notificationutility.

These deliverables are provided by way of features suchas data drill-down and graphical display, a personalizationutility, and an easy to use Web-based front-end.

A technical core team was assembled to drive theprogram and work with IT and company data providers tocreate a standards-based architecture and technicalframework. The team employed a small group of keycustomer stakeholders to guide implementation, utilizedrapid application development and deployment (RADD)techniques to fast-track development, and employed

software already licensed by Intel to create and deploythe product in time. Using an add value, ease the painnow deliverables approach, the team produced anddeployed the pilot product in six months; subsequentupdates were done in 90-day cycles. Since data is crucialto customer decision making, significant effort was paidto modeling and validation of source data. The tool iscurrently employed in ATM OLGA∝ /Flip Chip VirtualFactories and in ATM operations management.

APPLICATION OVERVIEWThe ATM Cockpit is a Web-based tool that allows usersto create reports on-the-fly. The Cockpit allows end-users to design reports known as views by selecting easy-to-use navigation controls. Users can define and savepersonal views or drill-down from high-level to detailedviews. Saved views can be accessed on the user’sCockpit front page by clicking a thumbnail report icon.Alternatively, users can subscribe to these views whichare then sent to their desktop. The Cockpit also allowsusers to manage business through an exceptionmanagement system, which monitors predefined andpersonalized indicator trigger points. The tool notifiesusers when an exception triggers a fixed value or deviatesfrom the goal. Additionally, the Cockpit provides acomplimentary real-time event delivery mechanism thatincorporates e-mail and channel-based notification.Configurable to individual needs, the delivery mechanismscans the business environment pushing news and otherevents of interest to the user for convenience and easyaccess.

MANUFACTURING CASE STUDYThe following case study⊕ demonstrates how theintegrated Cockpit application directly impactsmanagement’s day-to-day decision-making process. Thiscase study focuses on a specific instance of how a VFproduct manager uses the Cockpit to identify and confirma potential yield issue and proactively take the necessarysteps to correct the problem.

∝ Organic Line Grid Array.⊕ These examples are for informational use only and donot represent actual data from Intel.



Figure 1: Cockpit user’s customized front page

Figure 1 shows a typical Cockpit product manager’scustomized front page with thumbnail icons displayed.These icons represent favorite views the manager wouldwant to see on a daily basis, providing insight intobusiness operations. Each manager specifies whichthumbnails to display by selecting from a standard list ofdefined views, or a list of personalized views previouslycreated and saved. With one mouse click, the managercan select any of the thumbnails to examine the fullactive view displaying data from the most recent OLAPcube update.

Figure 2: Manufacturing-specific customized views of“VF Forecast, Yield by Products” and “VF Forecast,

Yield by Output”

Scrolling down the front page, the manager reviews theVF Forecast, Yield by Products view. Focusing attentionon detail, the manager sees that the VF is not meetingyield goals.

From a manufacturing perspective this means that the VFwill either miss committed output or require moreproduct starts to stay on target. The manager elects todrill-down on yield to further analyze current activitiesand clicks the right thumbnail, VF Forecast, Yield byProducts to display the active view of the data shown inFigure 3.

Figure 3: Drill-down in Active View to reveal Yieldtrending for specific product

Shifting the time frame to the current quarter, the graphdisplays a downward trend. With the ability to easilychange time frames and indicators and dimensions toelaborate on the current problem, the manager deselectsall products except ProdX, which seems to be presentingthe problem.

Figure 4: Expanded data view revealing increasedproduct yield in Technology Development (TD) sites

In Figure 4, data were expanded so that they could beviewed by site to determine if this downward trend isunique to a site, or if the problem is spread across theentire VF. By deselecting other sites, the active view canbe simplified to focus on specific sites highlighting theyield anomaly. Analyzing apparent indicators, themanager concludes that all High Volume Manufacturing(HVM) sites are displaying a downward trend whileTechnology Development (TD) sites display an upwardtrend. This prompts questioning the engineering managerto determine what has changed since the last quarter andwhether TD is developing some new Best KnownMethods (BKMs). Electing to analyze the data further toelaborate on possible options, the manager modifies thetime frame of the chart a second time to provide a largerwindow. Looking at prior quarter performance to analyzehow yield was trending for the product, the managerevaluates that ProdX was steadily improving as expected.



Figure 5: Enlarged time frame view revealing steadyimprovement with significant spike in Q2 yield

The manager now knows, as expected, that ProdX yieldat HVM sites was improving as the product matured. Atquarter end, however, all yields at HVM sites went down,but TD remained constant and even showed a slightincrease.

Looking at a yield trend, several options are available tothe manager:

• Determine course of action to improve yield.

• Consider modifying forecast (lower) to avoid over-commitment.

• Maintain the output forecast and either over-startunits or lower Throughput Time (TPT) tocompensate for lower yields.

In Figure 5, noticing that yield improved as expected witha maturing product, the view displays a visible knee at the2nd quarter boundary. The manager wonders what newor different actions occurred: new stepping, new die,engineering change, etc.

Figure 6: Modified view to “VF Output, Actual vs.Scheduled” to reveal Yield-Output correlation

Electing to correlate the previous trend, the managerswitches to VF Output, Actual vs. Scheduled displayingthe same time period as the previous yield view. Theanalysis validates that while yield trended up through the

end of the quarter, the VF consistently beat the outputschedule.

When yield trended down, at the beginning of the newquarter, the VF consistently missed the output schedule.The manager concludes that yield and output arepositively correlated. With this new information,management can take adequate action to improve yield.

Through the Cockpit’s interactive live charts and drill-down capability, a correlation that could have taken days,even weeks to identify was detected in a timely manner,demonstrating the benefits of the Cockpit as a decisionsupport system.

To facilitate early notification of similar yield-relatedproblems, the manager can request that the applicationmonitor the yield indicator. The manager can set apersonalized threshold and be notified if yield is outsidethe acceptable range. This early notification will shortenthe problem-solving cycle time without requiring constantindicator monitoring.

COCKPIT ARCHITECTURE

Architecture Design Guidelines

The single most important attribute of a softwarearchitecture design is the ability to handle change. TheCockpit’s architecture is designed to adapt to ever-changing technology, user base, and businessrequirements. Optimized for application-specificrequirements, the criteria driving the Cockpit’sarchitectural design and technology selection included thefollowing:

• maintainability, scalability, extensibility, andinteroperability

• multidimensional query support and drill-downcapability

• highly interactive active and static views

• various levels of user sophistication

• guaranteed and consistent performance for LAN,WAN, and off-line environments

• push/pull information delivery, coupled with theability to provide scanning and notificationinformation in near real-time.

The ATM Cockpit addresses these system andapplication-level requirements through adoption of an n-tier Internet architecture, utilizing subsystem interfacesthat conform to open industry standards and the Intelinternal architecture reference model.



The Cockpit’s component architecture is based onWindows∗ Distributed interNet Applications (DNA)∗ , andits decision support implementation is consistent withMicrosoft data warehouse and Intel DSS Utilityarchitecture frameworks.

Overview of Component Architecture

The Windows DNA backbone provides the foundation forthe Cockpit’s Component Object Model (COM) basedcomponents to easily deploy, scale, and interoperate inthe Internet environment. Based on Internet Explorer∗ ,the Cockpit’s presentation layer relies on MicrosoftOffice 2000∗ Web Components (PivotTable andPivotChart) to present multidimensional data in table andgraphical formats. Custom Cockpit ActiveX∗ objectsprovide user-friendly navigation that drives Office 2000∗Web Components. These components facilitate seamlessintegration with Excel 2000∗ . For future extensibility ofthe presentation layer, the Cockpit’s architecture supportsreplacement of these components with other vendorsolutions. The Cockpit’s business layer provides thesecurity and personalization services necessary to ensurethat the Cockpit application is secure and easy to use. Theabove services are provided through server-sidecomponents built on Site Server and Active Server Pages(ASP).

Site Server offers data persistence to the Cockpit’spersonalization information through a Lightweight DirectAccess Protocol (LDAP) interface that provides themigration path to the future Windows 2000∗ ActiveDirectory∗ service.

Figure 7: Reusable framework components for theCockpit UI Indicator Reporting module

∗ Other brands and names are the property of theirrespective owners.

Reusable UI Framework

The largest benefit of the Cockpit’s architecture isrealized through the use of OLAP technology. TheOLAP engine encapsulates multidimensional queryprocess complexity from the Cockpit’s user interface (UI)and business logic. Communication between client-sideOffice 2000 Web Components and OLAP cubes (data)are handled through the OLE DB† for OLAP (PivotTableService) database standard on the client.

The abstraction layer, supplied by client-side PivotTableService, allows the Cockpit client to connect to OLAPdata on the server over the LAN, across a WANconnection, or on the user’s PC. This abstraction layeraids in insulating the UI from the data source.

Any OLAP cubes implemented by other organizations aredirectly accessible from the Cockpit’s UI and middlelayer components. Other non-OLAP data sources or datastructures can be accessed through a custom OLE DB forOLAP Provider.

Consequently, the Cockpit’s UI and middle-tierimplementation can be used as an enterprise UIframework for all OLAP applications within thecompany. This helps the bottom line by avoidingredundant cross-company development time and cost.

The Cockpit User Interface

Figure 8: Reusable framework components for theCockpit UI Scanning and Notification module

The Indicator Reporting ModuleThe Cockpit’s Indicator Reporting module allows accessto all information contained in the Cockpit OLAP

† The OLE DB for OLAP (PivotTable Service) databasestandard from Microsoft offers an SQL client/serversolution with a standard access method to data and theability to use various front-ends.



database. When we were considering the look-and-feelfor the Cockpit user interface (UI), we initially reviewedseveral commercially available tools that provide Web-based OLAP analysis. Although these tools weregenerally quite capable, further research revealed severalnotable problems when it came to the Cockpit’s businessrequirements.

Firstly, these tools are aimed at users with OLAPexperience. The Cockpit’s customer base of seniormanagers with minimal or no OLAP experience wouldhave to undergo significant training.

Secondly, these tools allowed for little customization andwould require a comprehensive single vendor solution.Clearly, our implementation would benefit from acustomizable front-end and the ability to interoperatewith industry standards at all levels of the application.

The Scanning and Notification ModuleThe Scanning and Notification module, as the namereflects, scans Intel’s internal and external environmentfor potential problems and events of interest, and reportsthe results to the user via a push mechanism. The currentimplementation uses an active channel approach todeliver events to the user. This implementation requires aseparate front-end and a small stock-ticker-like tool to runon the user’s machine. We are exploring other simplerways to disseminate the event information; MicrosoftOutlook∗ and e-mail are two possibilities.

Overview of DSS Architecture Framework

The Cockpit’s back-end data services comply with Intel’sDSS Utility architecture framework. Intel IT DSS Utilitypartners provide OLAP cubes that are suitable formultidimensional query capability. These cubes adhereto an established extract, transform, and load process.All Cockpit indicator data resides in databases optimizedfor OnLine Transaction Processing (OLTP) in themanufacturing execution and other operationalenvironments. Extracted data are sent through acleansing process for storage in the corporate-wide datawarehouse.

The Cockpit dependent data mart• is derived from thecorporate-wide data warehouse. OLAP cubes are built ∗ Other brands and names are the property of theirrespective owners.• Dependent data mart is a type of data mart that acts asa subset of a larger data warehouse which combinesdatabases across an entire enterprise. Data marts areusually smaller than larger data warehouses and focus ona particular subject or department.

from Cockpit’s staging data mart. The resulting cubescan then be hosted on the same or on a different server tobe accessed by the Cockpit’s UI front-end.

DATA ORGANIZATION

Known Issues

At the outset of this project, we faced a number ofchallenges from a data perspective:

• Data was stored in a number of databases within theorganization. To meet initial customercommitments, we needed to consolidate data fromfour separate systems: two manufacturing WIPsystems, a planning system, and a safety and healthsystem.

• There were no existing sources for common data,such as factory calendars, product masters, orfactories.

• Use of OLAP technologies was relatively new atIntel and the selected product, Microsoft OLAPServices∗ , had just been released. No resources wereavailable to draw on. There were no consultants,books, or classes.

Team Approach

To maximize utilization of resources available within theorganization, and to minimize the number of developerswe would need to hire, we implemented dimensionalmodeling techniques to create the data mart. This provedto be the easiest method to consolidate data fromdisparate systems, allowing for an easily supportableOLAP design. We decided to take a team approach todeveloping the data mart and so did the following:

• Implemented dimensional modeling techniques tocreate the data mart. This proved to be the easiestmethod to consolidate data from disparate systemsbelonging to different groups, allowing for an easilysupportable OLAP design.

• Partnered with IT for technical assistance. ITprocured and configured the hardware infrastructure,provided data loading expertise, and performeddatabase modeling.

• Worked with data owners to understand data sourcesand design data validation routines. We developedcross-departmental relationships to quicklyunderstand data sources. Because these departments



traditionally owned the data, they were interested inensuring that it was accurately reported.

Database Server and OLAP Engine Selection

Due to the development time frame for delivery of thesolution to upper management, little time was available toperform a complete evaluation of various database serverand OLAP engines. With only three months to deliverydate, we selected Microsoft’s SQL Server 7.0∗ and OLAPServices∗ .

Many developers on the team were already familiar withthe SQL Server and knew that it would provide adequatepower to support a database size of 10-20GB.

We then began work on creation of the initial data mart,and we selected dimensional database modeling as thedesign path. Much of the Cockpit’s design was based onthe teachings of Ralph Kimball, an expert in the field ofdimensional modeling [1].

Dimensional ModelingDimensional Modeling (DM) is a logical design techniquethat attempts to present data in a standard, intuitiveframework allowing for high-performance access.Essentially, this is a relational database modelingtechnique, but with some important restrictions [3]:

• Every dimensional model is composed of one tablewith a multi-part key, called the fact table. This tabletypically contains one or more numeric facts ormeasures.

• The model also includes a set of smaller dimensiontables, each having a single-part key. These tablescontain descriptive information about such things asproducts, factories, and time. Dimension tables areused to describe the facts in the fact table.

• When these tables are joined together, they create thecharacteristic star-like structure, commonly called astar join.


Figure 9: Typical star join dimensional model [6]

Dimensional models are inherently additive. Users rarelyever retrieve a single record from the table, but insteadretrieve hundreds, thousands, or even millions of recordsat a time. The only useful operation to perform on such alarge number of records is to add these items [4].

Figure 10 displays the facts Units Shipped. This fact isdescribed at each intersection of the dimension tables; inthis case, Products, Time, and Factories.

By simply filtering the result set, you easily obtain thenumber of units shipped for any combination of productsand factories during a given time period. From an end-user viewpoint, this type of database design is easy tounderstand and use [2].

Figure 10: Dimensional Model composition displayingfact and Dimension Table relationships [2]

Data PopulationPopulating the dimension tables proved the most difficultpart of this task. Most of the work was not technical innature, but required a large amount of business analysisto determine how products are categorized, and how togather the various shift schedules from each factory.

The product dimension was the most difficult table topopulate, due to the need to cross-reference shop floorproduct codes used in manufacturing and planningsystems, with common product nicknames used by most



end-users. This information did not exist in any database.Instead it resided in spreadsheets, e-mail messages, andwith individuals in factories. After gathering the data, webuilt and populated an entire set of cross-reference tablesto maintain attributes in the product dimension table.

Initial customer requirements indicated the need to viewweekly data, with a possible future desire to view dailydata. Viewing data at a factory shift level was not arequirement. During the design, we decided to store it atthe lowest level possible, and chose to store data in facttables at a shift level, just as a precaution. This laterproved advantageous due to a change in customerrequirements requesting shift level data.

After gathering requirements and identifying datasources, data analysis consumed at least 50% of the totaltime allocated to development. Due to the size of thedevelopment window, full up-front data analysis was notpossible. During data analysis, we leveragedrelationships with other groups within the organization toidentify the data experts to help construct the properqueries to extract data. Once data analysis for eachindicator was 70-80% complete, Extract, Transform, andLoad (ETL) work started. Developers in IT usedInformatica* to visually extract data from the source,transform it using a variety of business rules, and load itinto the data mart. Data loading jobs were scheduled toload all dimension tables first, followed by fact tables.

OLAP (On-Line Analytical Processing)

OLAP provides organizations with a means of accessing,viewing, and analyzing data with high flexibility andperformance. First and foremost, OLAP presents data toend-users through a natural, intuitive data model. Second,OLAP accelerates delivery of information to end-usersviewing these multidimensional structures by preparingsome computed values in the data in advance, rather thanat execution time. The combination of easy navigationand fast performance allows end-users to view andanalyze data more quickly and efficiently than is possiblewith relational database technology only. The end resultis more time spent analyzing data and less time analyzingdatabases.

In an OLAP data model, information is conceptuallyviewed as cubes, which consist of descriptive categories(dimensions) and quantitative values (measures). Themultidimensional data model makes it simple for users toformulate complex queries, arrange data on a report,switch from summary to detail data, and filter or slicedata into meaningful subsets. Within each dimension ofan OLAP data model, data can be organized into ahierarchy that represents levels of detail on the data. Forexample, within the time dimension, there can be years,months, and days levels. Similarly, within the geography

dimension, there can be country, region, state/province,and city levels. A particular instance of the OLAP datamodel would have the specific values for each level in thehierarchy. Users viewing OLAP data can move up ordown between levels depending on whether they wantmore or less detail [5]. One common misconception isthat OLAP technology is similar to that of relationaldatabases. OLAP cubes are queried usingmultidimensional expressions or MDX. Though MDXshares a number of keywords with SQL, such asSELECT, FROM, and WHERE, there is little similaritybetween them.

Figure 11: Multidimensional OLAP cube, dimensionsand relational schema [7]

Before engaging in a full OLAP development project, it isadvisable to train developers in the differences betweenthe two. For our team, however, creating OLAP cubeswas relatively easy, mainly due to the design of theunderlying data mart. The majority of issues encounteredinvolved implementing MDX inside of calculatedmeasures. Once data was cubed, data validation becameincreasingly important. Some of the validation wasstraightforward, especially where standard reports existedfor comparison. However, we encountered challengesvalidating OLAP cubes against spreadsheets maintainedby users who gathered information from a variety ofsources.

PRESENTATION LAYER

Application Goals

From an application requirements perspective, theCockpit’s presentation layer has two main goals: topresent an intuitive user interface that allows managers toview key indicators across the VF and to informmanagement of events related to internal business orsupply chain situations.

The Indicator Reporting module addresses the goal ofpresenting an intuitive UI that supports and easilymaintains a catalog of important, personalized data views,



as well as performs ad hoc analysis in support of high-level decision making. The Scanning and Notificationmodule addresses the second goal, informing the managerof events related to either internal business situations orsupply chain related events. These two elements,appearing as two separate applications, work in unisonand share content to provide the total solution.

Ease of Use

Although the goal was to design a very simple UI, wewanted to ensure that the design did not limit theusefulness of the tool as users gained expertise. For userswilling to learn, we needed to provide additionalcapability. To accomplish this goal, Microsoft OfficeWeb Components∗ (MSOWC) were selected as thefoundation for the UI. These components are compatiblewith the underlying data tier and have the majority of thefunctionality provided by other OLAP tools.Additionally, the look-and-feel of the components isfamiliar to Microsoft Office∗ users and is part of the Intelstandard application suite. Nevertheless, thesecomponents are difficult for the novice user to operate.While no knowledge of the tool is needed to start using it,full functionality is available by selecting Expert Mode.To this end, we built a simple-to-use auxiliary set ofcontrols for viewing and analyzing OLAP data, consistingof two list boxes and two tree view controls.

Figure 12: Left panel displays auxiliary set of controlsfor viewing and analyzing OLAP data

Using these controls, users select subject area, choosemeasures to view, and then slice and dice data accordingto dimensions of interest. Additionally, we developed anautomated time dimension matching the Intel calendar.Continuing to manage all OLAP interaction through


MSOWC, these auxiliary navigation controls succeed inhiding details and expert style navigation from the user.

Metadata-Driven UI

Having elected to provide custom navigation controls, itwas important to design the framework so that thenumber of OLAP cubes accessible by the applicationcould be easily increased. To achieve this goal, weadopted a metadata-driven method of handling navigationinformation. When cubes are processed on the databaseserver, several XML files are created to include metadataabout each cube that is required for the UI to populatenavigation controls.

To include a new cube in the application, it needs only tobe designed according to Cockpit guidelines and hostedon the local server. A few simple modifications to anXML file will cause the UI to present the cube and all ofits information to the user through the same user friendlyinterface.

Scalability and LAN/WAN Performance

Though the Cockpit interface was well received andprovided quality performance for those with high-speedconnections to the server, there were significantperformance problems for those across long WAN links.This is due to our adopted rich client model. If users inAsia tried to access the Cockpit’s cubes hosted on aserver in California, response times for certain operationsincreased by several orders of magnitude. Moving allprocessing to the server-side would have taken away thevery interactive experience provided by the rich clientmodel.

Figure 13: Metadata-driven method for navigationwith XML files created during processing on

database server and distributed cubes forperformance

Instead, we chose to replicate the OLAP cubes on anarray of OLAP servers at each remote site. Since the



presentation layer was primarily data-driven, we wereable to add an entry into the XML cube description filesto specify servers that had cube copies. Now, when auser connects, the UI can connect to the closest databaseserver. This has significantly reduced response times atremote sites without requiring major changes to thepresentation layer.

Multiple Form Factors

One of the original goals of the Cockpit application, toprovide a rich and interactive user experience whileanalyzing OLAP data, led to the decision to hostMSOWC in the browser and handle the majority of reportgeneration on the client.

However, as the application’s user base increases and thedata scope widens, it will become necessary to supportvarious client form factors in the near future. Theseclients may include Internet Explorer 5.0∗ hosting allActiveX controls locally, a pure HTML browser with noActiveX control capability, or even a Windows CE∗ orPalm OS∗ device.

All of these form factors should be able to benefit fromCockpit content while providing users with an appropriatelevel of interactivity. To support these multiple formfactors, Cockpit components are capable of providingmost of the reporting content in static image formatconsumable by almost any client.

END-USER PERSONALIZATION

Importance to Manufacturing

The Cockpit’s target users belong to such domains asmanufacturing, planning, logistics, materials, finance, andhuman resources. These users play different roles withineach of these domains. For example, the manufacturingarea includes factory and site managers, processengineers, quality engineers, and others with each roleviewing data from a different perspective. The OLAPsolution offered by Cockpit is optimal for slicing anddicing data, allowing users to pinpoint views of interest.It provides factory-level output figures to factorymanagers and provides process engineers interested in thesame output with detail-level information for keyconstraint equipment.


Personalization Engine

Instead of providing a fixed number of static reportscategorized by user types/roles, we elected to enabledynamic report generation, usage and retrieval. Tosupport this functionality, the system should remembereach user and maintain a user-specified personalconfiguration store. The personalization engine, amiddle-tier service based on Site Server, handlespersonalization in the Cockpit. All registered Cockpitusers have a dedicated account defined in the Site Serverdatabase with each user account holding four datasegments:

User Defined Views: Users can create their own reportson-the-fly and save them for later retrieval. The user’spersonalization store saves, in XML format, only thenecessary connection and parameter information. Whena user selects a saved view, XML is retrieved from thepersonalization store and creates the report using themost recent OLAP cube data.

Front Page Views: Users have the ability to specify whichof their favorite views will be displayed on the Cockpit’sfront page. Generated on the server, these views arerendered as images and served to the user’s machine ondemand.

Subscription Views: Users can receive selected views asimages by e-mail or desktop channels.

User-Defined Exceptions: The normal trend is for usersto analyze various static or active views and determinethe current state of operations. Manual in nature, thisprocess consumes valuable time. Instead of requiringusers to scan reports and make decisions, the Cockpit’suser-defined exception facility is designed to automatethe report-checking process. Users have the ability to setexceptions at the indicator level, instructing the system tocheck for exceptions each time the data set changes. Thesystem notifies the user via e-mail or desktop channelswhen it detects an exception, eliminating the need formanually checking of reports to identify the exception.

Scanning and Notification

The Indicator Reporting module excels at providing easyaccess to a wide array of data in the OLAP database.However, all data provided by the tool exists within thecompany, is quantitative, and requires users toproactively search for information. The Scanning andNotification module was designed to compliment theIndicator Reporting module by scanning the businessenvironment for relevant qualitative and quantitativeinformation, and pushing this information to the user inreal-time.



Figure 14: Scanning and Notification modulearchitecture

Scanning EngineThe scanning engine is an automated agent that monitorsa variety of sources for information on behalf of the user.The information gathered by the agent is passed to theuser through a variety of channels. The two primarytypes of information monitored by the agent are Web-based news feeds and internal databases. News feeds arepre-defined by a system administrator and organized intopublic channels with sub-channel categories. If thechannel is of interest, the user can subscribe to it.

For example, there is a channel that monitors news feedsfor information regarding our largest customers. Theagent delivers news headlines to the user for scanning. Ifan article is of interest, the user can click through to thecomplete text. Additionally, the tool can monitor internaldatabases, including OLAP cubes behind the CockpitIndicator Reporting module, for a mixture of personalizedand predefined trigger points.

For example, if the user is always interested in the yieldof a single product at a manufacturing site, that user cancreate and track the exception. An exception wizard willwalk the user through the process of choosing whichconditions to monitor. Generally, an exception can betriggered if it crosses a fixed value or deviates from agoal by a certain percentage. Assuming all the necessaryexceptions are being monitored, users do not need to pollthe Cockpit’s Indicator Reporting module to ensure thefactory is running within acceptable limits. Instead,managers can be assured that they will be notified whenthey need to take action.

Delivery ChannelOnce the information is found, it needs to be pushed tothe user in an easily consumable, timely way. Since typeof information varies by user, the Cockpit provides avariety of methods for pushing this information to theuser. The first method, pushing information through e-

mail, is appealing since users are comfortable with thetechnology, and it provides a fairly real-time deliverymechanism.

Figure 15: The Cockpit Scanning and Notificationmodule, desktop event delivery mechanism

The downside is that important notices may get lost in theInbox along with the huge volume of mail. Additionally,it is not very efficient for news headline scanning. Toovercome these shortcomings, a second method is offeredthat provides a faster response time and informationcategorization, but requires a new application.

This application runs as a desktop event deliverymechanism organized according to channels andcategories mentioned earlier. It uses a true pushmechanism that notifies the user immediately when newinformation is available just by changing icon colors andalert lights on the application. Users can view thisinformation at their convenience. These two choices arejust the first of many that may become available.

Future implementations could easily include suchchannels as a Web portal or public folders. Sinceinformation agent and delivery channels are separate, thissame information could be pushed through almost anychannel. Users will be able to select the channel which isright for their choice of information and is the mostcomfortable.

BENEFITSImpacts of the tool on manufacturing operations havebeen immediate, and are both quantitative and qualitative.A problem or issue that once took 3-4 days plus manypeople and multiple meetings can now be identified inless than 30 minutes. A factory manager can now seeupstream in the production pipeline to view the status ofproduction moving his or her direction, allowing forbetter planning, more agile response to change, etc. TheVF management team now works off the same data fordecision making and problem solving. Instead of amanager directing a subordinate to go to the source data



system and generate a useful summary, managers cannow generate the data for themselves, instantly. With theCockpit as enabler, the ATM VF management team canperform better, faster, and cheaper than before andproactively respond to dynamic demand and supply chainoperations variability.

CHALLENGESThere are four key challenges: reliably deliver new dataand product capability quickly, extend the Cockpit to newconsumers, continue vigilance on data quality andreliability, and drive to near real-time data currency insupport of supply chain management.

The team will continue the successful approach of lettingthe business drive priorities and deliver the critical fewevery 90 days. The tool will be expanded to supportadditional information domains and extend productimprovements. Future plans include advanced datafiltering, better graphical display, and a thin clientversion.

The Cockpit team will expand the base of consumers byteaching and helping others. A Linux∗ -like federateddevelopment approach will be implemented, and theproduct will be given to Intel’s IT DSS Group to offer toother customers with similar needs.

To continue the focus on data quality, we are going todrive data owners to standards, drive use of datawarehouse utility, clean up manufacturing data modelsand create data quality focus groups for continuousimprovement. Finally, to support the supply chain GlassPipeline∅ requirement, working groups will beestablished to identify and drive requirements for supplychain visibility.

CONCLUSIONThe ATM Cockpit is adding business value today byhelping ATM preclude and solve problems faster andmake decisions faster. It fills a key business managementcapability gap for VF operations and will help ATM meetthe competitive challenges of tomorrow. Along with theoutstanding contributions of the extended ATMisdevelopment team, the tool and methods employed are

∗ Other brands and names are the property of theirrespective owners.∅ Glass pipeline is an (Extranet) public communicationsystem with privileged and protected configuration thatallows value-chain constellation constituents and theirconsumers to determine location/movement of goods andservices. [8]

breakthroughs that give Intel an edge in this brave newworld.

ACKNOWLEDGMENTSWe thank David Marsing and Mike Splinter for giving usan opportunity to fail. We also thank OLGA/Flip ChipVirtual Factory for providing guidance and direction.

We acknowledge the Extended Cockpit DevelopmentTeam and the IT and Enterprise Application Groups fortheir technical contributions. And, we thank Natasha DuBois for the technical editing/writing, and research.

REFERENCES[1] Kimball, R., The Data Warehouse Toolkit: Practical

Techniques for Building Dimensional DataWarehouses, John Wiley & Sons. 1996, pp. 370-388.

[2] Kimball, R., “A Dimensional Modeling Manifesto:Drawing the Line Between Dimensional Modelingand ER Modeling Techniques,” DBMS and InternetSystems, August 1997.http://www.dbmsmag.com/9708d15.html

[3] Thomsen, E. et al., OLAP Solutions: BuildingMultidimensional Information Systems, John Wiley &Sons 1997, pp. 575-592.

[4] Kimball, R. et al., The Data Warehouse LifecycleToolkit: Expert Methods for Designing, Developingand Deploying Data Warehouses, John Wiley & Sons,1998, pp. 790-800.

[5] “Microsoft SQL Server 7.0 OLAP ServicesWhitepaper,” Microsoft Corporation 1998.http://www.microsoft.com/sql/productinfo/olapservices.htm

[6] Kimball, R., “Help for Dimensional Modeling Helpertables let you design and manage multivalueddimensions successfully,” DBMS Online, MillerFreeman, Inc. 1998. http://www.dbmsmag.com

[7] Becker, D., “How to Build Business Solutions withMicrosoft SQL Server OLAP Services and Excel2000,” Microsoft Corporation 1999.http://microsoft.com/Seminar/1033/19990826TQN1007DB1/Seminar.htm

[8] Poirer, C., “The Path to Supply Chain Leadership,”Supply Chain Management Review, July 1998.http://www.manufacturing.net/magazine/logistic/archives/scmr/07path.htm



AUTHORS’ BIOGRAPHIESPaul Calame is a 23 year Intel veteran. Paul has held avariety of management roles in Manufacturing,Automation and IT. He received a B.S. degree ingeology and political science from Iowa State Universityin 1976. His e-mail address is [email protected].

Ravi Nannapaneni received a B.S. degree in ComputerScience from Andhra University, Waltair, India and anM.S. degree from Arizona State University, Tempe,Arizona. He has been with Intel for nine years andcurrently manages the Cockpit & Emerging Tools andCapabilities (ETC) group within Intel's ATMorganization. Ravi's prior duties included IT Engineeringdesign support for new Intel international site startupactivities and introduction of Information Businesstechnologies to Intel's HVM facilities. His e-mail addressis [email protected].

James Yu holds an M.S. degree in MIS from theUniversity of Arizona. Before that, he also studied underthe M.B.A. program at the University of Texas and holdsa B.S. degree in Diplomacy from the National ChengChiUniversity in Taipei. James Yu is the Architect for theCockpit ETC team and has been in a softwarearchitecture role within Intel's IAL and the ITorganization. Prior to joining Intel, James spent 12 yearswith Honeywell’s Industrial Automation & ControlDivision in various software development, design, andarchitecture positions. His e-mail [email protected].

Jay Turpin received a B.S. degree in marketing fromEastern Michigan University, Ypsilanti, Michigan. Hehas been with Intel for three years and is currently incharge of "all things data" for Cockpit ETC group withinIntel's ATM organization. Jay's prior duties includeddesigning and developing the databases to support EATSyield analysis application and managing the corporatecomputer systems at Honeybaked Ham Company in Troy,Michigan. His e-mail address is [email protected].

Scott Peterson received B.S. and M.S. degrees inelectrical engineering and an M.B.A. from the Universityof Illinois at Urbana-Champaign. He has been with Intelfor four years and is currently a Senior Developer in theCockpit ETC group in Intel's ATM Organization. Prior tojoining this organization, he worked on communicationsand networking initiatives in the Intel Architecture Lab.His e-mail address is [email protected].

Copyright © Intel Corporation 2000. Legal notices athttp://www.intel.com/tradmarx.htm.

Intel’s Internet Connectivity: Evolution, Technical Architecture, and Future Directions 1

Intel’s Internet Connectivity: Evolution, TechnicalArchitecture, and Future Directions

Cindy Bickerstaff, Intel® Online Services, Intel Corporation Sally Hambridge, Intel Online Services, Intel CorporationLynne Marchi, Information Technology, Intel Corporation

Tod Oace, Intel Online Services, Intel CorporationStacy Purcell, Information Technology, Intel Corporation

Jeff Sedayao, Intel Online Services, Intel CorporationCharles Smothers, Information Technology, Intel Corporation

Ken True, Intel Online Services, Intel Corporation

Index words: the Internet, connectivity, firewalls, security, performance, measurement

ABSTRACT

This paper describes Intel’s Internet connectivityarchitecture, including the business drivers that led to itscreation and the technology developed to build andmaintain it. Intel’s first Internet connection was a 2400bit per second modem used to pick up and deliver e-mailfor a small community of engineers and researchers.With the advent of the Mosaic Web browser, Intel neededto develop a scalable, high speed, highly availableInternet connectivity architecture that is secure, available,provides good performance, and minimizes the impact ofInternet usage on Intel’s internal network.

An architecture was developed and implemented thatprovided multihomed Internet gateways at major Intelsites. A screened subnet firewall architecture providesdefense and depth. Several fail over modes wereimplemented to maintain connectivity even if a number ofgateway components failed. To manage this distributedgateway implementation, we borrowed methods fromIntel’s manufacturing world such as Copy Exactly! andcertain measurement and experimental methodologiesand statistical techniques. We modelled our set ofdistributed gateways as a single system and droveconfiguration through a Makefile.

Our architecture has proven highly successful. Itsupports Internet access for tens of thousands of Intelemployees’ Internet access, and Intel does $1 billion amonth in e-Commerce through this infrastructure.Current challenges include maintaining performance

while dealing with growth and security threats. Futureuses of the Internets are dial-in modem replacement,intercompany connectivity, virtual private networking,and streaming media.

INTRODUCTIONIntel initially connected to what would become theInternet in 1986 [1]. The first connection took placeusing a 2400 bit per second modem that was used to pickup and deliver e-mail for a small community of engineersand researchers within the company. Connectivityeventually evolved into a direct leased line into theInternet. With the advent of the Mosaic Web browser,Internet usage became mainstream. As we foresaw thatthe Internet would become as important a tool to businessas the telephone, Intel’s Information Technology groupfaced a number of challenges in providing Internetservices:

• provide Internet services with good performance

• scale service to deal with increasing numbers ofservers, services, and users

• make Internet services like Web access and e-mailhighly available and robust

• ensure that our Internet gateways are secure

• minimize any impact of Internet traffic across Intel’sinternal network



To meet these challenges, we implemented an Internetconnectivity architecture that provides Internet gatewaysat major Intel sites. Connecting to at least two InternetService Providers (ISPs) at each site increasedavailability and performance. We implemented ascreened subnet firewall architecture to provide defensivein-depth security, and we made our gateway capable ofseveral different fail over modes in order to make it ashighly available as possible.

The major challenge of a distributed Internet gatewaysystem is managing it. We had to ensure that gatewayswere operating correctly, the Internet service was good,and consistent security policies were being enforced.Moreover, the staff maintaining Intel’s Internet gatewayswas fairly small, less than ten people. To manage theInternet gateway system, Intel’s IT took a number ofconcepts from Intel’s manufacturing arm, such as CopyExactly! [2], a technique to simplify deployment andmaintenance of systems and configurations. We alsoused measurement and statistical methodologiesdeveloped and honed in wafer fabrication plants tomeasure and monitor Internet performance [3]. Inaddition, we simplified our maintenance by modelling ourdistributed gateway system as a single computer and therouter and server configurations as software modules tobe compiled and installed [4].

Our architecture has proven highly successful. Itsupports tens of thousands of Intel employees’ Internetaccess, and Intel does $1 Billion a month in e-Commercethrough this infrastructure. We have managed to scalealmost all of the components of the architecture, frombandwidth and ISPs to servers and users. Policies andtechniques developed while implementing thisarchitecture have gone into Internet Standards [5] andindustry white papers [6], and many of the techniquesdeveloped are being implemented in Intel’s new businessunit, Intel® Online Services.

The constant challenge that we have seen is keeping upwith increasing usage of the services we offer and thenumber of new services that users demand. Another keychallenge is dealing with the ever increasing and changingsecurity threats. In the future, we see the Internet usedincreasingly for dial-in modem replacement,intercompany links, and for virtual private networking.We foresee a time when every site will have its ownInternet connection, although we see challenges to thisvision in non-US locations where bandwidth costs areproblematic [7].

This paper describes how we designed and implementedIntel’s Internet connectivity architecture. It covers theearly drivers for connectivity, the requirements anddesign issues for a connectivity architecture, and the

technologies and policies needed to implement andmaintain the architecture that we built.

EARLY DRIVERS FOR INTERNETACCESSIntel first connected into what would become the Internetin 1986 [1] via an organization called CSNET. Theprimary driver for connectivity was the exchange of e-mail between Intel engineering organizations andconsortia outside of Intel, such as the SemiconductorResearch Corporation and Sematech. The Internetenvironment (then the ARPANET) was much differentduring that time period. It was designed primarily as aresearch environment, and there were explicit restrictionson commercial use. Direct connections through leasedlines were expensive and not easy to come by, so Intelexchanged mail through a system called PhoneNET.Intel dialed up a server, dropped off outgoing e-mail, andpicked up incoming e-mail. All this happened through a2400 bit per second modem.

The power and utility of being able to communicate withpeople in other organizations electronically made e-mailuse grow exponentially. This was despite the fact thatIntel didn't offer classes or documentation on how to usethe Internet and did not even advertise the fact that wewere connected!

By the late 1980's, large archives of freely distributed andhighly useful software began to appear. There wasincreasing demand for access to these archives, for whichdirect connectivity to the Internet was necessary. AsInternet mail use at Intel grew exponentially, the dial-upcharges began to approach the costs of a leased line. Atthat point, Intel obtained one 56 Kilobit leased line to theInternet for general use by employees. The Internet thenwas largely used by engineering personnel and systemsadministrators. Systems with Internet access werescattered haphazardly across the company and managedby a variety of groups.

The Mosaic browser and the World Wide Web broughtthe Internet into mainstream use at Intel. Before then, theInternet was largely text-based. It was mainly used for e-mail, FTP (file transfer), and remote login. The menuingsystem that preceded the Web, Gopher, was mostlytextual. The Web's use of the in-line graphics increasedInternet connectivity bandwidth requirements. As theInternet became commercialized and the Web began totake off, Intel needed to seriously revamp the way itconnected to the Internet. At that point, we knew Internetconnectivity and services were as critical as the phone.The state of Internet services at the time was haphazardand not secure. Servers used to provide Internet access



and provide Internet services like DNS were scatteredacross the internal network and inconsistently managed.

CONNECTIVITY ARCHITECTURE ANDIMPLEMENTATIONWe knew we had to move to an Internet connectivityarchitecture that supported Internet service as a robust,secure service used by Intel employees, customers, andsuppliers. This section talks about how we designed andimplemented an architecture that met that goal. First wediscuss key requirements and design tradeoffs. We thendescribe the connectivity and security design, followed bya section on how we maintain our large distributedsystem. Finally we discuss how we ensure that wereceive quality connectivity service, and how we craftedpolicies to guide Internet usage.

Key Requirements and Design TradeoffsWith the growth of the World Wide Web and theincreasing use of the Internet by all employees, wedecided to view the Internet as a key utility, as basic andnecessary a business tool as the telephone. Liketelephony service, access to web sites and electronic mailexchange with other companies would be used byeveryone, not only by a small community of engineersand scientists. With that as a model, several requirementsimmediately came to mind:

High Availability and Reliability. Internet services hadto be as available and reliable as the telephone.

Security and Accountability. The service could not leaveIntel vulnerable to intruders and could not be vulnerableto denial of service attacks. The Internet architecture alsohad to allow Intel to keep track of employee use so thatdata were available when deciding on growth. Also, Intelneeded to be able to track possible misuse by employees.

Good Performance. Our architecture needed to performwell so employees could be productive in their use of theInternet. Good performance is also related to security: asystem that is not performing well enough to be usablewill be worked around, usually in a not secure way.

Maintainability. The staff maintaining our Internetconnectivity was small and was not going to grow rapidly.Whatever architecture was implemented needed to beeasily maintained remotely by a small staff. Whensomething went wrong, we needed an architecture thatcould deal with it and continue to work.

Scalability. In the early years of Internet connectivity,we saw exponential growth in traffic to the Internet yearafter year. Whatever we designed had to be able to scalein almost every way we could imagine. We knew that thenumber of employees using this infrastructure and the

time they spent using this infrastructure would constantlyincrease. Our architecture needed to be able to growquickly to meet this demand—from increases inbandwidth to the number of servers providing service.

Impact on Intel's internal Network. Intel has a number ofcritical internal applications, such as chip design andsemiconductor manufacturing, as well as its internal e-mail system. Use of the Internet should not impact thesecritical uses of Intel’s extensive internal network.

Critical DecisionsA number of critical decisions had to be made at thispoint:

• Should the gateways be centralized or distributed.

• Should Intel connect to one ISP or several. If morethan one, how many.

• How should the network be laid out so that it ishighly available, secure, and yet scalable.

Centralized or distributed gateways. There are severaltradeoffs between having a single gateway withcentralized services and having several distributedgateways. Having a single gateway is much easier tomanage. It is also better from a security standpoint, asthere is only one gateway to watch and control. A singlegateway is also much cheaper to provision and maintain.

Having several gateways is better from an availabilitystandpoint, especially if you can fail over traffic from onegateway to another. Also, if traffic is directed to theclosest of several Internet gateways, the amount of trafficis minimized on the Intel internal network. It is alsoeasier to deal with traffic growth when there are severalInternet gateways. If one of the gateways becomescongested, only the users of that gateway are affected.Moreover, traffic can be moved to different gatewaysmaking it unnecessary to upgrade gateway resources.

For Intel, the availability requirements and the need tokeep Internet traffic off of the internal network drove usto have distributed Internet gateways. Since gateways arenot free, we decided to have Internet gateways only atmajor Intel sites—sites with thousands of employees.This minimizes traffic on our internal network whilekeeping the number of gateways manageable. Two keyconcerns with having multiple gateways aremanageability and security. These are discussed later.

ISP Connectivity—one vs. many. Another design choicewas ISP connectivity. The tradeoffs between havingmore than one ISP at all the gateways are betweenmanageability, performance, and availability. Having oneISP is clearly easier to manage, but having only one at allof our gateways leaves us vulnerable to a problem at that



ISP that propagates across the entire Internet and bringsall gateways down. We have seen this kind of problemoccur on a number of occasions. Having multiple ISPs isharder to manage from a troubleshooting and a vendorperspective, but it provides higher availability. MultipleISPs perform better because a packet to and from ourgateways could travel between Intel and other Internetsites more often without having to transit a peering point.Consequently, we decided on two ISPs. More wouldprovide better performance, but two are manageable andcost effective and still meet performance and availabilitygoals.

Network layout. The network layout has to be secure,scalable, highly available, and perform well. We neededa network that would still work and still be secure if asingle component failed. To achieve that goal, we madeeach of our Internet gateways capable of handling trafficfor another gateway. To implement defense in depth, weused the screened subnet firewall design [8]. To makesure that the design was scalable, we designed all of ourISPs and firewall complexes to land on a specificEthernet segment so that additions and changes wouldhave minimal impact.

Internet Connectivity ArchitectureLet's look at the network layout in more detail. At thehighest level of abstraction, our Internet connectivityarchitecture looks like that shown in Figure 1:

InternetIntelNetwork

Gateway 1

Gateway 2

Gateway n

Figure 1: Gateway connectivity between Intel and theInternet

There are a number of Internet gateways between Inteland the Internet, each located at a major Intel facility.Systems within Intel are configured to use the Internetgateway closest to them (typically on the same site) when

they want to utilize services such as Web access. Goingto the closest gateway minimizes the impact that Internettraffic has on Intel’s internal network. Also, if onegateway goes down, traffic can be automatically reroutedto another gateway. This feature increases theavailability of Internet service. If necessary, thearchitecture can be scaled by either adding gateways orincreasing the bandwidth between a gateway and theInternet.

At the next level of abstraction, each gateway looks likethat shown in Figure 2:

FirewallComplex 1

ISP 1 ISP 2

Service ProviderNetwork Segment

Internet

ISP N

FirewallComplex 2

FirewallComplex n

Intel internal network

Figure 2: Architecture of an individual gateway

Each gateway consists of one or more firewall complexesconnected to a service provider segment. Firewallcomplexes are groups of systems that have some commonpurpose, such as Intel’s Web site or systems that providebasic Internet services. Most gateways will have onecomplex just for basic services, while some gateways atIntel have as many as four firewall complexes.

Two or more ISPs connect to this segment. This networklayout has a number of advantages. With traffic flowingthrough the system, an ISP can be added, deleted, orupgraded. For ISPs that manage the router on theircustomer’s premises, the service provider makes anexcellent point of demarcation between the ISP and Intel.Also, each firewall complex can have its own individualrouting policies with the ISPs. This allows us to tuneperformance for a firewall complex by adjusting routing.

This design can be scaled in a number of ways. Thebandwidth can be increased by bringing in an ISP inparallel to the current connection and then cutting over.



Additional ISPs can be brought in if necessary or thenumber of firewall complexes can be increased.

This design contains a number of high availabilityfeatures. If any single ISP goes down, traffic can bererouted through another ISP. If the service providersegment goes down, then traffic can be routed throughIntel to another Internet gateway. Firewall complexes areindependent of each other. If one complex has aproblem, that problem is usually isolated to that complex.

Security Architecture of a Firewall ComplexAt the next level of abstraction is a firewall complex. Weused a "defense in depth" approach in designing thearchitecture of our firewall complexes. Defense in depthrelies heavily on the use of multiple components in thefirewall to reduce the risk of an intrusion. An attackermust compromise more than one firewall componentbefore gaining access to Intel’s network—there is nosingle point of failure. While this approach does notguarantee the security of our internal network, itdefinitely makes it harder and more time consuming foran attacker.

A firewall complex is designed using the screened subnetarchitecture [8] shown in Figure 3.

Service ProviderNetwork Segment

Internet

Intel internal network

Outer Router

Inner Router

DMZ 1 DMZ 2 DMZ n

host

host

host

host

host

host

Figure 3: Basic design of a firewall complex

The firewall components are the outer router, theDemilitarized Zones (DMZs) and the inner router. Theouter and inner routers are responsible for controllingboth incoming and outgoing traffic going to the variousDMZ network segments. The outer router allowscommunication between the Internet and the hosts in theDMZ while blocking direct communication with Intel’sinternal systems. In addition, the outer router blocks

specific well known attacks. The inner router heavilyfilters communication between Intel’s internal systemsand the DMZ servers. Access by DMZ systems to Intel’sinternal network is restricted because the DMZ systemsare exposed to attack from the Internet and cannot betrusted.

The method used by the routers to control traffic is calledpacket filtering. Each packet received by the router iscompared against a set of predefined rules. These rulesor access lists determine whether or not each packet isallowed to pass through the firewall and continue itsjourney towards the destination system. An access listconsists of a source IP address a destination IP address,and a port number that denotes the service beingrequested.

The systems that sit in the DMZ are called bastion hosts.Each bastion host performs a specific set of functions.For example, one server provides proxy services, whileanother server acts as an SMTP mail relay and DNSserver. This separation of services limits the damage thatcan be caused by a break-in, since the SMTP/DNS serverdoes not run the proxy services and vice versa. Inaddition, the underlying operating system on all of thebastion hosts has been made more secure: unnecessaryservices have either been disabled or removed. Specialprograms have been installed that limit access or provideadditional security. For example, the "sudo" programdefines which administrators can execute privilegedcommands, and the Secure Shell program providesadministrators with an encrypted tunnel so that anintruder cannot pick up passwords. We also run aprogram daily that compares the files on each of thebastion hosts with a set of master files. Differences arenoted and sent to our system administrators for furtherinvestigation.

This design can be scaled in a number of ways. To dealwith increased demand, more servers can be added to aDMZ segment. If more specialized services are needed,such as streaming media or authentication services, DMZsegments can be added.

Services ArchitectureInitially, the only service provided by Intel’s Internetconnection was electronic mail for Intel employees.Today, Intel’s Internet connections provide services fornot only Intel employees, but also for Intel customers andbusiness associates. These services allow employeeaccess to the World Wide Web, FTP, Usenet news, andstreaming audio and video. Intel customers can accessIntel’s Web site and can download product specificationsand software. Business associates can place orders forproducts on our e-Commerce servers. Understandably,



the complexity of the infrastructure to support theseadditional services has grown.

A single mail relay server was Intel’s first generalpurpose presence on the Internet. Later, to provideadditional services such as FTP and telnet to remotehosts, several more computers were granted access to theInternet connection. Users who logged into these serverscould freely access all the Internet had to offer. But evenbefore the explosive growth of the Internet brought aboutby the World Wide Web, this service model did not scalewell. Providing an account for Internet access to everyemployee is clearly not scalable.

To deal with this problem and to avoid incidents where auser might mistakenly or maliciously subvert the securityof an Internet access server, the user accounts wereremoved and replaced with several special softwareagents called proxies. Each proxy provides access to aspecific type of Internet service. Intel employees canaccess the Internet by employing these proxies. Onlyspecialized servers within DMZs can talk directly to theInternet. This improves security and it also minimizestraffic on Intel's internal network. DMZ servers such asWeb servers and DNS servers that provide services topeople on the Internet talk directly to the Internet withouthitting Intel's internal networks. For certain services suchas e-mail and DNS, we spread servers over multiple sitesand gateways. This spreads out the load and makes theservices available in case entire Internet gateways godown.

Maintaining Multiple Distributed InternetGatewaysDistributed Internet gateways make their management akey challenge. Having Internet gateways locatedthroughout the world, each with its own complexconfigurations, posed a significant maintenance challengeto the small staff responsible for managing Intel’sInternet gateways. Configuring each system by hand oneat a time would be time consuming and prone to error.Because break-ins are often carried out by crackers whoexploit misconfigured devices, system and networkadministrators need to make sure that configurations areconsistent and correct. Responding to changes in theenvironment would be increasingly difficult if we had tomanage each system individually. We needed to create asystem to help us manage all our firewall devices.Therefore, we designed, built, and currently use a systemand methodologies that manage all of our router accesscontrol lists and all of our bastion host configurations.Our techniques, discussed in the following sections,greatly reduce the likelihood of a misconfiguration.

Copy Exactly!The first key method we use was taken from Intel'smanufacturing sector. "Copy Exactly!" (CE!) [2] meansthat each gateway is copied exactly from the previous oneas much as absolutely possible. We specifically wantedto avoid engineering a solution for each gateway. Inmanufacturing, new factories ramp faster and moreproductively if a standard design is copied exactly ratherthan reengineered. CE! has proven to be the only way toensure that our configurations are implementedconsistently across systems. With regard to the hardware,each firewall is composed of identical sets of routers.Each bastion host is based on a small set of approvedplatforms. Our maintenance system ensures that thesoftware configurations are all CE!

CE! posed an interesting challenge for writing softwareand configurations that are distributed into ourarchitecture. The code on each of the Internet gatewayshas to be identical; yet, that code would have to operateon systems particular to each gateway. To deal with thisproblem, we developed a program called which_dmz thattells the code what gateway it is in and can return dataspecific to that gateway. If a code needs specific gatewayinformation it can call which_dmz. This way, we candistribute identical code and scripts to all gateways andhave the code use gateway-specific information.

Modelling the Distributed Gateways as A SingleSystemWe model all of our Internet gateways as a single largesystem. In our model, all the configurations and softwareof the gateway components are considered source code.This code is compiled into an "executable" form that isthen loaded into the system and run. Certainconfiguration files have dependencies: if particularconfiguration files change, then other files must changealso. Just as programmers use Make [9] to managecompilation and installation of source code, we use Maketo drive and manage the configuration of our connectivityarchitecture [4].

A single master set of host and network configurations iskept in a central repository. From this set, theconfiguration files for all the devices, whether hosts orrouters, can be derived. Once a new configuration hasbeen created, it can be “beta tested” by pushing out thechanges to a single device. Once the configuration istested, the remaining servers can be updated. As anadded benefit, we can compare the configuration of eachdevice in the firewall against the central copy to see if ithas been tampered with (which may indicate a firewalldevice has been broken into). The central computermakes use of encryption and authentication to gain accessto each device during maintenance. This prevents a



hacker from gaining access by impersonating the centralcomputer or by eavesdropping on passwords that may beused in an update process.

The architecture of the Make-driven update process isshown in Figure 4.

Make

Rdist Scriptinglanguages(e.g. PERL,Expect)

GenericShellcommands

ssh

configurations

To hosts andnetwork equipmentover the network

Figure 4: Architecture of the Make-driven updateprocess

Make controls all of the operations of configurationmaintenance. It can activate rdist, a file distributionprogram, run programs written in various scriptinglanguages, or execute any generic shell command to buildand distribute the proper configurations and software,which are sent out over the network to networkequipment and server hosts. rdist runs over SSH, thesecure shell protocol. We discuss how Make is used toconfigure network equipment and systems in the next twosections.

Maintaining Router ConfigurationsWhen Intel’s Internet connectivity was just a single ISPconnection through a single set of routers, routermaintenance was simple. Once we began to havemultiple gateways and once we also began to expect thetraffic to fail over from one Internet gateway to another,we realized we would have to synchronize the routeraccess lists that control what kind of packets can passthrough our gateways. A change in the permissions atone gateway needed to propagate to other gateways.Since we stored access list entries in files, we needed tohave a change in one file trigger changes in otherconfiguration files. Since Make is excellent at managingthe relationships and dependencies between files, itseemed to be a natural choice to manage the collection ofrouter access lists.

Any gateway has to fail over to any other gateway. Inorder to do this, we use variables for the access controllist (ACL) numbers that identify what access list anindividual access list entry is part of. Macro definitionsare used to define these variables to substitute the correctACL number for the appropriate router. In this fashion,an access list entry on one router could appear in adifferent access list on another in such a way that trafficthat regularly went out of one router could pass throughanother. Installation of access lists is driven through arouter access list Makefile. Make is used to call Expectscripts that automatically load new versions of accesslists.

Managing System ConfigurationsMake is used to manage the system configurations of allthe servers, wherever they may be in the world. SinceMake is so versatile, we use it to drive the remotedistribution program rdist. In turn, rdist has beenconfigured to run over Secure Shell to preventeavesdropping by potential crackers and to preventattacks where intruders masquerade as friendly systems inorder to penetrate defenses.

Our key system configurations are organized intoseparate source trees, each with its own Makefile, theconfiguration for make, and its own Distfile, theconfiguration for rdist. There are separate source trees,containing all the executables and configurationsnecessary for a given server subsystem like the operatingsystem, our IMCS software, sendmail, and othersignificant collections of software.

Once new software is installed into a tree, installing it onall of the servers is as easy as typing "make install."Typing "make check" shows what's waiting to beinstalled. Every morning we receive a report from ascript that does a "make check" in each tree. This reportis an extremely useful tool for keeping track of changesthat are being pilot tested on one machine and will needto be installed on the rest of the machines. "make check"reports on what files have changed and what needs to bedone in order to synchronize a given system with thedefinitive build.

By using Make we can easily insert hooks to massage orgenerate certain data files before they are distributed tothe servers. For example, the sendmail tree's Makefilecan automatically generate a sendmail.cw file thatcontains a list of all of the DNS domain names from theDNS tree.

By using rdist, we can easily customize which files getinstalled where and on which servers. We use rdist toautomatically run commands when certain files areinstalled or updated. For example, when the mail alias



file on a server is updated, a program called "newaliases"needs to be run to rebuild the file into an indexed fileformat. rdist can automatically run the "newaliases"command once an alias file is updated.

Our use of Make and rdist for server managementclosely parallels the "Copy Exactly!" methodologydiscussed earlier.

MEASURING THE QUALITY OFINTERNET ACCESSWith such a significant investment in Internetconnectivity, we needed to be able to measure its quality.We therefore developed the Internet Measurement andControl System (IMCS) [3]. This system measures keyindices of Internet performance such as packet loss,packet delay, Web page download rate and Web pageerror retrieval rates, and raw traffic volumes. When thesekey indices exceed predefined limits, the NetworkOperation Center (NOC) personnel are alerted and startworking through a script to debug the potential problem.We modelled this system on the statistical process controlsystems used in Intel manufacturing plants, where whencertain metrics are out of control, action is taken.

Early in the design of our measurement systems for theIntel firewalls, we decided that the most usefularchitecture was a distributed one that allowedmeasurement systems (such as IMCS and public domainsystems such as MRTG [10]) to collect their data on asystem in the DMZ and publish the results as HTMLpages on Web servers on the measurement system. Byusing Perl-based CGI scripts on the measurementsystems, it was possible to drill down to individualmeasurement values using GET-mode URLs. The factthat the measurements were available on the Web allowedus to make some simple Perl automation scripts thatwould routinely fetch current measurement thumbnailgraphs from all our measurement systems and displaythem on a browser anywhere inside of Intel. The side-by-side comparison of key metrics across firewall complexesallowed quick isolation of common faults to individualfirewalls, or conversely, allowed us to see Internet-wideevents in near real-time.

The common tools (ping, traceroute, whois, etc.) andcontrolled access to the tools’ current state of interfaceson firewall routers are made available by a combinationof JavaScript-enabled HTML pages and CGI scriptswritten in Perl and Expect. This allows us to allow NOCpersonnel access to tools without giving them directaccess to network equipment, thus avoiding potentialsecurity exposures.

INTERNET USAGE POLICIESWe felt that we could not offer Internet access unlesssome guidelines covering behavior on the Internet were inplace. We examined many Internet guides and wrote up aset of guidelines, which became the Intel Internetguidelines from 1993 to 1996 [11].

In 1996, Intel decided they needed one comprehensiveguideline to cover employees’ use of electronic mail,computers in general, and the Internet. Several of theInternet connectivity staff, as well as representatives frommarketing and legal drafted the current set of guidelinesin use today.

RESULTSOur experiences with the connectivity architecture hasgenerally been positive. In this section, we discuss howwell the architecture has scaled, describe some spinoffsof the technology and the policies employed to maintainthe architecture, and then outline some of the keychallenges we have faced.

Scaling to Meet DemandOur gateway architecture is quite scalable. From aservice standpoint, it provides Internet services for morethan 60,000 employees. The architecture is robustenough to support $1 billion per month in electroniccommerce. We have scaled the components of thearchitecture in almost every possible direction. To dealwith increasing use of services in an Internet gateway, wehave increased the number of servers. To deal withdifferent services in a firewall complex, we haveincreased the number of DMZs and created new firewallcomplexes. To deal with additional performancerequirements, we added additional ISPs and additionalbandwidth. To deal with acquisitions and new users, weeven added additional Internet gateways.

Despite this growth just mentioned, our architecture ismaintained by a small staff. At the present moment, astaff of five maintains the configurations of six Internetgateways and ten firewall complexes, which includesmore than 30 routers and about 30 servers around theworld. We are able to maintain thousands of lines ofrouter access list entries. Copy Exactly!, modellingInternet connectivity at Intel as a single system, andmaintaining configurations using Make has made thispossible. The availability of the design makes thearchitecture functional despite the failures of individualcomponents.

Connectivity Architecture, Tools, and PolicyThe architecture, tools, and policies developed for Intel’sInternet connectivity have found other uses. The



connectivity architecture for Intel’s use of the Internethas been adapted into the first two data centers of Intel’snew Web hosting business, Intel® Online Services (IOS).Many of the tools originating from within Intel’s IT havealso been implemented within IOS, including the Makeupdate methodology and IMCS. Our experience withmeasuring Internet performance lead us to contribute tothe Cross Industry Working Team (XIWT) white paperon measuring Internet performance [6]. XIWT is aconsortium of a diverse group of industries working toimprove information infrastructure.

The Internet usage policy that we described in [11] is oneof the more interesting spinoffs of our work. The paperbecame widely distributed in a number of collections ofInternet usage and security policies. At about the sametime that the paper was published, the User Services Areaof the Internet Engineering Task Force began looking forpeople and material to create a set of Internet usageguidelines to publish as an informational RFC. The workwe did on Internet usage policy became part of RFC 1855[5], Netiquette Guidelines, published in 1995.

KEY CHALLENGESWe experienced a number of key challenges as weimplemented our Internet connectivity architecture. Ourmake-driven maintenance process, while powerful,requires tremendous discipline to use. While softwareand configuration changes can be rolled out uniformlyacross all of the Internet gateways, it is just as easy to rollout a bad configuration change as a good one. It alsotakes discipline to maintain Copy Exactly! and distributechanges from the central repository. Occasionallyconfiguration changes are made on individual servers androuters and then lost when other changes are distributedfrom the central configuration repository.

While our architecture is designed to withstand the failureof a single physical component, the failure of a softwarecomponent is much harder to deal with. Because of CE!,we deploy the same version of software and configurationfiles to each type of network equipment and server. Ifthere is a failure in that software, then all the equipmentwith that version fails. We have seen this happen mostoften when there are scaling issues—where software orconfigurations cannot handle the demands made on them.The way to avoid this problem is to test new software orconfigurations under significant load. Unfortunately, notall load conditions can be simulated in tests.

Security continues to be a challenge. New threats andvulnerabilities crop up regularly and must be dealt with.Among the more difficult challenges are applications thatusers want to utilize across the Internet that are not welldesigned for firewalled environments. We have had to

refuse a number of requests for services that cannot beeasily proxied or do significant engineering to make thoseservices secure.

Scaling to meet demand for Internet services continues tobe a challenge. As the Internet becomes used more andmore, servers and network equipment can become heavilytaxed. Dealing with large numbers of firewall rulerequests is also very taxing. Another demanding aspect isthat changes or major projects often happen on shortnotice (also known as Internet time).

Our IMCS system relies heavily on active measurements,i.e., measurements that generate nonvalue-added trafficon the Internet and our infrastructure in order to obtainperformance data. That nonvalue-added traffic causes anumber of problems. We have received some complaintsfrom the targets of our measurements about themeasurements that we do. In one case, our measurementsadded significantly to a target site's bandwidth costs,prompting them to complain vigorously. We would liketo measure everything we can. However, activemeasurements add traffic and thus costs to ourinfrastructure and to the infrastructure of the targets thatwe measure.

International Internet gateways are another challenge.Bandwidth costs are often much greater internationallythan in the US, so it is not always clear that installation ofan Internet gateway will reduce costs and provide betterend-user performance. As noted by Cukier [7], theperformance going from a country to the United Statescan often be better and cheaper than performance withina region such as Europe or Asia. Since more bandwidthis being put between the US and Asia and the US andEurope than is being installed locally within thoseregions, this trend is likely to continue, making it difficultto justify the cost of international Internet gateways.

FUTURE PLANS AND EXPECTED TRENDSIn general, we see ever increasing use of the Internet formore and more functions. Two particular trends we seeare the use of Virtual Private Networking and streamingmedia.

Intel currently utilizes high-cost, high-maintenance legacytechnology to provide connectivity between our sites, toemployees when they are away from the office, and toour business associates. A new connectivity model,called Virtual Private Networking (VPN), can be utilizedto lower costs and to reduce the time required to establisha connection to a new site or business associate.

With VPN, we are able to utilize the Internet bandwidthto enable employees who are out of the office to connectback in to Intel and have access to our internal resources



using their own ISPs. ISPs typically offer much widerand cheaper coverage for dial-up connectivity than Intel.Using the triple DES encryption technology provided byprotocols such as IPSEC, we can ensure that Intel’sintellectual property is safe as it traverses the Internet.

The majority of Intel’s Wide Area Network is composedof costly privately leased lines. When we need to connectto a new site, we are often delayed due to the timerequired to order and install these circuits. Using VPNtechnologies, we will also be able to leverage the Internetconnectivity that is already installed at our major sites tocreate an encrypted, secure link between them throughthe Internet. We will be able to reduce the setup time oftwo to three months to one day or in some cases theconnection can be established within hours of receivingthe request.

Connecting business associates presents a slightlydifferent set of requirements because we only want toshare a subset of information. By building anenvironment that limits accessibility to specific resources,we can utilize VPN technology to reduce the cost andtime required to connect and implement the connection ina similar fashion to WAN replacement technologymentioned above.

We also see increasing use of streaming media such asvideo and audio. These type of media will substantiallyincrease bandwidth requirements. Because these mediaare jitter sensitive, service-level agreements between Inteland its ISP become more and more critical.

We see passive measurements becoming much moreimportant in the future. Mining data sources such as Webserver logs and router flow statistics can yield valuableperformance data without adding probe traffic. As thenumber of gateways increases, active measurementsbecome more and more of a burden on the measuredsites.

CONCLUSIONAs the Internet evolved from a network linking acommunity of researchers and engineers to themainstream medium that it is today, Intel’s Internetconnectivity evolved from a modem used for e-mail to thehighly available, multiple service distributed system thatit is today. Our design and implementation of an Internetconnectivity architecture has enabled Intel to handle $1Billion in e-Commerce per month and has spun offtechnology, methods, and policies used in other Intelbusinesses and across the Internet. Anticipating everincreasing use of the Internet, our architecture is ready todeal with new challenges.

ACKNOWLEDGMENTSWe acknowledge Suzanne Johnson, whose vision helpedland our first Internet dial-up connection those manyyears ago. We also acknowledge and thank the manypeople who have helped so much with deploying andmaintaining the architecture, including the CPS team,ITSP, the OSC, the WAN team, and countless folks insite IT organizations.

REFERENCES[1] Johnson, Suzanne M., "Internet Affects the

Corporation: Experiences from Eight Years ofConnectivity,” INET 95, Honolulu, HI, June 1995.

[2] McDonald, Chris J., "The Evolution of Intel's CopyEXACTLY! Technology Transfer Method," IntelTechnology Journal,http://developer.intel.com/technology/itj/q41998/articlOace, Tod, Sedayao, Jeff, Wong, Clinton, ”Don't Justes/art_2.htm.

[3] Bickerstaff, Cindy, True, Ken, Smothers, Charles,Talk About The Weather - Manage it! A System forMeasuring, Monitoring, and Managing InternetPerformance and Connectivity,” 1st UsenixConference on Network Administration, Santa Clara,CA, April 1999.

[4] Hambridge, Sally L., Oace, Tod, Sedayao, Jeff, andSmothers, Charles,”Just Type Make! ManagingFirewall Using Make and Other Publicly AvailableUtilities,” 1st Usenix Conference on NetworkAdministration, Santa Clara, CA, April 1999.

[5] Hambridge, Sally. “Netiquette Guidelines,” RFC1855, October, 1995.

[6] Cross Industries Working Team White Paper,"Customer View of Internet Service Performance:Method, Methodology, and Metrics," "CustomerView of Internet Service Performance: MeasurementMethodology and Metrics," September 1998.

[7] Cukier, Kenneth Neil, "Bandwidth Colonialism? TheImplications of Internet Infrastructure onInternational E-Commerce," INET 99, San Jose, CA,June 1999.

[8] Chapman, Brent D. and Zwicky, Elizabeth D.,Building Internet Firewalls, O’Reilly & Associates,Inc., Sebastopol, CA, pp. 58, 66.

[9] Oram, Andrew and Talbot, Steve, Managing Projectswith Make, O'Reilly and Associates, Inc., Sebastopol,CA 1991.

[10] Oetiker, Tobias. http://www.mrtg.org/.

http://developer.intel.com/technology/itj/q41998/articl

http://www.mrtg.org/.



[11] Hambridge, Sally and Sedayao, Jeff, "Horses andBarn Doors: Evolution of Corporate Guidelines forInternet Usage," Proceedings of the Seventh SystemsAdministration Conference, LISA '93, Monterey, CA,November 1993.

AUTHORS’ BIOGRAPHIESCindy Bickerstaff is a 20-year Intel veteran with the first15 in silicon process development especially in metrologyand process control. She received "The Five Year (or so)SPC Marathon Award" in 1988 for driving SPCimplementation in California Technology Development.She received an Intel Achievement Award in 1989 for"formulating, developing, and implementing the firststatistically based process control strategy for lithographyat Intel". After her 10th silicon technology developmentcycle she decided to try something new and moved intothe Internet group in 1995. Since then, Cindy has beenmapping many of the measurement and control methodsshe worked on in silicon processing into the Internetknowledge domain. Her email address [email protected] .

Sally Hambridge began working on the Internet in 1980when she joined the Information Sciences Institute inMarina del Rey, the home of the IANA and the RFC-Editor. She joined Intel in 1984 after a brief sojourn atAtari. She saw the advent of Internet connectivity atIntel, and has worked in the Internet group since 1992.Since 1996 she has been responsible for router accesscontrol lists and for routing in the Internet connections.She joined Intel Online® Services in August 1999. Heremail address is [email protected] .

Lynne Marchi received her B.S. degree in computerscience from California State University, Sacramento.She began work at Intel in 1992 as a co-op and thenjoined the Corporate Information Security group in 1993.In 1996, she joined Internet Connectivity Engineeringwhere she has focused on the secure implementation ofnew firewalls. Her email address [email protected]

Tod Oace is a UNIX and networking systems engineer.Over the past 20 years he has worked with a wide rangeof computer platforms from micro to mainframe, and isproficient in several programming languages andnetworking protocols. He pioneered the use of BSDUNIX for several enterprise applications since joiningIntel in 1991. He received division recognition awards forthe redesign and support of an in-house created IntranetSMTP/cc:Mail gateway, and for the co-creation of anIntranet LED readerboard display control system. He alsodesigned the first large-scale network for thewww.intel.com server complex. Tod is now working in

the Firewall Engineering group of Intel Online Services.His email address is [email protected] .

Stacy Purcell graduated from the Georgia Institute ofTechnology with a B.S. degree in computer science. Hejoined IT in 1994 where he has worked as a Front LineManager, a Network Engineer specializing in the design,maintenance, and troubleshooting of LANs for theFolsom site and in new acquisitions. He currently worksin the ICE team focusing on Intel's six Internet firewallsand plans to expand that number to 24 by the end of theyear. His emaild address is [email protected] .

Jeff Sedayao is a network engineer in Intel OnlineServices. Between 1987 and 1999, he architected and ranIntel's Internet connectivity. His primary interests are inInternet performance, security, and policyimplementation. He also serves as Intel's representativeto the Cross Industries Working Team's InternetPerformance Team and has participated in the IETF's IPPerformance Metrics Workgroup. His email address [email protected] .

Charles Smothers is a Senior Systems Programmer forInformation Technology. He joined Intel in 1982, whenhe wrote 8085 assembly code for testing the 27128EPROM. With the Memory Components Division, LowYield Analysis team he automated several of the tasksused in the visual inspection and defect categorizationprocess. In 1990, he joined the MCD Design Engineeringgroup as a UNIX systems administrator. Today, hespecializes in Internet Proxy servers. His email address [email protected] .

Ken True is Manager of Internet Firewall Engineeringfor Intel Online Services, Inc. He joined Intel in 1981and has provided technical management and individualcontributions in the areas of Mainframe SystemsProgramming, WAN Network Engineering, PC TechnicalServices, LAN Software Distribution (OfficeLAN),Intranet/Internet Engineering, and Internet ConnectivityEngineering. He and his organization are responsible forthe architecture, engineering, and security of the IOSfirewall systems worldwide. His email address [email protected] .


http://www.intel.com/tradmarx.htm.

Managing Enhanced Network Services: A Pragmatic View of Policy-Based Management 1

Managing Enhanced Network Services: A Pragmatic Viewof Policy-Based Management

John Vicente, Information Technology, Intel CorporationHarold Cartmill, Information Technology, Intel CorporationGlen Maxson, Information Technology, Intel CorporationShelby Siegel, Information Technology, Intel Corporation

Russ Fenger, Intel Architecture Labs, Intel Corporation

Index words: policy, policy-based management, PBM, QoS

ABSTRACTThe convergence of public and private networks and therise of the Internet as a global medium for informationexchange and economics are prompting corporations toaugment existing business computing models. Newinformation technology initiatives for collaboration andbusiness exchange are essential to gain competitiveadvantage. These initiatives are being realized throughemerging applications (e.g., e-Commerce, groupwareapplications, multimedia) over converging private andpublic boundaries. Enterprise strategies are requiringtimely evolution of network infrastructure andmanagement to support delivery and management of end-user and network services. Within this context, quality ofservice, security, productivity, and infrastructureefficiency are critical to achieving bottom-line businessresults. In this paper, we take a closer look at policy-based management as an enabling technology andparadigm shift for Intel's Information Technologyorganization. We explore the dimensions of policy-basedmanagement, and we provide a pragmatic review of thetechnology, discussing the deployment challenges,roadmap considerations, and practical usage scenarios.

INTRODUCTIONCurrent trends in corporate and Internet networks areshifting from best-effort, vertical network architecturetowards a more intelligent, end-to-end, service-awarenetwork paradigm. As evidenced over recent years, theneed for enhanced network services such as virtualprivate networks (VPN), quality of service (QoS),security, collaboration, and directory technologies

demonstrates that customers are demanding more fromthe core infrastructure for enabling productivity,flexibility, service differentiation, isolation, privacy, andmanageability. Moreover, critical network resourcesmust be aligned with business objectives where networksare i) more content or application-aware; ii) providedynamic features for service creation; iii) observe andenforce network-wide policies; and finally, iv) enablecontrol from the network provider to the administrator tothe end-user. The migration to a richer networkinfrastructure allows corporations to be more agile andoptimize infrastructure costs, while meeting the diverserequirements of emerging application demands. Thefollowing requirements are driving innovations in currentnetwork infrastructure technology.

Mission CriticalityIT organizations recognize that the availability andreliability of network infrastructure are essential tocritical applications and services that rely on them. Thesesame applications can compete for network resources(e.g., bandwidth) with other less critical and diverseapplications for successful transport delivery andperformance. Core network capabilities are required toensure delivery priority, security, access control, and fairperformance allocation. However, certain users (e.g.,company presidents) or groups in an organization mayhave a more critical need for access to resources, andthus, that person or group may get priority or have adifferent authorization from the rest of the organization.Mission criticality is raised as organizations movetowards extranets or public services for corporatebusiness computing or service transport.



Network Architectural AgilityCustomer demands and applications are growing rapidlywhile networks continue to be gridlocked by standardsand proprietary implementations. In general, enterprisenetworks have been implemented with multiple vendordevice solutions, while heterogeneity of bandwidthresources and legacy device inequalities are evident overgeographically dispersed regions. While suchinfrastructure complexity exists and diverse application-network demands continue, network architectures andmanagement frameworks must still support more rapidevolution, enabling faster service creation anddeployment while maintaining legacy integration.

Service FlexibilityThe application and network infrastructure need to beintegrated to meet the ever-changing needs of currentcomputing and distributed application demands.Providing dynamic network reconfiguration, timelyaccess control, or dynamic, class-based bandwidthallocations can provide greater flexibility to the end-useror network administrator. An increase in networkfeatures allows administrators to manage service levelsmore effectively, while allowing application developersmore control features for end-user services.

EfficiencyThe increased complexity of managing network systemsalong with the need for service-level managementrequires current network management solutions to bemore sophisticated. While corporations continue to grow,optimal planning and management of the distributedinfrastructure are essential. Better tools for Just-in-Time(JIT) bandwidth and service provisioning, effective use ofresources, and automation of management tasks canreduce the total cost of ownership while improvingservice-level management.

In the first section of this paper, we present thebackground on policy-based management including ourview of the requirements for this technology. We followthis with a discussion on the evolving IT business andoperational models. Next, we discuss the technologyimplications and challenges that IT organizations mustaddress to realize policy-based management. Followingthis, we present our current progress within Intel IT,providing a transitional discussion under the e-Businessmodel. We close with a summary and a brief look at howwe can move forward with this technology.

BACKGROUND ON POLICY-BASEDMANAGEMENT

As defined in [1], policy-based management is "thecombination of rules and services where rules define thecriteria for resource access and usage." Alternatively in[2], the authors define it as a "unified regulation of accessto network resources and services based on administrativecriteria." We view policy-based management as a viabletechnology to provide greater control and management ofunderlying networks via the creation and distribution ofhigh-level policies (business rules), integrated with theenabling mechanisms of the network infrastructure. Byway of automated and rapid configuration and theintegration of business policies with the networkinfrastructure, new opportunities for managing bothinfrastructure and network services are introduced. Tosupport these new initiatives, we define the followinggeneral requirements for policy-based managementtechnology:

Service differentiation. This is the ability to control ormanage the quality of the service or service deliverymechanisms in order to meet some predefined network-based performance delivery/metrics. This may extend toenabling Service Level Agreement (SLA) managementfor service-level validation.

Network provisioning and bandwidth management. Theseprovide proactive bandwidth management by facilitatingcontrol and allocation of bandwidth through deviceconfiguration management: that is, facilitating manual,multi-device network configuration and performingadmission control or traffic segmentation.

Integration with network management systems and legacydevices. Policy-based management must be integratedwith current paradigms for managing IT organizationalstructures. These include existing operational models(e.g., centralized control or change management),security requirements, and business computing models.The requirement to support or address legacy systemsand device limitations is also mandatory.

Scalability. The PBM technology infrastructure shouldarchitecturally scale to Intel's enterprise environment andprivate/public models for e-Business computing. Theseinclude the policy server environment to businesscomputing hierarchy, the directory and databaseinfrastructure, and scalable policy overhead in terms ofadministration and protocol communications.

Industry standardization. This means that policy-basedmanagement must conform to industry standards and usebest practices to support network device and policymanagement interoperability. There must be standardsfor such things as policy terminology [1] and protocols(e.g., COPS [3] / LDAP[4]). Moreover, an openframework for policy management schema [5] anddirectory integration must exist.



• Security. The policy-based management tools shouldfacilitate resource access control and authorization,and they should provide integration support forauthentication and accounting.

Policy-Based Management Framework

Functional OverviewThe components of a policy-based management system(PBM) include the policy console, policy server, policydatabase, and policy clients, all of which are shown inFigure 1 below. The policy console providesadministrative and operational access to the PBM system.

The Policy Decision Point (PDP), or policy server,embodies the decision-making functionality of policy-based management. One or more such policy serversexist in a control domain, with each server configured tosupport policy management for some defined group ofpolicy clients or Policy Enforcement Points (PEP) in thedomain. The policy server provides each policy clientwith policy information; the policy client in turn carriesout (enforces) the policies to the best of its abilities. Thepolicy server’s inner structure is shown in Figure 1 below.

Policy Server StructureThe policy client communication component handles allexchanges between policy clients and the policy server.The PBM client-server communications protocol uses the(proposed) IETF standard COPS protocol.

The message processing component is responsible forpolicy protocol message decomposition and composition,and for interpretation of wire-format objects for use inthe policy server.

The core-processing component embodies the logicneeded to support the policy server’s policy decisionmaking—rule processing and housekeeping. Thiscomponent also logs all relevant usage by policy clients insupport of accounting and billing applications. Theselogs may also be used for further tuning of resourcecontrol policies.

The policy console communication component givessupport for communicating with the policy console. Thisallows multiple PDPs to be maintained from one policyconsole. It also allows multiple consoles to access thesame PDP.

Policy Server(Policy Decision Point)

POLICY CONSOLECOMMUNICATION

CORE PROCESSING

POLICY CLIENT COMMUNICATION

POLICYDATABASEPolicy Console

Policy Clients(Policy Enforcement

Points)

Server(s)

Gateway(s)

Router(s)

Switch(es)

NIC(s)

Desktop(s)

ACCTDATABASE

LOGGING

MESSAGE PROCESSING

COPS Protocol

Figure 1: Policy server structure

Implementation AlternativesThere are alternatives to using a policy-based networkmanagement server as described above. For example,one can manually configure policy clients to performpolicy actions on packet flows, such as Type of Services(TOS) bit settings in the packet header that tag packetsfor priority Quality of Service (QoS) or client-initiated

RSVP (ReSerVation resource Protocol) [6] sessions.However, one of the key functions of the PBM server isthe centralized management of network resources andthe allocation of resources to clients based onorganizational rules (the organization owning andmanaging the network resources.) This ensures thatonly authorized clients have appropriate access to and



use of network resources. This function is extremelydifficult to manage solely at the client.

There are other approaches that use directories anddirectory protocols, such as Lightweight DirectoryAccess Protocol (LDAP) to set up and administerpolicy. These approaches work well when configuringnetwork devices and setting static rules. However,when dynamic control of network resources is required,directories are ill-equipped to deal with the complexdecision-making process, based on current resources inuse, priorities of requests, and administration of usage.Also, the LDAP protocol does not currently provide arobust level of information or a dynamic framework tomanage network resources in real-time.

Intel Architecture Labs (IAL) is helping drive thenetworking industry to fully use IETF’s Common OpenPolicy Services by making available the COPS toolkit,which enables clients to talk to policy servers. IAL hasdeveloped and is deploying a COPS Local PolicyModule that enables Windows 2000∗∗∗∗ clients to talkdirectly to policy servers.

EVOLVING IT BUSINESS MODELSCurrent trends in business computing are blurring theboundaries between the Internet and Intranet, whileapplication demands are becoming more content richand diverse in quality of service requirements. Networkvendors are building infrastructure components (e.g.,VPN) with more enabling network features (e.g., QoS)and management systems to support these new users orapplications. Nevertheless, it is unclear how ITorganizations should structure or restructure theirbusiness processes to map business units, applicationtypes, and transactional priorities to criticalinfrastructure resources. If this new paradigm is to berealized, policy-based management must emphasize andspeed up the development of new or the revision ofexisting business processes. For IT organizations tomove towards a more service-oriented and evolutionarycomputing model, processes will have to be developedto manage the integration of business objectives withnetwork and distributed infrastructure through policy.Figure 2 depicts this integration with a proposedhierarchy (see also [2]) with which policy is introduced,translated, and propagated within the networkinfrastructure. At the highest level, business rulesshould dictate global (i.e., domain-specific) directiveson the effective use and priority of resources to support


the business objectives. These objectives are translatedinto network-wide policies within the administrativedomain on the necessary bandwidth, communicationresources, and topology requirements. Network-widepolicy rules are translated to node policy rules that arespecific to the required behavior of the node to manageits local resources. Finally, local policy rules areapplied and enforced through one or more specificcommand instructions on device-level functions (e.g.,scheduling and queuing parameterization).

Figure 2: Integrating business rules with networkpolicy

We propose the following methodology from whichpolicy-based systems can be implemented as an aspectof the management process within a corporateenterprise environment or managed network domain:

1. Create a network baseline and track network usageagainst key applications, network services, and businessunit users or usergroups.

2. Establish network domains, policy groups, groupidentities, and hierarchies that map to core businessactivities.

3. Establish organizational directives and createcorresponding policy rules and service-levelrequirements over various applications supportingpolicy groups/users on the allocation or priority use ofcritical infrastructure resources.

4. Administer and deploy policies across the networkinfrastructure, typically on a domain or inter-domainbasis.

5. Audit and validate network policies against servicerequirements.

6. Refine business directives and network policies basedon policy-enforced behaviors.

7. Repeat steps above.

Operational Models—Shifting to PolicyThe operational management models of today's ITenvironment are based on traditional models of systemsand network management. These evolved from a

Business rule or directive

Network (domain) policy

Node policy

Device instruction(s)

Node policy . . . Node policy

Device instruction(s) Device instruction(s)



central, mainframe-orientation to a client-server anddistributed systems management paradigm. A currentchallenge, both for IT management and solutionproviders, is managing both the end-to-end servicemodel and the vertically disjointed infrastructureelements. With recent developments in network QoScapabilities and policy-based management, thischallenge may be met. Operationally, however, service-level management and change management processeswill need to evolve or adjust to support a policyframework. Furthermore, training of operationalpersonnel on business-oriented policy management,network QoS/CoS, and the supporting managementtools will be a requirement. These are discussed inmore depth in the next section.

Change ManagementA critical component of the IT operational environmentis understanding infrastructure changes and managingoperational schedules. For example, avoid changing anetwork the same day software is updated for a largepopulation of servers or clients. Generally, the changeprocess tends to have a high overhead in terms ofadministration and personnel, which is complex tomanage and difficult to administer over a largeenterprise environment. As we move to a policy-managed environment, using existing IT changemanagement processes will require greaterinterdependency and be more complex and costly.Thus, we anticipate an evolution in the administration orchange management process.

The existing change management system is built on ITbeing the direct provider of all IT services. The currentmodel lacks operational flexibility or agility. Policy-based management, on the other hand, will alloworganizations to map the network to the businessrequirements, thus changing the role of the ITorganization to that of a coordinator for serviceprovisioning. This is an evolutionary step for traditionalIT organizations as the change process will directlyengage customers in the change processes.Furthermore, IT project managers are generally focusedon managing the customer and the expectations for theirbusiness unit, application, or area of focus. Thisapproach works but has unforeseen consequences forthe enterprise network and end user when a customer'sapplication is fully turned on. The PBM environmentshould motivate business groups' project planners todrive changes in the IT project office to consolidateplanning activities. The evolutionary process willrequire project leaders to coordinate their activities toengage stakeholders and to ensure that theirapplication/service is properly prioritized and businessrules are communicated and tested. Engaged parties

responsible for sustaining business processes mustunderstand the implications of specific agreed policieson the business environment.

As a policy is implemented, the project manager andoperations change bodies must tightly integrate businessrequirements and the policies deployed within theoperational environment. This requires cross-trainingpersonnel on the integration of business and policymanagement so that they understand the effects orramifications of change policies within theinfrastructure. We envision a consolidation of existingprocesses and tools through the integration of changeand problem management with PBM systems. Bystreamlining processes for change and problemmanagement groups, we will enable data sharing andtrue interdependence. Thus, we believe the generationof Customer Resource Management databases and toolsshould speed up and personalize the change/problemmanagement system to provide customers andstakeholders a view of the relevant data that supportstheir activities.

Service-Level ManagementService-Level Management agreements (SLAs) arecontracts between the provider delivering a service andthe recipient of the service. The SLA codifies theunderstanding between the parties to ensure delivery ofservices and value for payment. A network/application-oriented SLA places value on service delivery of the keycomponents of availability, delay, throughput,customer service, and affordability. A service-levelpolicy is a method for controlling and regulating servicedifferentiation. Together, service differentiation andservice-level policy form an integral part of the serviceSLA function that has become increasingly important asTCP/IP networks evolve.

The shift to PBM will require IT organizations to havethe tools and motivation to manage to tighter servicelevels on service performance, security, reliability, andcustomized agreements. This will require a moredynamic and granular auditing and reporting functionfor the underlying services and the managedenvironment. Additionally, the reporting function mustbe designed to refer to the negotiated SLA and provide ameaningful report that supplies the customer withvalidation that services are being delivered as agreed.Moreover, the customers must have objective proof ofreliability and transaction responsiveness, serviceavailability, as well as be able to rely on the operationssupporting their core business.

TECHNOLOGY IMPLICATIONS



Policy-based management technology can be appliedboth to areas with obvious Returns on Investment(ROIs), such as bandwidth management (e.g., savingson WAN circuit costs) and to areas where the benefitsare harder to measure, such as productivity for certainusers. Since it potentially can require a large capitaloutlay to implement (i.e., if a large amount of legacyequipment needs to be replaced), policy-basedmanagement initially will be implemented as pointsolutions targeted to specific conditions with definableROIs. In this section, we discuss the key barriers ITorganizations have to clear and the key technology areasthat they must address in formalizing an ROI androadmap strategy.

Network Technology

Multi-Vendor InteroperabilityThe infrastructure of an IT organization is made up oftechnology from a host of different vendors. The typeof management policies desired will determine whichequipment will be involved in receiving and enforcingpolicy information. For example, if a company wants togive high priority to certain SAP R/3 users, it may haveto configure the network composed of mixed vendorequipment, the SAP servers, the SAP application itself,the users’ client systems, and possibly some auxiliaryservers (e.g., database and directory servers) in order tomake this happen. As mentioned above, because of thenascent level of policy-based management technologyand products and the embedded base, multi-vendorissues will persist for some time. This issue is beingexacerbated by the move to the Internet ande-Commerce, where policy-based management promisessome of its biggest rewards but the multi-vendor issuesabound.

Quality of Service (QoS) TechnologyPolicy-based management technology solutions andtools presently focus primarily on the enablingmechanisms for delivery of QoS and resource(bandwidth) management. QoS policy can be defined[2] based on some criteria including the endpoints ofcommunication, route or communication path,community of interest or usergroup, application types,network or traffic characteristics, or specific timeperiod. There are primarily two methods to supportend-user QoS: signaled or provisioned. Within the IPcommunication model, RSVP with Int-Serv (IntegratedServices) [7] performs signaling to ensure QoS on a perflow basis by using dynamic resource reservations.RSVP leverages policy-based controls to supportadmission control and flow regulation. Alternatively,Differentiated Services [8] or DiffServ operates on aslower time-scale. It is essentially a provisioning model

to reserve or establish service classes. The DiffServmodel operates on a traffic aggregate basis where flows(one or more) are bundled together according to a setpolicy and treated based on a negotiated class ofservice. The administrative provisioning model is basedon SLAs translated to Traffic Conditioning Agreements(TCA) and enforced through underlying mechanisms(e.g., classification, scheduling) within a router or IPdevice.



Multi-protocol Label Switching (MPLS) [9] and 802.1p[10] are both layer-2 protocols that can be used inisolation or paired with DiffServ or RSVP to deliverQoS on a flow QoS or provisioning basis. MPLS labelswitch routers (LSRs) are positioned (among othercapabilities) to support the integration between ATMand IP by using Label Switching Path (LSP) protocolsto map layer-2 ATM VCI/VPI identifiers with a layer-3IP device to deliver (not exclusively) QoS within anATM WAN core. Within the LAN environment, the802.1p protocol enables priority or service classeswithin a LAN environment, where once again, mappingof layer-2 and layer-3 service classes or trafficprecedence can be supported. The original scope of the802.1p protocol was to support the integrated servicesmodel within the local LAN.

In current vendor solutions, early QoS/CoS capabilitiesto support these protocols/mechanisms exist, but thereare many problems with their adoption. Int-Serv/RSVPhas been plagued by the scalability issue where theclaim is that maintaining per flow state within theInternet or a large enterprise network breaks thefundamental IP architectural model, which is based onend-system state maintenance and control. On the otherhand, Diff-Serv has not reached standardization orindustry maturity. This latter point is especially truewhen it comes to deployment, where the need to supportinter-domain SLAs has not been fully hammered out.However, the concept of the Bandwidth Broker (BB)[11] has been proposed by the DiffServ community tosupport provisioning within intra-network domainboundaries and across inter-domain boundaries todeliver end-to-end quality of service. This work is stillin the early stages and requires more investigation.Nevertheless, the motivation is there to establishstandard building blocks towards enabling QoS withinthe Internet/IP model.

A broader analysis of some of the aforementionedservices and protocols is presented in [12] includingalternative QoS-supported models for Constraint BasedRouting and traffic engineering. Alternative proposalsleveraging the best features or motivations behind theseprotocols or services have also been suggested to deliverscalable, signaled, and provisioned QoS [12, 13, 14].

Within current IT environments, proprietary networkdevice features that support service differentiation aswell as legacy devices are variables in the ROI decisionprocess. Support for legacy systems and proprietarydevices is an essential aspect of our early QoS andpolicy investigation. This includes looking at trafficsegmentation (e.g. VLAN switching) or access control(e.g., multicast filtering) to manage traffic propagation.Moreover, over-provisioning, especially within a LAN

environment, is a practical alternative in the short-term.Nevertheless, with increasing bandwidth requirements,QoS-dependent applications, and the move towards theInternet, the industry QoS models described previouslymay clearly be the favorable long-term choices. Todeliver or manage QoS, the consensus is that policy-based controls must be integrated with devicemechanisms (proprietary or otherwise) to support theprovisioning, admission control, and regulation oftraffic.

Network ManagementAs the demands for distributed and global computingincrease, and Internet-based electronic businessescontinue to grow, network growth (e.g., traffic volume,traffic types) and complexity (e.g. devices types andcounts, network events, interoperability) increase alongwith it. With demand for innovative services (e.g.desktop video collaboration, knowledge-basedmanagement) extending current user communicationmodels, the requirements for these new services willcontinue to increase with a corresponding increase innetwork growth and complexity. While this growth isessential for business and the evolution of informationtechnology, it places rigorous demands on ourinfrastructure. IT managers are faced with thechallenge of managing the infrastructure for optimalservice delivery while at the same time reducingoperational costs (e.g., manpower, operational tools).Finally, traditional network management tools aredevice centric and require manual configuration, whichleads to duplication and task redundancy.

Because of these constraints and limitations, policy-based management is justified. PBM will abstractphysical and virtual elements of the network thatfacilitate the automation of traditional networkmanagement tasks across multiple objects of thenetwork. The automation feature lessens the need forhuman administration, thus speeding the changemanagement process. Furthermore, abstracting networkdevices or components raises the issue ofimplementation or proprietary differences acrossnetwork devices. This allows (under certain conditions)management interoperability and reduces thecomplexity associated with managing implementation-specific devices over alternative interfaces. However,management capabilities available in current policy-based management solutions are not broad enough todeal with the overwhelming activities associated withnetwork management. Nonetheless, the focus onbandwidth management and QoS is certainly the rightchoice in the move to this new paradigm for managingnetworks.



Managing Network ServicesA complementary view of the enabling capabilities ofpolicy-based management is the notion of regulating orcontrolling distributed and abstract network resourceobjects in concert to deliver or manage end-usernetwork services. Such enhanced network servicesinclude video-based distance learning, voice over IP(VoIP), quality of service, multicast services, andsecurity. The service orientation of networkmanagement is a major shift from the traditionalapproach of managing networks. This has been enabledthrough the introduction of middleware [15] servicesand the corresponding abstraction or raising of thelower-level physical communication infrastructure. Thenotion of policy is introduced here as "the ability toadminister, manage, and control access to the networkresources of network elements in order to provide a setof services to clients of the network" [15].

Security Services

InteroperabilityCorporations need to protect their business assets fromcompetitors. This traditionally has been done withphysical barriers and firewalls for informationtechnology assets. E-Commerce and the use of theInternet to provide IT services (such as Web andapplication hosting) is blurring the line between insideand out, requiring that security be implemented on amore information-specific level. The IPSec standard,for example, allows information to be encryptedbetween sender and receiver, thus satisfying a privacyrequirement. However, the use of this encryption mayalso hide pertinent information from any policy-enforcing infrastructure in the path between the senderand the receiver. Thus, some implementations ofsecurity services could eliminate the possibility ofestablishing other policies. We recognize that bothstandards groups and vendors are working to reduce oreliminate this conflict.

Policy-Driven SecurityWhile the current focus of network-based policymanagement tools is on quality of service, themotivation behind policy work extends beyond thefunctional area within the network transport. Clearly,privacy, complexity management, dynamics, resourceaccess, and authorization, etc. will require similarfacilities or capabilities so that growth and trafficpropagation in the context of a security policy can bemanaged. This is especially true for the e-Businessmodel of computing.

However, just as the network vendors have been movingtoward policy-based management so has the security

industry. Abstractions away from managing individualaccess controls on objects, management ofheterogeneous environments, and training of personnelon multiple administrative products all support policy-based management initiatives in the security arena. Wetherefore see a parallel to the network PBM toolset inthe applications and server space for policy-drivensecurity. E-Business may provide synergy for mergingfuture policy-based management tools in thesetraditionally separate areas.

There is an interesting perspective that arises primarilyin the application/server space that divides the policyproducts in half. This comes in the positioning ofmanagement of policy versus execution(implementation) of policy. In the network PBM space,typically one expects that one vendor might provide themanagement tools, while another vendor provides theimplementation, or run-time enforcement of policy.Given the underlying infrastructure of networkmanagement protocol convergence, it is reasonable toexpect a network management toolset to have thecapability to manage across a heterogeneousenvironment.

In the application and server PBM space, on the otherhand, there is no standard for administration across aheterogeneous environment. As a result, administrativetools must either be built with interfaces to manyproprietary targets, or they must provide a singlecentralized security server implementation, and expectthe secured resources to query the security server in acommon language. This split delineates twosignificantly different architectural solutions andbifurcates the product solution space. Intel’s currentarchitecture selection is the former one: a commonpolicy-based management tool manages multipleheterogeneous targets, and the execution-time securityis native to the proprietary target environments.

As we move forward with PBM solutions in theapplication and network space, it is likely we will mergeand simplify this overlap of administrative functions inthe security space. Administrators will associate policywith people, and that policy will apply to a variety oftarget objects. Whether these targets were classically“servers” or “network devices” will be irrelevant. Usersand administrators will both have abstracted views ofthe physical computing environments, defined bybusiness intent and not by topology or by a quirk of avendor’s security implementation.

Directory ServicesHistorically, directory services have been designed withspecific requirements in mind. The relatively static



nature of the directory information update model wasaccepted, and instead the design optimized the high-speed access and replication characteristics. However,the advent of directory-enabled networking, DynamicHost Control Protocol (DHCP) leases, data flow-rate,network state, and routing statistics have made itnecessary to store and distribute low-latency, transient,and dynamic data. When a directory service is capableof supporting ‘dynamic’ data types, it becomes useful topolicy-based network operations, supporting serviceprovisioning or delivery of network state information toapplications. An active association between a user orapplication context stored in the directory and thenetwork can be discovered and used. New possibilitiesenabled by this technology follow:

• provide secure management of information from avariety of sources, including applications andnetwork devices

• define, register, and provide publish/subscribefeatures for network events

• process network events for applications, devices,and users

• expose APIs to applications to take advantage ofdirectory-based services

• maintain state information for devices, users, andapplications to support policy-based management

The core directory service acts as the single point ofadministration for all resources, including users, files,peripheral devices, databases, Web access, and otherobjects. Extended functionality such as that providedby CNS/AD [16] provides access to vendor-specificnetwork elements and services, and securely andefficiently propagated dynamic data. High-speedreplication services securely propagate cached dataamong all directory servers, enabling them to managedynamic network information such as IP lease or userpassword. Core directory functionality relates relativelystatic information about users and applications todynamic information describing a given service requestand the context in which the request has been issued. Insummary, directory services will play a significant partin the establishment and the long-term success ofpolicy-based management.

TECHNOLOGY EXPERIENCE ANDUSAGEWithin Intel IT, our current agenda is to qualify thesuitability of policy-based management for the

corporate enterprise environment as well as to definethe transitional models to support e-Business. Webelieve this approach matches current industry andproduct roadmaps. Our initial technology evaluationobjectives will primarily focus on QoS and bandwidthmanagement to enable service management andresource management, although we hope to gain insightinto other usage areas including security and networkmanagement in a wider context. To facilitate theseobjectives, as illustrated in Figure 3, we have developeda Quality of Service Network for EmergingTechnologies ("QoSNET"). QoSNET is a production-level and policy-managed network environment tosupport proof-of-concept of QoS and policy-basedmanagement technologies. QoSNET will bring togetheremerging technologies including multimediacollaboration technologies and other next-generationapplications (e.g., VoIP) to investigate intelligentbandwidth management capabilities to support scalableand manageable deployment of these emergingcapabilities. Our current activities are focused ontechnology evaluation to support the integration ofpolicy and network QoS features, while recording theoperational procedures necessary to support policy-based management. One of our goals is to validate thetechnology capability as it merges abstract policy ruleswith device QoS capabilities, namely 802.1p and IPprecedence/TOS control mechanisms within layer-2(switches) and layer-3 (routers) enabled devices. Usingthe most recent developments in network equipment, weare also investigating key QoS capabilities including ratecontrol and application-based recognition managedthrough policy invocations. Secondly, through hands-onexperience using policy-based management tools, wewill assess potential process or operational managementimprovements in bandwidth management and networkand service management. Interoperability betweenalternative policy-based management tools and multi-vendor devices is critical to our success. We are alsoinvestigating opportunities to policy manage or controlstandard QoS-oriented technologies being definedthrough the IETF to support QoS/CoS, specifically,RSVP, Differentiated Services, and MPLS. Otherplanned areas for technology evaluation will includepolicy-based routing and evaluation of directorytechnologies to support the integration of system orapplication-based policy information. This latter itemwill include looking at scalability issues on policy withina large enterprise network (i.e., Intel's corporatenetwork) or across administrative domains to supportInternet-based service provisioning and policymanagement.



IT Network Transport Labs

QoSNET Trials Network

Net Com

CH

PBX

PBX

JF

SCATM

Cisco

JF

FM SantaClara local

network

NetMeeting client

Web server & Real Player

Viewer & Ganeymede

client

Viewer &Ganeymede

client

MS Media server

NetMeeting client Web

client

NetMeeting client

Viewer & Ganeymede

client

IntelIntel

IAL/CAL Lab

IT DSM Lab

Viewer & Ganeymede

client

JF3-CRTeamStation

FM1-CR TeamStation

PBM agent & backup

PBNM server& console

Intel

Packeteer

Packeteer

Web/FTP server

Cisco

Cisco

Cisco

Cisco

Cisco

Cisco

Intel

IT Tech R&D Lab

IT Network Transport Labs


Net Com

CH

PBX

PBX

JF

SCATM

Cisco

JF

FM SantaClara local

network

NetMeeting client


Viewer & Ganeymede

client

Viewer &Ganeymede

client

MS Media server


client

NetMeeting client

Viewer & Ganeymede

client

IntelIntel

IAL/CAL Lab

IT DSM Lab

Viewer & Ganeymede

client

JF3-CRTeamStation

FM1-CR TeamStation

PBM agent & backup


Intel

Packeteer

Packeteer

Web/FTP server

Cisco

Cisco

Cisco

Cisco

Cisco

Cisco

Intel

IT Tech R&D Lab


Net Com

CH

PBX

PBX

JF

SCATM

Cisco

JF

FM SantaClara local

network

NetMeeting client


Viewer & Ganeymede

client

Viewer &Ganeymede

client

MS Media server


client

NetMeeting client

Viewer & Ganeymede

client

IntelIntel

IAL/CAL Lab

IT DSM Lab

Viewer & Ganeymede

client

JF3-CRTeamStation

FM1-CR TeamStation

PBM agent & backup


Intel

Packeteer

Packeteer

Web/FTP server

Cisco

Cisco

Cisco

Cisco

Cisco

Cisco

Intel

IT Tech R&D Lab

Figure 3: Information Technology QoSNET Trials Network

Usage Scenarios: e-BusinessWith the introduction of e-Commerce and the move bycorporations towards Internet-based businesses, theinformation technology model will need to support awide range of applications from a diverse IT customerbase including existing internal employees, newemployees from acquisitions, corporate suppliers, orindividual consumers. Further, the user communicationmodel may source from the corporate private network,the telecommuter accessing corporate resources via aremote access connection (e.g., DSL or satellite), orperhaps a corporate partner operating over an extranetvia an ISP virtual private network. InformationTechnology must meet the diverse connectivityrequirements as well as specific user requirements forproductivity, network flexibility, service differentiation,isolation, privacy, and manageability. The complexityof the network along with a diverse user base is greatlyincreased under this new computing model.Furthermore, there will be a shift to delivery andmanagement of services. Under this new paradigm,

motivated by e-Business computing, we identify thefollowing applied areas for policy-based management.

Service ManagementAs QoS technology and policy become morecommonplace, the Internet will support increasedbusiness-to-business communications and inter-domainnegotiations to ensure resource preservation and SLA's.This will require SLA specifications to be translated intotraffic management policies. Moreover, by way of staticor dynamic provisioning the SLAs would be enforcedthrough traffic conditioning and device controlmechanisms. The introduction of policy managementcan facilitate these changes by providing the means totranslate high-level business policies into device-specific mechanisms to support SLA management.Such a model is proposed [17], including customerpolicy, service policy, and flow policy. This modelclosely follows the policy and provisioningadministration models.



Dynamic ProvisioningThere is a general industry trend towards more cost-effective models for bandwidth provisioning (e.g.,VPNs). Static models, based on leased lines throughnetwork carriers, are substituted for dynamic virtualpipes or remote access connections available througheither service providers or network carriers. Theexpectation of support for both short-lived and long-lived networks based on changing capacity andgeographical domains, supporting either unscheduled ornegotiated schedules, is a reasonable one for ISPcustomers or IT organizations. In addition to traditionaloffice and business applications, provisioning modelswill require support for alternative QoS traffic typesenabling emerging computing applications: VoIP, video-based technologies supporting real-time streamingapplications for online training, or real-time applicationsto support video conferencing or desktop collaboration.The provisioning or signaling support for intra-domainand inter-domain SLAs, in addition to resourcemanagement features for load balancing or rate control,will require tools that can facilitate network control andadministration automation over complex networks andalternative user models. Policy-based managementtools will provide obvious value here.

Internet Pricing and BillingModels for Internet-based e-Business will involve someform of accounting for subscribed services and use ofshared resources. Although several proposals [18, 19]have been published in the area of Internet pricing, it ispossible that the right model is somewhere between theshared flat rate model of Internet ISP's and the networkcarrier telephony model of usage-based pricing.Moreover, with alternative levels of service (i.e., QoS)becoming available to Internet users or organizationalsubscribers, the pricing model may be extended tosupport subscribed service levels, in addition toresource use. Policy-based data stores will support theavailability of such information to support billing, SLAauditing, and validation. For example, a billing modelbased on the following policies is proposed [17]:

• Billing policy: customer type and credit associatedwith a request for a given service

• Charging policy: charging tables describing service,resources, time, and cost for a domain

• Accounting policy: accounting information aboutresources used and the cost for each customer

SecurityThe requirements for policy-based security focus on theintegration of policies across administratively separate

or heterogeneous domain boundaries. Security has toallow individual consumers, corporate suppliers, andacquisitions/mergers as well as corporate businesspartners to operate under a shared, and what isperceived to be, border-less infrastructure. End-endsecurity will require a tighter integration of policyacross the separate security realms, which aretraditionally disjointed across networks, applications,and servers. Finally, dynamic policies will be arequirement for the administrator, providing him/her themeans to perform on-the-fly control or automation ofshort-lived policies to secure content andcommunications (e.g., inter-company video conferenceor online supplier training sessions). Enabling such aparadigm will require a higher level of abstraction,automation, and integration across infrastructureelements. We feel that policy-based managementsolutions would work here by aiding in the definition ofallowable security associations, integrating policyabstractions across administrative or heterogeneoussecurity boundaries, facilitating encryptionparameterization, and by rapid and dynamicconfiguration of boundary devices (e.g., VPN, firewall,and proxy services).

SUMMARY AND FUTURE WORKIn this paper, we introduce the key drivers for current ITinfrastructure evolution: mission criticality, networkarchitectural agility, service flexibility, and efficiency.To enable these drivers, we presented the backgroundon policy-based management beginning with thenecessary set of requirements to realize the technology.We then presented a framework for policy-basedmanagement and discussed implementation alternativesto support the proposed framework. We believe that ITbusiness and operational models have to be augmentedto transition to a policy-driven management approach;yet, we argue that the changes should help ITorganizations align business objectives with moreeffective use of infrastructure resources. We proposeda simple methodology, which could be implemented byenterprise managers and we suggested that changemanagement and service-level management evolve toalign with policy-driven business and operationalprocesses. The ROI decision criteria for policy-basedmanagement and technology challenges that ITorganizations must address will include the selectionand integration of QoS technology, vendor technologyinteroperability, enabling policy-based security, networkmanagement, and directory service integration.

Finally, there are still pending issues that will not beresolved until policy-based management maturesindustry-wide. Industry standards (primarily IETF and



DMTF) in the areas of policy-based directory schemas,QoS technologies (e.g., DiffServ, RSVP, MPLS), andpolicy and directory communication protocols (e.g.,COPS, LDAP) are still under development and maydelay full vendor adoption. Intel is very active indriving technology in the direction of these standards.Additionally, policy scalability, QoS and securityconflict resolution, and interoperability will furtherinfluence IT strategies and the adoption of PBMtechnologies.

Over the next year, Intel IT will investigate these issueswhile gaining experience with policy and QoStechnologies. We anticipate continued convergence inthe directory arena, as this technology should serve asthe foundation for the success of PBM. A widelydeployed solution will depend on the eventualintegration of alternative technology. The move to e-Business and Internet-based computing will forceorganizations as well as ISP's to focus on and speed thedelivery of a policy-driven approach to managingInternet-based IT infrastructure and enhanced networkservices.

ACKNOWLEDGMENTSThe authors thank David Mills for his insights and hiscontributions to the security sections presented in thispaper, as well as folks from the Intel Architecture Labsand various groups from within Information Technologyincluding Finance, Strategy & Technology, andInfrastructure Products Engineering for theircontributions.

We also thank NSD for their technology support and ourexternal associates, including Hewlett Packard, CiscoSystems, Packeteer, and Net Com Systems for theirtechnology support and collaboration.

REFERENCES[1] J. Strassner, E. Ellesson, “Terminology for

Describing Network Policy and Services,” InternetDraft draft-strassner-policy-terms-01.txt, Feb.1999.

[2] R. Rajan, D. Verma, S. Kamat, E. Felstaine, S.Herzog, “A Policy Framework for Integrated andDifferentiated Services in the Internet,” IEEENetwork Magazine, Sept./Oct. 1999.

[3] J. Boyle, R. Cohen, D. Durham, S. Herzog, R.Rajan, A. Sastry, “The COPS (Common OpenPolicy Service) Protocol,” Internet Draft draft- ietf-rap-cops-07.txt. Aug. 1999.

[4] W. Yeong, T. Howes, S. Kille, “LightweightDirectory Access Protocol,” RFC 1777, March1995.

[5] B. Moore, E. Ellesson, J. Strassner, “PolicyFramework Core Information Model,” InternetDraft draft-ietf-policy-core-infomodel-00.txt. June1999.

[6] R. Braden, L. Zhang, S. Berson, S. Herzog, and S.Jamin, “Resource ReSerVation Protocol (RSVP),”Version 1 Functional Specification, RFC 2205, Sept.1997.

[7] R. Braden, D. Clark, S. Shenker, “IntegratedServices in the Internet Architecture: an Overview,”RFC 1633, June 1994.

[8] Y. Bernet, S. Blake, J. Binder, M. Carlson, S.Keshav, E. Davies, B. Ohlman, D. Verma, Z. Wang,W. Weiss, “A Framework for DifferentiatedServices,” Internet-Draft draft-ietf-diffserv-framework-01.txt, work in progress, Feb. 1999.

[9] R. Callon, P. Doolan, N. Feldman, A. Fredette, G.Swallow, A.Viswanathan, “A Framework forMultiprotocol Label Switching,” Internet Draft,draft-ietf-mpls-framework-01.txt., May 1998.

[10] M. Seaman, A. Smith, E. Crawley, J. Wroclawski,“Integrated Service Mappings on IEEE 802Networks,” Internet Draft, draft-ietf-issll-is802-svc-mapping-03.txt., Nov. 1998.

[11] K. Nichols, V. Jacobson, and L. Zhang, “A Two-bitDifferentiated Services Architecture for theInternet,” Internet Draft, draft-nichols-diff-svc-arch-00.txt. Nov. 1997.

[12] Z. Xiao, L.M. Ni, “Internet QoS: Big Picture,”Dept of CS, Michigan State University.

[13] I. Andrikopoulos, G. Pavlou, “SupportingDifferentiated Services in MPLS Networks,” Proc.7th International Workshop on Quality of Service(IWQOS'99), London, May 1999.

[14] R. Yavatkar et al., “A Framework for use of RSVPwith Diff-serv Networks,” Internet Draft draft-ietf-diffserv-rsvp-00.txt., June 1998.

[15] B. Aiken, J. Strassner, B. Carpenter, I. Foster, C.Lynch, J Mambrette, R. Moore, B. Teitelbaum,“Terminology for Describing Middleware forNetwork Policy and Services,” Internet Draft, draft-aiken-middleware-reqndef-01.txt., May 1999.

[16] Cisco White Paper: Cisco Networking Services forActive Directory, available at



http://www.cisco.com/warp/public/cc/cisco/mkt/servprod/sms/common/tech/cnsad_wp.htm

[17] T. Ebata, M. Takihiro, S. Miyake, M. Koizumi, F.Hartanto, G. Carle, “Interdomain QoS Provisioningand Accounting,” Internet Draft, draft-ebata-inter-domain-qos-acct-00.txt., Oct. 1999.

[18] N. Semret, R. F. Liao, A. Campbell, A. Lazar,“Market Pricing of Differentiated Services,” Proc.7th International Workshop on Quality of Service(IWQOS'99), London, May 1999.

[19]S. Shenker, D. Clark, D. Estrin, S. Herzog, “Pricingin Computer Networks: Reshaping the ResearchAgenda,” ACM Computer Communications Review,pp. 19-43, 1996.

AUTHORS’ BIOGRAPHIESJohn Vicente is a member of Intel's IT organizationwhere he is involved with strategy and technology in theareas of Internet-QoS, policy-based networking,multimedia, and programmable networks. John is alsoworking with Dr. Andrew Campbell as a Ph.D.candidate at the Center for TelecommunicationsResearch at Columbia University, New York. Hereceived his M.S. in Electrical Engineering from theUniversity of Southern California, Los Angeles, CA in1991 and his B.S. in Computer Engineering fromNortheastern University, Boston, MA in 1986. His e-mail is [email protected].

Harold Cartmill, since joining Intel in 1990, hasapplied his extensive LAN, WAN, server, mainframe,and system management expertise to solvingadministration and support problems in Intel'smainframe and Windows NT server environments.Harry has served as Technical Lead for Intel's RemoteLAN, cc:Mail, Global Remote Server, and EventManagement projects. His most recentaccomplishments have been in the design,implementation, and support of systems managementmethodologies. Concurrently, Harry is activelypursuing his B.S. degree in electrical engineering atCalifornia State University, Sacramento, CA. Hisestimated completed date is the Spring of 2001. Hisemail is [email protected].

Glen Maxson worked for the Boeing Company inSeattle, WA for 17 years before coming to Intel in 1994.While at Boeing, Glen spent the majority of his timesolving the ‘Enterprise Directory’ problem, a quest thatcontinues to challenge him as Intel’s Directory Servicearchitect. He completed his undergraduate studies atPennsylvania State University in 1977. Glen holds a

Certificate in Data Resource Management (1991). Hise-mail is [email protected].

Shelby Siegel is a network architect in Intel’s ITStrategy and Technology organization where he doesstrategic planning for Intel’s worldwide network. Hereceived his B.S. degree in mathematical sciences andhis M.S. degree in computer science: computerengineering from Stanford University in 1975. His e-mail is [email protected].

Russ Fenger is a senior architect and engineeringmanager in the Intel Architecture Labs where he leadsthe research and development of Policy Based NetworkManagement architectures. Russ's other researchinterests are quality of service in the Internet and innetwork security. Russ has been with Intel since 1983after graduating from Iowa State University. His e-mailis [email protected].


Corporate Portal Framework for Transforming Content Chaos on Intranets 1

Corporate Portal Framework for Transforming Content Chaoson Intranets

Atul Aneja, IT Strategy & Technology, Intel Corp.Chia Rowan, IT Marketing, Intel Corp.

Brian Brooksby, IT Marketing, Intel Corp.

Index words: corporate portal, enterprise portal, intranet, personalization, categorization, taxonomy,search engine, information overload, repository, architecture, framework, profiles, knowledgemanagement, content management, metadata, XML

ABSTRACTIn this paper, we describe a strategy for managinginformation overload on corporate Intranets. We define acorporate portal and describe the framework andcomponents that are essential to providing the capabilityto organize content using categorization; to provide aWeb-based interface to information; to personalize theportal, allowing employees to tailor information for theirindividual requirements; to search multiple repositoriessuch as e-mail, file and data stores, and the World WideWeb; and to access different sources of informationthrough a universal client interface.

INTRODUCTIONIn the last few years, information technology and theInternet have exponentially increased the amount ofinformation that Intel employees must process every day.Information is delivered at an astonishing pace and from adizzying array of sources such as e-mail, news,documents, reports, articles, digital files, video and audiofiles, and transactional data. Yet, it is difficult to takeadvantage of this wealth of information because it isburied in separate, often disconnected and disorganizedrepositories. In addition, the volume of data leads toinformation overload for our employees.

Herbert Simon, an economist, describes informationoverload as follows:

What information consumes is rather obvious: it consumesthe attention of its recipients. Hence, a wealth ofinformation creates a poverty of attention and a need toallocate that attention effectively among the

overabundance of information sources that might consumeit.

We must confront the problem of information overload atmany different levels, using a combination of approaches.A corporate portal will help transform some of the chaosexisting today on Intel’s Intranet.

INFORMATION OVERLOADE-mail is a key cause of information overload at Intel. Butit isn’t e-mail alone that creates this problem. We havemore than one million URLs on our Intranet, with morethan 100 new Web sites introduced every month. Much ofthis information is stored in a disorganized fashionresulting in some business decisions being based onincomplete or out-of-date information.

The following is a short list of key issues that make itdifficult to access information in a timely manner:

• Information is scattered throughout Intel in personaldocuments, e-mails, transcripts of discussions, etc.

• Finding relevant, accurate information is time-consuming, difficult (if not impossible), and oftenrequires searching multiple systems.

• Information is accessed through different methodssuch as Web browsers, e-mail clients, andapplications.

These combined issues result in the loss of productivetime spent searching for information, the increasedlikelihood of making decisions based on incomplete andinaccurate data, and the failure to effectively respond toimportant messages and information. Our research onthese issues suggests a corporate portal (sometimesreferred to as an enterprise portal) as a potential solution.



THE CORPORATE PORTAL CONCEPTCorporate portals are a relatively new business concept. Acorporate portal is an Intranet site that is similar in designto popular Internet sites such as My Yahoo∗ . It offers asingle point of access for the pooling, interaction, anddistribution of organizational information [4]. Thisbrowser-based system provides universal access tobusiness-related information in the same way that anInternet portal acts as a gateway to the wealth of contenton the Web [1]. A corporate portal enables a company toprovide users with a single gateway to the personalizedinformation they need to make informed businessdecisions [1]. Thus, a corporate portal can increaseemployee productivity by addressing many of the issuesdescribed in the previous section [1,2,3,4].

Figure 1 shows the corporate portal adoption rate based ona Delphi Group survey of Fortune 500 companies [4].About 35% of these companies have implemented acorporate portal and another 30% are in thepilot/experimental stage of development.

Figure 1: Corporate portal adoption rate

CORPORATE PORTAL STRATEGYThe key steps to Intel’s corporate portal strategy includethe following:

• Identify the content that is or will be available, andidentify where this content resides.

∗ Other brands and names are the property of their respectiveowners.

• Leverage existing systems, resources, andrepositories.

• Include both structured and unstructured information.

• Organize content into categories that can be browsedand searched.

• Integrate search functionality across multipleinformation repositories.

• Build a platform for publishing and subscribing tocontent.

• Deliver personalized content and services to usersbased on their preferences and roles.

• Develop the corporate portal in phases.

• Create online “communities” to connect people andenable collaborative work.

• Develop an extensible architecture that allows forextended functionality.

• Sustain a collaborative portal by “ institutionalizing”it within daily business operations and weaving it intolong-term strategies.

• Purchase an integrated portal product rather thanbuilding custom portal functionality.

Business BenefitsThe main benefit of a corporate portal is the increasedemployee productivity that results from the followingimprovements:

• organized and structured information, which is easierto navigate

• quick access to relevant personalized news,information, services, applications, and documents

• a highly interactive and personalized interface thatprovides targeted information based on employees’roles and preferences

• enhanced search capabilities that reduce the amountof time necessary to find sought after information

• filtered, targeted, and categorized information sousers receive just what they need

CORPORATE PORTAL FRAMEWORKAND CAPABILITIESAfter an extensive R&D project the team identified anappropriate set of capabilities and infrastructure elementsfor a corporate portal (see Figure 2).



(Due to the breadth of corporate portal functionalpossibilities, focusing on the key requirements is critical.

This framework could be adapted to fit different businessrequirements.)

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�� !��!��

��

��

�� "�"�

�� "��"��

��

�� #$�� %�� #$�� %��

�� "��"��

��

�� &��'&��'

�� ()()

��

��

�� *�+��*�+��

��

��

!��"��!��"��

#��#��

!��!��

ExternalExternal

InternalInternal

Scalab

ility

Scalab

ility Extensibility

Extensibility

XMLXML

ProfileProfile

SecuritySecurity GadgetsGadgets

Brow

ser

Brow

ser

Figure 2: Corporate Portal Framework

CategorizationThe sheer volume of data maintained on Intel’s IntranetWeb sites is expanding rapidly and is scattered throughoutthe company without any context. A portal can providethe structure, organization, and context necessary totransform this information into a significant Intel resource.A Yahoo∗ -like structure that organizes information underspecific categories is an efficient way to enable employeesto find specific information of interest.

Categorization of corporate information is critical becauseit provides employees with a navigation directory that canbe browsed to find specific information. During thecategorization process, each piece of information must belabeled to indicate what it contains and how it relates toother pieces of information. For example, a collection of


documents can be organized alphabetically, or by subject,function, business group, geography, projects, products,etc. The challenge is to create categories that make themost sense to business users. To address the needs ofdifferent users, it can be an effective strategy to createmultiple views of the same sets of documents.

The first step in the categorization process is to identifythe high-level categories under which information can beorganized. Once this classification, or taxonomy, iscreated, the next step is to define the process of evaluatinghow content and documents will be indexed within thetaxonomy. In the past, this was done manually. Today,automatic categorization technology is maturing quickly;although it still requires manual intervention by a librarian.In either case, users must be able to use a Web browser tobrowse the taxonomy.

One of the key issues for searching and assessing contentis establishing context: what is it, what it is about, whenwas it created, who created it, and for what purpose was it



created. Metadata can help establish this context.Metadata is information about information. For example,metadata consists of keywords and summaries that helpprovide context for information on the Web. In addition,metadata from different documents can be used todescribe the relationships among documents.

Content Publication and ManagementContent lifecycle management helps to prevent contentfrom becoming dated. One useful practice is to specify anexpiration date for all content. When the content expires,the owner is notified and can then extend the contentexpiration date or let the content be archived. Therefore,when creating a corporate portal, the process forauthoring, approving, and publishing different types ofcontent must be defined. Utilizing an automated workflowcan ease this process.

The most effective time to capture metadata is during theauthoring process because authors usually have the mostknowledge about the content. Some document formatsallow metadata to be stored inside the document itself.HTML contains <TITLE> and <META> elements forstoring such metadata as titles, keywords, descriptions,and author information. Microsoft Office∗ documents canstore standard summary information such as title,description, and author.

Where possible, XML tags should be used to identify eachsearchable attribute in a document so that search engines,robots, and applications can locate the document whensearched. Using metadata to search within documents forspecific fields such as author, title, or location enhancesthe ability to pinpoint information.

Integrated SearchBecause corporate information resides in numerousplaces, an integrated search capability across multipleinformation repositories (Web/Intranet, e-mail, discussionforums, databases, and applications) is essential.

A full-text search function can produce highly accurateresults. As mentioned above, the capture and use ofmetadata across content is another way to improve searchresults. Whether searching text or metadata, the searchcapability must function both within and among selectedrepositories.


PersonalizationThe goal of personalization is to deliver content relevantto an individual user or group of users based on their rolesand preferences. The ability to present a personalized,relevant set of information is critical to portal success.Each user’s portal experience must be tailored to thatuser’s preferences, security levels, and accessauthorizations. Also, users should be able to subscribe tospecific content, and should be informed (by a notificationservice) when that content changes.

Personal profiles are the architectural components thatcontain user information and preferences. These profilescan be created from information manually entered by theuser, or by gathering user information from existingdatabases. User profiles can then be used by portalservices to tailor their content for the user.

Several architectural considerations are necessary forpersonal profiles. Should there be a profile within eachapplication that will be personalized. If yes, should theprofile be based on a common schema. Or, is the personalprofile commonly shared across applications. Using acommon personal profile offers several advantages,including a single place to store and manage user profiledata. This removes the necessity of having the same user(who needs access to different systems) in multipleprofiles.

When creating a personal profile, it is often difficult forusers to specify all their preferences up front. Over time,their preferences will evolve and change. Through userbehavior monitoring and analysis, automatic profiling canhelp to address the issue of out-of-date user profile data.In addition, users need to have the ability to modify andupdate their profile data.

Moreover, because user profiles contain personal data,privacy is a significant issue. Privacy strategies must beestablished to effectively protect private data. Becausemany countries have specific privacy laws andrequirements, an understanding of international privacylaws is also critical.

Goal-Oriented Interface and Navigation

The design of the interface and navigation scheme is keyto the success of a corporate portal. Information must beavailable through a common, browser-based userinterface. Moreover, the interface must be intuitiveenough that users do not need to understand and configurea complex system in order to gain value from the site.

Developing an intuitive user interface requires significantresearch. The first step in the process is to define who theusers of the site will be. Next, learn about their goals and



motivations, such as what problems are they trying tosolve. An understanding of how users do their work isalso very helpful. Then ask how the interface can help theuser reach these goals.

Formulate your usability criteria, and then test it with thepotential users of the portal site. Show the site to themand document their experiences. Watch them navigate.Note their body language as pages appear on the monitor.Most importantly, let them control the mouse. Finally, usethe information you gain during the testing period to makeadjustments to the interface. And, make sure the designhas built-in flexibility so that adjustments can be madequickly and easily throughout the life of the portal site.

IntegrationThe ability to present a unified view of corporateinformation depends on integration capabilities. A keyenabler is the definition of an information architecture forthe corporate portal.

As mentioned previously, corporate information is spreadacross many sources within Intel. The seamlessintegration of information from both structured andunstructured repositories is a challenge. The appropriatelevel of integration must be determined. Figure 3 showsvarious information repositories that need to be integratedthrough the corporate portal.

EnterpriseApplications

Unstructured Information Repositories

DataWarehouse

DatabasesOffice

DocumentsWebPages

DiscussionForums

DecisionSupport Sys.

OnlineInfo. Feeds

CorporatePortal

Structured Information Repositories

Figure 3: Structured and unstructured repositories

CollaborationA corporate portal can effectively create a sharedcommunity across the organization. This community iscritical in an age when virtual teams and groups arebecoming more significant.

The corporate portal also can be used to project theorganizational identity, deliver corporate messages, andinteract with employees.

Advanced capabilities to build community are critical to acorporate portal. Asynchronous capabilities include e-

mail, discussion forums, etc. Synchronous capabilitiesinclude online meetings, video conferencing, and chat.Network bandwidth must be considered whenimplementing synchronous collaboration capabilities.

CORPORATE PORTAL ARCHITECTUREFigure 4 shows a corporate portal capability architecture.A few of the architectural principles underlying acorporate portal are described below.



Real-time

PersonalProfile

Crawler

Gadget

EnterpriseApplications

Unstructured Information Resources

DataWarehouse

DatabasesOffice

DocumentsWebPages

DiscussionForums

DecisionSupport Sys.

OnlineInfo. Feeds

Structured Information Resources

Gadget Gadget GadgetSearch Index

TaxonomyEngine

SearchEngine

PersonalizationEngine

Security

SubscriptionFacility

MetadataIndex

Interfacing

Portal

Real-timeNews

StockQuotes

Travel

ExternalResources

Others

Gad

get

Figure 4 : Corporate portal capability architecture

Plug-and-PlayThe portal framework needs to offer a plug-and-playcapability that will allow additional functionality as theportal grows to meet future requirements.

Standards-Based ToolsThe portal framework utilized several emerging standards,specifically, HTTP, IP, HTML, and XML.

The eXtensible Markup Language (XML) separatescontent from how that content is presented. Data is keptcompletely independent from its known, and maybe yet-to-be-determined, output format(s). The separation ofdata and format allows the same content to be presented inmultiple views. One possible application for the corporateportal is building personalized views based on userpreferences.

Style sheets determine how an XML-based document isrendered for presentation. Style sheet standards includethe following:

• Document Style and Semantics SpecificationLanguage (DSSSL) provides facilities for display andtransformation of SGML and XML data.

• Cascading Style sheets (CSS) provide straightforward,generally sufficient features for using styles in thecontext of XML and HTML.

• Extensible Style Language (XSL) is designed in atemplate-oriented fashion to provide powerfulfeatures to display and transform XML data.

Extensibility Through “Gadgets”“Gadgets” are new application tools and services in theportal, provided via modular components. “Gadgets”provide the architectural construct to enable futureextensibility without having to completely redevelop theportal. By using gadgets, new functionality could beeasily added.



CHALLENGESThere are two key challenges facing developers ofcorporate portals: available technology and viability ofcorporate information. Today, many technology suppliersare repositioning themselves as portal vendors, but not allof these vendors have comparable solutions. A carefulanalysis of these vendors is necessary to fully understandtheir specific capabilities and technical approach.

It is also important to realize that a portal is only as goodas the corporate information repositories that it accesses.If the information in these repositories is not correct or upto date, then the portal information will not be correct orup to date.

DISCUSSIONDo integrated packaged portal systems provide a bettersolution than custom-developed corporate portals?

Two potential approaches can be taken when creating acorporate portal. The first approach is for developers toselect and use the best solution for each capability definedin the corporate portal framework. The second approachis to look for an integrated solution that offers most of thedefined capabilities. Our research suggests that trulyintegrated corporate portal solutions contain morefunctionality, are easier to maintain, less expensive overthe complete lifecycle, and are faster to deploy.

CONCLUSIONCorporate portals are the logical evolution of Intranet sitesas organizations continue to look for ways to improveemployee productivity and job satisfaction. To this end,Circuit, Intel’s Intranet home page, is evolving into acorporate portal. Currently there is a team working on anext generation prototype of this corporate portal and apilot of this new technology is planned for Q2, 2000.

Due to the fast pace of change in portal technology and inbusiness requirements, the strategy and architecture of acorporate portal should be flexible enough to evolve overtime in response to these ongoing changes.

ACKNOWLEDGMENTSWe thank IT R&D Council for allowing us to research thiskey emerging area. We also thank IT management forrecognizing this area of importance and for believing inus. We acknowledge the cross-functional R&D CorporatePortal Team .

REFERENCES[1] Enterprise Information Portals, Merril Lynch, 16

November 1998.

[2] G. Phifer, Portal Trends Emerge Among Confusion,Gartner Group, April 1999.

[3] M. West, A Framework for Enterprise PortalSimplifies Intranets, Gartner Group, April 1999.

[4] Corporate Portals, Delphi Group, April 1999.

[5] Reynolds, H. and Koulopoulos, T., EnterpriseKnowledge Has a Face, March 30, 1999, IntelligentEnterprise.

AUTHORS’ BIOGRAPHIESAtul Aneja is an architect in the IT Strategy &Technology group where he leads Intel’s R&D efforts inthe areas of corporate portals, personalization, and e-Business architecture. His other research interests includesoftware architecture and emerging applicationcapabilities. His e-mail is [email protected].

Chia Rowan is a manager of Circuit, Intel’s corporateIntranet site and is responsible for its evolution into acorporate portal. Her email is [email protected].

Brian Brooksby is the Technical Marketing Engineer for“Circuit,” Intel’s corporate Intranet site. He has designeddocument management and Web publishing systems forseveral Intel Web sites and is currently focused ondeveloping Intel’s corporate portal. His e-mail [email protected].


Implementing an Industry e-Business Initiative: Getting to RosettaNet 1

Implementing an Industry e-Business Initiative:Getting to RosettaNet

Patricia J. O’Sullivan, IT Strategy & Technology, Intel Corp.Don S. Whitecar, PE, IT e-Business Integration, Intel Corp.

Index words: e-Business, e-Commerce, standards, standards implementation, enterprise applicationintegration, EAI, RosettaNet, B2B, trading partner automation, trading partner integration, B2Bgateway, XML, WWW

ABSTRACT

As Intel looked at the cost of its own successful earlyimplementation of Web-based e-Commerce, it becameclear that an industry-wide standards-based approach toe-Business is the only way to go.

We decided to help build the right business-to-business(B2B) specifications with the right industry initiative(RosettaNet∗ is our main focus) and then implementthose specifications. As early adopters, this has turnedout to be much more of an enterprise readiness effortthan initially appreciated. Team composition, technicaland business knowledge coalescence, formal andinformal communication channels, cross-enterprisevisibility, and establishment of appropriate resourcelevels are just some of the challenges we face.

We anticipate that an evolving, more robustinfrastructure, together with lessons learnt from pilotprojects, team experience, and more mature standardswill lead to the full realization of expected benefits fromRosettaNet. However, we offer here a “readinessmodel” that we hope can be used by others to “spin up”faster.

INTRODUCTIONIn early 1998, Paul Otellini, then Sr. Vice President ofIntel’s Sales and Marketing Group, crystallized much ofour early thinking and experimenting with Internet-based e-Commerce into a simple challenge: take in$1Billion in sales orders via the Web in Q4’98. Wetook our first such order in July 1998 and had arrived at

∗ Third-party brands and names are the property of theirrespective owners.

$1B per month by the start of Q4’98—success beyondour wildest dreams!

So, with that success, one may well ask what theproblem is. Well, each customer’s internal processesand systems are almost always different from ours, sowe had no way to ensure that our applications, whichworked well for us, did not introduce extra work orbecome otherwise burdensome for our customers. And,our customers buy many products from many suppliersin order to make up their complete product lines, so theyare potentially facing extra work from each of theirsuppliers. As we moved forward with various plans toInternet-enable the way we do business with ourcustomers, we also realized that the part of Intel thatbuys products was getting ready to establish a wholeseries of web-based procurement applications that wewanted our suppliers to use.

So, not only were we building a suite of applicationsthat did not necessarily optimize e-Business for ourtrading partners (the phrase used generally in the e-Business arena to refer to other companies with whichwe do business, whether as customers, suppliers, orother), but we were facing the prospect of developingand/or buying a whole slew of applications that we (andour trading partners) would have to support andmaintain over time.

What we needed were standards! However, we did notwant standards that took years to develop; rather, weneeded those that evolved at the same pace as theInternet, at the same pace as the emerging “killer app”of e-Business, and at the same pace as the technologiesthat underlay that growth. Furthermore, these standardshad to focus on the real-world business processes of thesupply chains that we are a part of, not those that triedto create a single universal e-Business solution or thatsacrificed implementation to elegant technical solutions.



Of equal importance, these standards would need asound, extensible architecture; would have to beadopted rapidly; and would have to be demanded bymanagement and supported by business and technologystakeholders within our business environment.

Figure 1 illustrates the tension between some of Intel’se-Commerce solutions and our desire to use standard

solutions, as projected over time.

Figure 1: Intel standards challenge

As is true throughout history, but even more noticeablein today’s Internet economy, timing is everything. Inlate 1997, an executive from one of the largestcomputing products distributors in the worldapproached us (as well as other leading players in the“IT Products” supply chain) with a vision and a plan.The vision was to create a common vocabulary andprocess set for e-Business in the context of a wellunderstood supply chain. The plan was to pull togethera fast-moving business consortium of companiesrepresenting over half the revenue of that supply chain,managed by a board of top executives from membercompanies, who would precommit to implementing thespecifications that their members would jointly developand vote upon.

At Intel, we pulled together a quick evaluation team,surveyed both our internal e-Business initiatives (suchas Web Order Management (WOM), Supply LineManagement (SLM), and eFORM) and the external e-Business standards/initiatives environment, and wequickly concluded that we needed to help realize thiseffort, and the sooner, the better. In the words of ColinEvans, Intel’s Sales & Marketing e-Business architect,our competitive advantage would have to move from“first to move” to “first to standards.”

And thus begins the lessons we have learned (and arestill learning) about implementing business-to-businessstandards across the enterprise.

INTEL’S ROSETTANETIMPLEMENTATIONToday we are preparing to meet our first majorcommitment to have a production-level implementationof at least one of the RosettaNet “partner interfaceprocess” (PIP) specifications (which are described morefully below) running on a robust infrastructure, with atleast one trading partner. Among RosettaNet members,this milestone is known as “2.2.2000,” which is the datethat we will all be ready to demonstrate our success.We are in the thick of this implementation, learninglessons every hour.

The main focus of this paper is on what it takes to beinternally ready to adopt and implement the suite ofspecifications collectively known as RosettaNet. We tryto make these observations as concrete as possible,without being necessarily RosettaNet-specific. Theyshould be of interest to anyone who is preparing toimplement any standards-based approach to e-Business.

In order to understand the magnitude of ourimplementation effort (both initial and longer-term), it isnecessary to provide a little background on theRosettaNet business and technical architecture. Thebulk of this paper focuses on our use of a readinessmodel to ensure that our solutions could be deployed.

ROSETTANET OVERVIEWAlthough RosettaNet’s supply chain scope began withIT products (e.g., boards, systems, peripherals, finishedsystems), it has expanded to include electroniccomponents (e.g., chips, connectors). Intel obviouslyplays a role in both of these supply chains (oftenabbreviated as IT and EC). As maturity is gained inthese environments, it is likely that RosettaNet’sbusiness scope will expand to other supply chains aswell. Each supply chain’s standardization efforts areoverseen by a managing board composed of membercompany executives, who prioritize efforts, ensuresynergy between supply chains as much as possible, andoversee resource allocation as administered by a paidstaff.

RosettaNet focuses on three key areas requiringstandardization in order to automate businessinterchanges between trading partners. First,vocabulary needs to be aligned; this includes bothbusiness and technical terminology germane to thetransaction at hand. The RosettaNet Dictionary,drawing upon existing industry standards whereverpossible, fills this need. Second, the way in whichbusiness messages are wrapped and transported must bespecified. The RosettaNet Implementation Framework,which specifies the use of XML (Extensible Markup

EDI

EForm

RosettaNet

WOM

WebEDI

Standard

Standard

Standard

Intel

Intel

PhoneFax Intel

1998 1999 2000Prior



Language), the World Wide Web (WWW), and otherprotocols serves this need. And third (and mostimportant) the business processes governing theinterchange of the business messages themselves mustbe analyzed, harmonized, and specified. RosettaNetterms these “Partner Interface Processes” or PIPs.Figure 2 shows these RosettaNet “ingredients.”

Human-to-HumanBusinessExchange

Partner-to-Partnere-BusinessExchange

PIP

e-Commerce Application

Business Process

Telephone

Grammar

Words

Alphabet

Sound

DIALOG

XML

Internet

Dictionary

Framework

e-Business Process

Ro

settaNet

Figure 2: RosettaNet ingredients

To perform the work of analyzing, recommending, anddocumenting proposals for voting by the membership,RosettaNet member companies volunteer expertresources, both business and technical people, to leadand/or be a part of project teams. These people areeither on part-time project duty or on detached full-time(short-term) assignments.

At present, six “clusters” of business activities (such as“Order Management”) have been identified as initialtargets of RosettaNet standardization efforts by theRosettaNet Managing Boards. Within those clusters,“segments” have been identified (e.g., within the OrderManagement cluster, “Quote & Order Entry” is one offour segments. Each segment is then analyzed inworkshops that identify the necessary PIPs anddocument the choreography and business requirementsaround each PIP. RosettaNet anticipates that between100 and 120 PIPs will result from the six clusters.

It is worth noting that when the second supply chain(EC) was added to RosettaNet’s business scope, onlytwo additional segments (and no additional clusters) hadto be added. There is more synergy among relatedsupply chains than many had guessed; this gives usmore optimistic expectations for the addition of relatedsupply chains.

Current RosettaNet clusters and segments are asfollows:

• review segments: partner review; product/servicereview

• product introduction segments: preparation fordistribution; product change notification

• marketing management segments: marketingcampaign management; lead and opportunitymanagement; design win management (EC only)

• order management segments: quote and order entry;transportation and distribution; productconfiguration; returns and finance management

• inventory management segments: price protection;collaborative forecasting; inventory allocation andreplenishment; inventory and sales reporting; shipfrom stock and debit/credit (EC only)

• service and support segments: warrantymanagement; asset management; technical support

RosettaNet member companies are increasinglyrealizing that, although the PIPs specify processes onlyat the point of interface between trading partners, thefull value of their implementations will come when theyalign their internal processes with the PIPs as well. Thismakes it all the more imperative to have a tool withwhich to evaluate internal readiness for making the shiftto standards-based e-Business.

IMPLEMENTATION READINESSMODELImplementing a business-to-business (B2B) messageexchange environment such as RosettaNet∗ has turnedout to be much more of an enterprise readiness effortthan initially anticipated. Team composition (size,diversity), technical and business knowledgecoalescence, communication channels, at-largeevangelism, cross-enterprise visibility, and establishingappropriate resource levels are just some of thechallenges.

As an early adopter, we of course experience more painthan those who will follow. More supply chainexperience, B2B gateway products, internalinfrastructure, pilot projects, internal experience, andstandards maturation will all ease the way. However,knowing where to look for “readiness” (or lack thereof)is critical to putting together a workable implementation




plan. To that end, our Implementation Readiness Modelhas identified four primary and six secondary readinesstracks to date.

The primary readiness tracks are as follows:

1. business strategy

2. B2B infrastructure

3. business process

4. application development

The secondary readiness tracks are as follows:

1. B2B external initiative

2. trading partner

3. solution provider

4. legal

5. security

6. audit

These tracks were identified as Intel went through thefollowing process:

• early pilot (proof of concept) initiated by our Salesand Marketing Group (Internet Marketing and e-Commerce organization) and one trading partner(completed in August 1998)

• engagement with Intel IT (e-Business Integration)to provide necessary infrastructure to ramp intoproduction mode

• involvement of Intel Planning and Logistics Groupto rationalize current business processes and ERP(Enterprise Resource Planning) systems withRosettaNet processes

• cross-enterprise collaboration to get to 2.2.2000

Our next steps will be to drive a significant increase inparticipation by business process and applicationdevelopment groups in order to further deployRosettaNet.

We are using this Readiness Model to help us get there.

BUSINESS STRATEGY READINESSThe purpose of this track is to assess the maturity of anenterprise’s B2B strategy at large. This is importantbecause B2B solutions are currently strategicallydivided between browser-based (user-interface) on-demand applications and automated service applicationsthat do not require user interfaces. RosettaNetimplementers are primarily focused on trading partnerautomation to either rehost their Electronic Data

Interchange (EDI) processes or reduce the need forbrowser-based applications. At present, little or noattention is being focused on feeding a B2B gatewaywith RosettaNet messages generated by browser-basedapplication backend processes. Implementers shouldrecognize that over time, as more trading partnerbusiness processes are automated, RosettaNet wouldreduce the need for on-demand B2B applications thatprovide a user interface. Therefore, many currentbrowser-based B2B projects, funding initiatives, androadmaps need to be re-evaluated to see whether an endof life timeframe exists. Key criteria in this trackinclude a company’s B2B strategy and a company’sbuy-side and sell-side motivations.

Intel, like many other companies, is using the Internet asa means to improve and simplify processes and serviceswith its trading partners. Due to RosettaNet and similarinitiatives, B2B solutions are evolving from tradingpartner portals or point applications requiring userinteraction to automated solutions. This B2Bautomation evolution is enabling a shift from “engagecustomer eyeballs” to “customer at work,” allowingcustomers (indeed, all trading partners) to use their owninternal solutions while having immediate access to allinformation within their global enterprise. In otherwords, by enabling RosettaNet, companies should beable to reduce overall data entry and data interpretationcosts. Automated data exchange and processes providefor higher quality data and faster processing time, aswell as create the possibility of an event-driven globalenterprise.

The motivations to implement RosettaNet may begreater within a company’s buy-side or sell-side.However, implementing RosettaNet ultimately needs toencompass both the buy-side and sell-side of anenterprise’s at-large B2B strategy. Implementers needto be sure to identify benefits by looking at the entiresupply chain; that is, their customers’ customersthrough their suppliers’ suppliers. (This level of impactupon one’s strategy will depend greatly upon how muchof a company’s purchased materials and finishedproducts fall within the RosettaNet consortium scope ofcoverage.) Implementers should identify their othersupply chains on both their sell and buy sides, and theyshould investigate how other B2B initiatives for tradingpartner automation are evolving.

The push to implement RosettaNet currently appears tobe driven more by buyers in an attempt to simplify andimprove productivity and margins. This may bebecause buyers naturally tend to engage suppliers withwhom they can work more easily. However, as B2Bautomation spreads, we should see suppliers using theirproven benefits to persuade their non-automated



customers to participate in automated services. Forexample, a seller should be able to provide its customerswith improved pricing and availability when itscustomers provide real-time demand and inventory/salesout reporting instead of infrequent non-automatedinputs. (Now, imagine the gains if an entire supplychain were to automate its demand and inventory/salesout reporting from end to end—this is one ofRosettaNet’s objectives).

A company’s B2B strategy also needs to take intoaccount the integration of mergers and acquisitions.Another of RosettaNet’s benefits would be improvedflexibility and agility as companies grow their corebusiness by enabling a standard message exchangeframework.

Finally, a company’s B2B strategy needs to recognizethe full potential of RosettaNet. Through the use of aself-describing message structure that includes a supply-chain dictionary-driven schema and meta-model formore than 100 business processes, RosettaNet suppliesa strategic benefit. This message structure holds thepotential of becoming a de facto message exchangestandard in the near future as agent-to-service andservice-to-service architectures evolve. The RosettaNetmessage structure is in fact sufficiently rich that it couldbe said to be a document database; RosettaNetmessages could be used as disconnected documentspassed between applications and databases within adisparate and distributed architecture.

B2B Infrastructure ReadinessThe purpose of this track is to define the infrastructureand to assess the level of effort needed to achieve it.This is important because becoming RosettaNet-compliant is only a small portion of the big picture.Although RosettaNet specifies a message structure, amessage dictionary, a message exchange framework,and a message exchange protocol, it does not specifythe infrastructure needed nor the backend processesrequired to receive, process, or send messages.Infrastructure is individually managed by each tradingpartner. Key criteria in this track include infrastructurecomponents, e-Business standards and guidelines, andB2B gateway capabilities.

B2B message integration involves both public andprivate aspects. Receiving, unpacking, and routing amessage, or assembling and sending a message (thepublic part) is relatively easy. The private (and moredifficult) part of message integration includes processautomation, workflow, and application integration thatlink into enterprise applications. In other words, theprivate part is the intra-enterprise application integration

portion of enterprise application integration (EAI),while RosettaNet is the public trading partnerapplication integration part of EAI.

RosettaNet is targeted for use within e-Businessapplications, predominantly B2B service-to-serviceapplications; however, much of the infrastructureneeded to support this is the same as required for B2C(business-to-consumer) and B2B browser-basedapplications. Many infrastructure components need toexist in order to proceed. Major infrastructure elementsinclude an e-Business “landing zone,” facilities,firewalls, proxy servers, networks, routers,communication services, and web servers.

A B2B gateway is needed. It must provide for inboundmessage receipt, authentication, authorization,entitlement, logging, and routing to a processautomation workflow tool. This gateway also mustsupport outbound message construction, packaging andlogging. The B2B gateway must be able to provideRosettaNet-compliant messaging and also should becapable of supporting other B2B specifications, perhapseven EDI, file transfer protocol (FTP) and simple mailtransfer protocol (SMTP) transport protocols. Thebuild vs. buy study needs to be completed, whilelooking at maturing product offerings from solutionproviders.

RosettaNet provides for two basic types of messages: atransaction process message (Figure 3); and asubscription model message (Figure 4).

TradingPartner

A

TradingPartner

B

Non-Substantive Acknowledgement

Initial Push (e.g., Request Order)

Push Substantive Acknowledgement(e.g., PO acknowledge)

Non-Substantive Acknowledgement

Step #1

Step #2

Step #n

Figure 3: Transaction process model

In order to implement subscriptions, a collection ofdocument repository, subscription, notification, andpublication services needs to be provided. This ispotentially a very large effort, and again, a few solutionproviders are working in this space.



PublicationServ ice

Notif icationServ ice

Subscrip tionServ ice

DocumentReposito ry

PartnerB2B

Server

Docum entsAdded,

UpdatedDeleted

Docum entchangeevent

Notificationof

ava ilab ility

M anageSubscrip tion

RequestDocum ent

SendDocum ent

Acknow ledgeSubscrip tion

GetDocum ent

3

45

6

8

7Validate

subscrip tion

1

2

Figure 4: Subscription model

The major services within a B2B gateway include thefollowing:

1. a trading partner database for a directory of tradingpartners, trading partner processes, and processparameters and their entitlements

2. non-repudiation (legal proof) archiving of messageorigin and content

3. public key infrastructure (PKI) repository of digitalcertificates and signatures for encryption,authorization, and authentication

4. PIP templates for integration to public and privateprocess/workflow automation processes

5. virus detection capabilities for message attachments

The B2B gateway will likely coincide or integrate withexisting gateways for FTP, value added network(VAN/EDI), and SMTP. Each of these gateways shouldcomply with similar guidelines, designs, andimplementations of authorization, entitlement,authentication, privacy, confidential document, andlegal trade agreement practices.

RosettaNet is based upon the hypertext transferprotocol/secure (HTTP/HTTPS) protocol in anautomated service-to-service framework that does notneed visible or attended Web pages. Because HTTPS isneeded for security, internal corporate guidelines forPKI and secure socket layer (SSL) encryption must beestablished. These guidelines should be compatiblewith existing B2B browser-based implementations thatuse HTTP/HTTPS.

The B2B gateway will also need a set of complementaryservices, such as the following:

• Receipt and routing—a public processing area thatreceives, authenticates, validates entitlement,archives, and routes inbound messages

• Package and delivery—a public processing areathat packages, encrypts, validates entitlement,digitally signs, archives, and delivers outboundmessages

• Process automation and application integration—aprivate processing area that provides for processautomation and backend integration of inboundmessages and outbound messages

• Infrastructure for non-repudiation database (NRdb),trading partner database (TPdb) and PKI

• Notification services for e-mail, pager, etc.

• XML/HTML scraping—ability to extract data fromremote trading partner Web pages, in addition to orin lieu of data passed within RosettaNet messages

• Trading partner portal—a portal where tradingpartners can self-administer their RosettaNetprocesses and subscriptions

• Testing facilities—the ability for trading partners totest their RosettaNet messages against a test site;after self-testing, the B2B gateway would thenpromote the trading partner from “test” to“production” status, and thereby allow tradingpartners to control their production messagingprocesses

• Satellite capability—a “host” could provide its non-automated trading partners with a B2B satellitesolution, thereby acting as a hub, developing andsupporting a B2B application at its trading partnerfacilities

Figure 5 illustrates an integrated B2B gateway,complete with support for RosettaNet, other B2Binitiatives, and SMTP, FTP, and VAN/EDI.



Figure 5: An integrated B2B gateway

Infrastructure readiness can be a huge task. However, itis not necessary to do it all at once. It is possible toinstall a solution from a RosettaNet solution providerwithin a few weeks and be up and running for a smallimplementation. Performing a comprehensive review ofthird-party solutions and then choosing a solutionprovider could take several months. However, eventhen one has only just started on the journey to overallinfrastructure readiness.

Business Process ReadinessThe purpose of this track is to assess business andtechnical resources, legacy applications, and businessprocess repositories. This is important because eachRosettaNet PIP provides a mutually agreed supply-chain

view of key business processes. Key criteria in thistrack include understanding the RosettaNet messagestructure and process message sequencing, identifyingbusiness process architects, and defining new businessprocesses.

Intel is completing its initial B2B infrastructureplanning and design requirements gathering, whichcoincides with initial RosettaNet pilots. A key findingfrom our initial efforts is that a significant increase inparticipation by “business analysts” (one of two areas inwhich we had difficulty obtaining resources) is neededin order to forward engineer and plan for the anticipatedlevels of adoption.

We have also realized that process re-engineering forRosettaNet must be performed within the context of re-engineering enterprise at-large business processes that

ExternalAuthentication& Revocation

Self-ViewIndica-

tors

SelfServeAdmin

XML HTMLTrans-action

Processes

Subscrip-tions

FTPex-

changes

Notific-ation

Services

B2B S2S Partner Services

Apps InternalPKI TPdb NRdb

Load Balancing

B2BServer

#1

B2BServer

#2

B2BServer

#n

Primary B2B Gateway

SelfTest

Services

B2BPartner

Test

Notific-ation

Services

Load Balancing

B2BServer

#1

B2BServer

#2

B2BServer

#n

Disaster Recovery Site

B2BPartner

Test

EAI Bus & WAN

CertificateAuthority

Entitlement, Authorization, Authentication, Transformation, Mapping, Process Automation, Fault Tolerance, Routing

EAIRepos-itories

EAI Authorization,Authentication, Transformation,Mapping, Process Automation,Fault Tolerant, Messaging

External Firewall

B2BSatelliteSolution

Gateways for EDI, RosettaNet, other XML standards, secure FTP and SMTPManagement, Surveillance, Monitoring, Administration, Indicators



include requirements for trading partner integration.Therefore, our RosettaNet effort is considered anintegral part of our EAI initiative. RosettaNet istherefore one element of a strategy to become a real-time event-driven global enterprise. Process re-engineering is a huge task.

To appreciate why business process readiness is such abig task, we need to understand how constructing adistributed Internet application using a robust messagestructure with a rich meta-model impacts enterprisereadiness.

A RosettaNet message is intended to be predictable(open standards-based format), somewhat humanreadable, and portable between trading partners. Inorder to produce a widely supported and long-livedmessage format, the RosettaNet consortium agreed todefine a message structure incorporating a completedata and meta-data model common to the significantbusiness processes within the IT and EC supply chains.

A RosettaNet message consists of several nested XMLstructures and data structures, namely,

1. nested XML envelopes to define action,transaction, service, agent, message, transfer, andsecurity sections

2. XML message sections for preamble, header, andbody

3. attributes expressed using XML tags based on asupply chain dictionary

4. meta-data schema structures expressed using XMLdocument type definitions (DTDs) or XML schemaconsisting of attribute data type definitions, taghierarchy, cardinality (1:1, 1:n), permissible values,and parent/child dependencies

5. data as message content

Therefore, this message was deliberately designed as aself-contained, stateful and intelligent message,complete with data, persistent state information, and ameta-data model. Conceptually, it could be used topopulate an object class or produce a database structure.Moreover, it could be abstractly considered as asnapshot of a transactional sequence in a file-baseddatabase expressed using XML.

It is therefore important to recognize that a RosettaNetmessage contains more information than data alone. Itis a rich, fully stateful, self-describing package ofinformation.

A RosettaNet message does not include any implied,hard-coded positional, or delimited structures. On thecontrary, other formats for message and document

exchange (namely EDI and non-standard comma-separated values (CSV) or tab-delimited file formats)provide a lesser degree or no level of schema definition,data constraints, dictionary-driven taxonomies, andprocess state information.

The completeness of a RosettaNet message structureacross a supply chain (as defined in PIPs) requiressignificant forward engineering by trading partnerswithin the RosettaNet consortium. As a result, tradingpartners should expect to re-engineer their back-endsystems to become RosettaNet compliant. This mayinvolve creating processes that currently don’t existinternally or mapping processes that are currentlydifferent from RosettaNet processes.

Up-front business process architects need to participatein many activities:

1. RosettaNet PIP workshops to define each process,meta-model schema, dictionary, taxonomy,message sequencing, and run-time parameters (e.g.,wait times, retry duration, acknowledgements)

2. determining impact upon existing businessprocesses and existing applications

3. optimizing existing business processes byleveraging the capabilities provided by RosettaNetwithin the context of an at-large enterprise processre-engineering effort

4. determining new processes and data services

Application Development ReadinessThe purpose of this track is to prepare PIPimplementation development plans and roadmaps. Thisis important because this step represents how and whenexisting processes and systems will be modified androlled out to support RosettaNet. Key criteria in thistrack include statements of work, budgets, and plans.

As in the Business Process Readiness track, substantialparticipation by application development group(s) isnecessary to forward engineer and plan for theanticipated levels of adoption. Application developmentgroups realize they need to re-engineer processes forRosettaNet within the context of re-engineeringenterprise at-large business processes while at the sametime including requirements for trading partnerintegration.

Key deliverables for this track include work scope;identification of impacted systems; identification of keybusiness analysts and process architects; determinationof RosettaNet compatibility with existing processes;preparation of project budgets and schedules; setting ofrelease dates; provision of consolidated test



requirements; definition of necessary API components;and setting of incremental upgrade roadmaps.

This track is similar to most enterprise applicationdevelopment efforts and can use a variety ofdevelopment methodologies (e.g., traditional waterfall,rapid application development (RAD), etc). This track,more than any other, is likely to require the greatestamount of effort and resources. What’s important tounderstand is that this group is usually the last toparticipate in the RosettaNet implementation planningeffort, yet it has to be the first to implement the plan inorder for deployment to progress. Therefore, gettingup-front participation from the application developers ismandatory.

B2B EXTERNAL INITIATIVEREADINESSThe purpose of this track is to assess the completenessand usability of the work of the chosen B2B externalinitiative (in our case, RosettaNet). Specifications,policies, and architectures provided by the initiativemust be understood and evaluated against internalpolicies, procedures, guidelines, and strategies. This isimportant because implementing RosettaNet is not“only” a technology; it is part of a strategy that mustpermeate an enterprise’s trading partner integrationstrategy. Key criteria in this track include review ofconsortium supply chain, implementation framework,and process frameworks.

Each B2B initiative provides technical specificationsthat present the functional design and technicalframeworks for message structure, message transport,and/or message content. In the case of RosettaNet,many technical documents and specifications have beenwritten. For example, below is a collection ofguidelines and specifications that are necessary in ourimplementation of the “Manage Purchase Order” PIP(which covers submit, acknowledge, change, and cancelpurchase orders). This material addresses one ofapproximately 100 PIPs.

1. RosettaNet Implementation Framework v1.1

2. Manage Purchase Order Specification (3A4)

3. 3A4 Purchase Order Acceptance MessageGuideline

4. 3A4 Purchase Order Acceptance Guideline DTD

5. 3A4 Purchase Order Cancellation MessageGuideline

6. 3A4 Purchase Order Cancellation Guideline DTD

7. 3A4 Purchase Order Change Message Guideline

8. 3A4 Purchase Order Change Guideline DTD

9. 3A4 Purchase Order Request Message Guideline

10. 3A4 Purchase Order Request Guideline DTD

11. Preamble Part Message Guideline

12. Preamble Guideline DTD

13. Service Header Part Message Guideline

14. Service Header Guideline DTD

15. Acceptance Acknowledgement Message Guideline

16. Acceptance Acknowledgement Guideline DTD

17. Acceptance Acknowledgement Exception MessageGuideline

18. Acceptance Acknowledgement Exception GuidelineDTD

19. Receipt Acknowledgement Message Guideline

20. Receipt Acknowledgement Guideline DTD

21. Receipt Acknowledgement Exception MessageGuideline

22. Receipt Acknowledgement Exception GuidelineDTD

23. General Exception Guideline DTD

24. General Exception Message Guideline

A given consortium’s documentation is usually targetedto a specific supply chain or e-Business market segment.The consortium’s pervasiveness within its targetmarkets must be considered. Moreover, due to therelative youth of Internet e-Business, frameworks andspecifications may not be as complete or thorough asthey could be. Therefore, participation in and achievingtime-tested experience within the initiative enablestrading partners to more accurately assess theapplicability of the initiative to their businesses, as wellas providing a means for influencing the initiative suchthat it does deliver the needed benefits. Finally,adopting a B2B framework needs to include a review ofits compatibility with best known methods (BKMs)within one’s company.

Trading Partner ReadinessThe purpose of this track is to assess the readiness ofkey trading partners. This is important because onecannot implement RosettaNet without at least one andhopefully many trading partners ready to do so. Keycriteria in this track include selecting trading partners,



choosing processes, detailed integration, and achievingreliable results.

Each trading partner will need to provide a similar levelof effort. It will be several years until the B2B tradingpartner automation technologies have matured toprovide relatively inexpensive plug and play solutions;therefore, these next few years will only include tradingpartners who consider themselves early adopters.Trading partners must have the will and desire todeliberately re-engineer business processes based upona rapid schedule and evolving processes. They must beable to move quickly, often with ad hoc funding andscavenging for equipment and resources. Althoughmanagement commitment is essential to successfulimplementation of a RosettaNet-sized initiative, askunk-works and entrepreneurial mentality in the earlydays can be helpful.

Selecting a RosettaNet trading partner is currently easybecause only early adopters are playing; and, with alimited set of PIPs to choose from, it is easy to define aproject. A key expectation is that the use of RosettaNetspecifications will eliminate the currently high level ofup-front trading partner analysis needed to conduct e-Business. This may lead to a rush of trading partnerswishing to engage each other using RosettaNetprocesses (after initial successful implementations byearly adopters) before the PIPs have matured and PIPimplementation is a widely understood experience. Atpresent, early adopter trading partners spend significanteffort figuring out how to use the RosettaNetspecifications with one another. Once sufficientinfrastructure is in place, the full benefits of RosettaNetcan be realized as trading partners self-administer theirprocesses and subscriptions.

Currently, readiness must be planned with exact testingand production dates and known versions ofspecifications and guidelines. Legal issues need to benegotiated up front (see “Legal Readiness” below).Precise details of Global Trade Identification Number(GTIN), United Nations Standard Products and ServicesClassification (UN/SPSC), and Dun & Bradstreet-assigned unique corporate identifier (D-U-N-S∗ ) must bemanaged. Personalized trade parameters such as partnumber, product lines, and interpretations of timeouts,retry and acknowledgements need to be exactlydiscussed. Trading partner agreements (TPAs) need tobe signed. Current EDI processes with the tradingpartner may need to be changed. Digital certificates and


digital signatures will be needed. And, as always in anew venture, backup plans will be needed.

Solution Provider ReadinessThe purpose of this track is to assess the readiness ofyour selected B2B gateway solution provider. This isimportant because the tool you have selected may notprovide all the capabilities needed to implement a PIPwith trading partners. Key criteria in this track includereview of public and private PIP processes, review ofPIP templates, and concurrence of PIP interpretation.

Some solution providers provide only the plumbing toenable RosettaNet. When no PIP templates areprovided, the end user must provide all aspects of PIPimplementation. In these cases the tool is ignorant ofthe exact meaning of retry periods, duplicate messages,acknowledgements, failure to receive, and other processspecifics. These build-your-own solutions will requireinternal infrastructure for non-repudiation database(NRdb), trading partner database (TPdb) and PKI.

Other solution providers provide a robust framework forPIP implementation where the PIP template is quitecognizant of the PIP framework. PIP implementationwould be easier and faster using these tools; however,the tool must be sufficiently flexible should the PIPframework prove incomplete in any given tradingscenario. These all-encompassing solutions includeinfrastructure for NRdb, TPdb, and PKI.

Trading partners need to assess the capabilities of theirsolution provider(s). Some key questions include thefollowing. What level of compliance does the toolprovide for the implementation framework and processspecifications? When is beta and general availability?Has the tool been sufficiently stress-tested for a varietyof PIP scenarios? Does the tool provide diverse role-based control so different groups cannot access othergroups’ processes? Many other questions will be on theminds of individual trading partners.

Finally, RosettaNet is working on a Solution ProviderCertification program and certification standards, whichshould help RosettaNet implementors perform theirassessments more quickly and with greater assurance.

Legal ReadinessThe purpose of this track is to assess relevant legalissues. This is important because RosettaNet hasexpanded the capabilities of trading partner integrationbeyond the current terms and conditions found with EDIagreements; therefore, legal precedence has not yet beenestablished for RosettaNet interactions. Key criteria inthis track include trading partner agreements and earlyparticipation by legal counsel.



Performing RosettaNet message exchange with tradingpartners will require a trading partner agreement (TPA)between each pair of trading partners. These legalagreements need to be managed by each company’slegal counsel. TPAs currently exist for EDI; however, ageneralized RosettaNet TPA does not exist as of thiswriting (although creation of a model TPA is nowunderway). In addition, legal expertise for Internet-based e-Business using RosettaNet has not yet beenattained. Experience gained in RosettaNet pilotprograms will help legal counsel to understand thedifferences between RosettaNet and EDI and facilitatethe preparation of a comprehensive TPA.

Because RosettaNet will be enabling supply chainautomation across a lengthy chain of buyers(customers) and sellers (suppliers), the goal is to writethe TPA from a neutral perspective. Use of such aneutral TPA may be a challenge for many companies,whose organizational practices may have dictated thatthey prescribe different terms and conditions withintheir EDI TPAs depending upon their role as buyer orseller.

The list of legal concerns is being compiled as we moveforward. Although many issues have been identified,the full impact will likely not be comprehended until theinfrastructure is in place and more time is spent inunderstanding legal ramifications. To date, some ofthese issues are

• encryption export to controlled countries

• frequently changing e-Commerce and e-Businesslegislation

• strict privacy laws

• the potential for hundreds of trading partners withvarying capabilities

• restriction on use of confidential, proprietary, ortrade secret information

• constantly changing landscape of trading partners,processes, messages, documents

• personalized TPAs with specific and different run-time parameters

• self-administered processes and subscriptions

• proper use of digital certificate and signatures forthe accompanying document/message

• signed non-disclosure and confidentialityagreements

Security ReadinessThe purpose of this track is to assess the securityrequirements for encryption, authentication, andauthorization at both the network and the trading partnermessage exchange level. This is important becauseimplementing RosettaNet means that trading partnersystems penetrate their corporate external firewall andsecurity mechanisms. And undoubtedly, most data willneed to pass through the internal firewalls to coreenterprise applications. RosettaNet also will enabletrading partners of different types and privileges toexchange documents for many critical businessprocesses (e.g., purchase order, quotes, productinformation, pricing, availability, inventory, technicalspecifications, trade secret and confidential documents,CAD drawings, design specifications, etc.). Key criteriain this track include an understanding of corporatesecurity and document confidentiality policies; andencryption, authentication, and authorization.

Security needs to initially address the front-end and theback-end. Front-end security issues apply to firewalls,proxy servers, network routing and protecting thesystem from malicious attacks. Back-end securityissues apply to the controlled access to message contentto system users and intermediaries using a right-to-seeapproach. Unlike current point-to-point solutions wheredata handling is decentralized, a B2B gateway willprovide for a centralized flow of critical businessinformation; therefore, only users with the right to seespecific data should be entitled. Role-basedadministration of the B2B gateway should beconsidered.

Security readiness is also a significant challenge due tothe inherent solution complexities, need for managedrisk, and elevated concerns. The RosettaNetimplementation framework incorporates a public keyinfrastructure (PKI). Intel’s current RosettaNetimplementation is based on a single corporate guidelineusing multiple certificate authorities, digital certificates,and digital signatures. Intel also requires the use of128-bit encryption, which is greater than common usageand also is prohibited for export to controlled countries.Obtaining, understanding, and incorporating theseguidelines and technologies into the B2B gateway,although logically simple, has been technically difficultdue to the inherent complexity of PKI.

An important aspect of security is the ability toimmediately revoke the privileges of a trading partner,or of any of their processes or subscriptions. It is alsoimportant to be able to confirm that trading partners aresending messages as agreed. This includes being able todetect when a message was not correctly assembled and



transported according to the TPA in place between thetrading partners. This also includes the ability to detectwhether encryption, digital certificates and digitalsignatures were correctly used.

Audit ReadinessThe purpose of this track is to assess one’s readiness tobe audited by internal company officials. This readinessis important because RosettaNet trading enables theinteraction of critical business processes. Managers andexecutives should not be casual with their views ofimplementing RosettaNet. Key criteria in this trackinclude understanding the seriousness of globalelectronic trading, and preparing for audits.

Knowing that a B2B gateway will eventually transportand manage a majority of e-Commerce transactions ande-Content interactions with trading partners, it isimportant to design the B2B infrastructure up front towithstand frequent and diverse auditing.

By design, RosettaNet and the capabilities it enablesrepresent considerable risks to a company shouldsomething go wrong. Auditing is actually a good thing,as one should feel more assured that risks are undercontrol. Some of the risks identified include

• potential to be majority revenue channel• binding $M transactions• binding legal agreements• international trade with an easy global reach• rapidly changing trade and Internet laws• many government enforcement authorities• sensitive and confidential document/information

exchange• many micro projects with intangible ROI where

something will be unforeseen• potential for lost potential or mistakes• needs to be fault tolerant without data loss• many critical success factors• pivotal and timely information exchanges• potential for significant impact on internal systems• significant visibility and expectation levels• competitors waiting for your misstep!• centralized administration of enterprise processes

and data (need for role-based administration andmanagement using a limited right-to-see basis)

CHALLENGESAchieving a common language for e-Business offerschallenges in a number of areas, including (but notlimited to) the development of the specificationsthemselves; correctly identifying the internal barriers tosuccess and successfully overcoming them; and

planning to keep up with an ever-changing business,technical, and standards environment.

Some of the specific challenges we see ahead include

1. Internet Speed. RosettaNet∗ is caught up in thefrenzy of Internet time. As such, trading partnerautomation and XML messaging are very hottechnologies; the leaders in this race will likely reapthe greatest rewards. Most significantly, getting itdone faster, better, and cheaper will remain arequirement that cannot be understated. Manychallenges exist when trying to compress andaccelerate planning, funding, scheduling,evangelizing, designing, building, and testing,especially when considering the Readiness Modelpresented above.

2. Sustaining will (internally). Maintainingmomentum in the face of “short attention spans”seems to be a systemic symptom of today’s Internete-Business mentality. At an increasing rate,everyone seems to have less time to make informeddecisions. An increased level of risk-taking will benecessary to proceed; management needs to remaincommitted even when the inevitable mistakes aremade.

3. Sustaining will (externally). Early adopters ofRosettaNet will find it neither easy nor inexpensiveto initially embrace. Each supply-chain orendorsing adopter will face an inevitable debate ofwhether to continue or disengage. So far the will ofthe RosettaNet consortium is withstanding thesestresses and the key motives for moving forwardremain steadfast; however, further tests of will arelikely before RosettaNet’s adoption is widespread.

4. Obtaining resources. Planning in advance forresource needs is a challenge within any company;however, RosettaNet, like all e-Business initiatives,is driven by its constituents faster than anycompany could anticipate. Obtaining business andtechnical resources is a challenge; however,expanding to include sufficient forward-thinkingresources from business analysts, technicalanalysts, system architects, and applicationarchitects requires resource allocation. This can beachieved either by additional funding or bycancelling other planned projects. This can beespecially challenging if resources are being pulledfrom competing B2B initiatives.




5. Creating the implementation plan. Defining,planning, and estimating the scope of work toimplement which of the ~100 RosettaNet PIPsacross the enterprise at-large requires a diversegroup of resources and a PIP-centric approachrather than a business group approach.

6. Choosing a project management methodology.RosettaNet implementation needs to be executedusing a hybrid of rapid application development(RAD) project methodology. Determining amethodology could be challenging withincompanies that do not have a conscious process forselecting a methodology.

7. Finding an optimal team structure. Initialimplementations of RosettaNet require participationfrom diverse groups within an enterprise (exactcomposition depends heavily on the PIPs chosenfor implementation). Each PIP implementationbecomes a mini-project within the bigger context ofRosettaNet and B2B implementation. Maximizingteam productivity and effectiveness is essential,especially considering that B2B and e-Businessprojects need to proceed at Internet speed. It willbe challenging to form an optimal team structure,then clone it for the many PIP mini-projects.

8. Managing information overload. Implementingany enterprise-wide project (especially one whichhappens to affect the very way the enterpriseconducts its business) is hugely complex andinvolves a tremendous amount of informationassimilation. Implementing the same set ofspecifications across most of the members of anindustry magnifies the problem of synchronizedinformation assimilation enormously. Participantsin the implementation process must remain currentwith respect to RosettaNet specifications; each ofthe open standards on which RosettaNet is based(e.g., XML, SSL, HTTP); software and hardwaresolution options; internal company guidelines;requirements and functional specifications; testplans; meeting minutes; and other commonmaterials. Participants must also keep abreast ofsimilar materials from trading partners with whomthey are implementing the plan. Methods forassimilating and managing frequent knowledge andinformation change in the e-Business sphere aresadly lacking.

RESULTSOn a practical level, we have identified eight distinctroles within our B2B RosettaNet∗ deployment strategy.Table 1 lists these eight roles; it also shows the level ofparticipation of each of these players within thereadiness tracks discussed above. (As a point ofdeparture for readers, the staffing levels for each role aswe worked through to our 2.2.2000 deployment planswas as follows. One person each fulfilled roles 1through 5. Role 6 consisted (in our case) of one full-time person plus parts of numerous other folksparticipating in PIP workshops, for another full-timeequivalent. Multiple people participated for roles 7 and8, typically one person for specific groups of PIPs orcore applications. A total of 22 people participated for2.2.2000 -- 13 from IT and 9 from the business units)

Intel performed several key tactical steps to address thediverse issues within the Readiness Model.

First we assembled the Intel RosettaNet DeploymentTeam consisting of six people in roles 1 through 6 inTable 1. We were slow in getting participants for roles7 and 8 because these groups were extremely busy andup-front resource planning was required. In hindsightwe recommend engaging these business analysts andapplication development groups in the early stages ofRosettaNet planning.

Next, we engaged one trading partner (a majordistributor) as part of our RosettaNet proof-of-conceptpilot (August 1999) and initial implementation(2.2.2000). With our trading partner, we selectedPIP3A4 (“Order Management”). Each of us selectedour own solution provider and tools. This meant thatfour companies had to synchronize development andtest plans. Since we were all first implementers, gapsand changes in the RosettaNet ImplementationFramework and PIP guidelines needed to be ironed out.Infrastructure planning was a key focus from thebeginning. Engineers within the core environmentsupporting EDI and our e-Business engineering groupsworked together to integrate e-Business designrequirements with existing EDI requirements. Atpresent, we are creating a production environment thatsupports both EDI and RosettaNet running on WindowsNT*.

∗ Third-party brands and names are the property of theirrespective owners.* Other brands and names are the property of theirrespective owners.



After a few months of assessing RosettaNet readinessand formulating the Readiness Model, we prepared andsent a PIP assessment and business impact survey to allbusiness groups having a need for trading partnerautomation. We are now waiting for enterprise-wideresponses. These responses, and additional partnerreadiness discussions, will be reviewed and become thebasis for our post-2.2.2000 rollout.

The following recommendations are provided to assistwith first-time RosettaNet deployment:

• Look for a quick win: pick one strategic PIP withone partner. Plan for the process to take 2-4months. Assign 4 to 6 people.

• Engage the solution providers, letting them educateyou on B2B and partner integration architectures.Perhaps even contract with one of them to build a

limited production pilot. Defer committing to yourB2B vendor until a successful pilot is in production.

• Require the Business Manager and TechnicalManager to hold weekly meetings to reviewprogress and status.

• Incorporate the RosettaNet roadmap strategy withinthe company’s overall B2B strategy.

• Include other B2B channels within the scope of theB2B gateway (e.g., secure file transfer, SMTP,EDI).

• Consider the impact on existing browser-basedapplications and partner portal strategies.

Table 1 :Participation levels of key roles in readiness model tracks

Readiness Track

Role#

Description1

BizStrat

2Infra-

structure

3Biz

Process

4AppDev

5B2B

Initiative

6TradingPartner

7SolutionProvider

8Legal

9Security

10Audit

1 RosettaNet BusinessProgram Management

L M S S M S M M M M

2 RosettaNet TechnicalProgram Management

M L S S S S L L L L

3 PIP Management S M S S S S M M M M4 Pilot Management S S S S S L S S S M5 Application

IntegrationManagement

M S S L S S S M S S

6 RosettaNet Standards-Development

M M S M L M M M S N

7 Technical andBusiness Analysts,Business ProcessAnalysts

S M L S S S M M S S

8 Back-end ApplicationDevelopmentManagement

M M S S N M N N M S

Legend: L = Leader S= Significant Participation M = modest participation N = little to no participation

CONCLUSIONOur team continually expands its understanding of whatit takes to implement RosettaNet. As we complete asecond-phase pilot, plan for future implementations,design the infrastructure, and expand our circle ofinfluence, we foresee many new challenges. It isunclear when the rate of discovery of new issues andchallenges will diminish. It is likely not to be until

widespread trading partner/PIP implementation occursin several years.

ACKNOWLEDGMENTSSpecial thanks are due to our fellow members of Intel’senterprise-wide RosettaNet Deployment Team and tothe architects, strategists, and technologists working one-Business for their frequent business, strategic, andtechnical contributions to our understanding of issues



related to the implementation of RosettaNet and B2Btrading partner automation capabilities.

Intel RosettaNet Deployment Team Members: ColinEvans (SMG IM&E Director, eBusiness Architectureand Operations), Alan Court (SMG IM&E, RosettaNetProgram Manager); Andy Keates (SMG IM&ERosettaNet Pilot (e-Concert) Manager); Thomas Vlach(PLG, RosettaNet PIP Analyst).

IT Architects, Strategists and Technologists: Bert Cave(IT Engineering, EAI Program Manager), Ralph Nitta(IT e-Business Integration Manager), Ed Balthasar (ITStrategy and Technology).

REFERENCES [1] RosettaNet specifications

(www.rosettanet.org).

AUTHORS’ BIOGRAPHIESPatricia J. O’Sullivan is a staff architect in IT Strategy& Technology. Her current focus is on increasingproductivity in the use of information, currently asapplied in e-Business standards. As part of this focus,she leads the RosettaNet’s Implementation Frameworkteam. She holds a masters degree in library andinformation science. Her e-mail [email protected]

Don S. Whitecar, PE, is IT’s e-Business EAI/B2B andRosettaNet Program Manager. His goal is to provide arobust, scaleable, and secure infrastructure for B2Btrading partner automation. Don is also responsible forIntel’s technical implementation of RosettaNet, workingclosely with business groups, business analysts, andapplication architects. His e-mail is:[email protected].
