internet computing the next generation starts from now richard p. sinn

Internet Computing

The Next Generation Starts from Now

Richard P. Sinn

Agenda

• Introduction- Who am I, Style of Presentation- Internet/Intranet Technology

• Problems and Solutions

– Big Picture

– Client Side Processing: Java, JavaScript, Image Map, Code Generation

– Server Side Processing: CGI-BIN, Integrated Solution

– Bring Backend to the World (DB Access and Workstation Gateway)

• Security of the Internet/Intranet

• DBCS Processing - A World Wide Problem

• Integration and Build Problems

• Internet Agent

• Conclusion

Introduction• Web as a “top-level” protocol:

Source (Browser) Destination (Server)

• Using MINE (Multipurpose Internet Mail Extensions) to define rules for exchanging informationWeb encapsulate other protocols including FTP, Gopher, WAIS (Wide Area Information Server), NNTP (USENET News) and telnet, etc.

Web

FTP

TCP

IP

Web

FTP

TCP

IPPhy Net

Introduction

• Web as a powerful Business Tool

Web Server = Platform + Software + Information

- Information Can be accessed in different Platforms

- Local Network Ring, PC Lan, Mainframe and the World can link together

• Enhance the Re-engineering of business process

(e.g. Traditional Q&A process for new products, Support Line, Networked Marketing, etc)

• Collaborative Management and Communication(Intranet as the heart of business administration)

• Business Applications(Web link to legacy systems, data warehouse and data mining)

Introduction: Business Case

Server Support Area 12 Month Volume Volume Trend

Communication 1712 Unchanged

Hardware 1469 Unchanged

Language and Database 754 Unchanged

Remote Access 2336 Negative

Performance 322 Negative

Print 559 Unchanged

Small Products 179 Unchanged

Backup 582 Negative

Work Management 411 Negative

Other functions 247 Positive

Total 8571 Negative

Introduction: Business Case

• How can we get the volume of support questions down ?

• Can we market new products better using the internet ?

• Can we use the question and answer better (data warehouse, data mining) ?

• Can we use the internet as proactive tool instead of traditional reactive ?

• Will chat room help ?

• What is the cost and profit ?

Interactive New products Preview

• Lessons on new products

Interactive Support Wizards

• Knowlege Base with problem reports

• Software Troubleshooting Wizards

• Frequently Asked Questions

• Download help files and fix packs

• Newsgroups and Chat Room

• Email A Question

• Support Options and Phone Numbers

Introduction

• Web as a new source of problems• Performance Problem

• Availability on Different Platforms

• Scaleability

• Security

• DBCS Processing on the Web

• Business Case

• ISO 9000 Standard

The Big Picture

Servers- Files- Mail Service- Print- Directory (x.500, LDAP)

ServiceApplicationServer

Traditional ApplicationSystemMVS, VM, CICS, etc

Internet/Intranet

Web ServersHTML, JavaScriptJava Applet/Appl

Client 3270, 5250, TCP/IP

Socket

Directory, File, Printing, Mail

HTTP

SecurityAvailablilityDBCSIntegration

PerformanceSecurity, Maintance

AvailabilitySecurity

Performance, Available

Current/Future Server Architecture

Divide And Conquer

• Client Side Processing– Java

– JavaScript

– Browser Tricks

– Code Generation

• Server Side Processing– Server Integrated Solution

– HTML Gateway

– WWW DB2 Gateway, Java JDBC API

Client Side: Java

Advantage

• Platform Independent (Portable)

• Enhance Software Reuse

• “Hot” in the market

• Relative short Learning Curve

• OO Internet Programming

• Secure ??

• High-performance ??

• Dynamic (Load class when needed)

• Multithreaded

Disadvantage

• Do not have information about browsers (vs JavaScript)

• Not available to all platforms(Only 56% of OS is windows base)

• Firewall block java applet from running

• Multithreaded problems (Scheduling information needed)

Client Side: Java

<html><head><title>An Applet</title> </head>

<body>

<p><applet code="HelloApplet.class" WIDTH=300 HEIGHT=50>

<PARAM NAME=inputString VALUE="TESTING input string">

<PARAM NAME=colorString VALUE="cyan">

</applet>

<FORM METHOD=POST ACTION="list.cgi">

<P><PRE>

Link name: <INPUT NAME="name" SIZE="45">

Link URL: <INPUT NAME="url" SIZE="45">

</PRE>

<P>

This URL will be added to the <SELECT NAME="section">

<OPTION>home page

<OPTION>commercial

<OPTION>misc

</SELECT> section of the list.

<P><INPUT TYPE=submit VALUE="Add link"> <INPUT TYPE=reset>

</FORM>

</body>

</html>

AppletApplet

Button 1 Button 2

Client Side: JavaScript

• Derived from Java, From LiveScript to JavaScript by Sun 1995

• Integrated into HTML

• Supported by Sun, Netscape, MS

• The Java Glue(“Glue Java Applets into Web Pages” by using capabilities to trap user events and pass relevant information to Java applets)


Advantage

• Integrated with Browser- Have access to history list- Use cookies to remember

• Integrated with HTML

• Supported by Sun and Netscape

• Object based language ??

• Can be used in client and some server side

• Good for simple user interface

Disadvantage

• Slow performance (Interpreted Language)

• Not fully extensible (limited set of base object)

• No code hiding

• Lack of debugging and development tools

• Not good for Big Application


• Cookies provide a method to store information at the client side and have the browser provide that information to the server along with a page request

• Cookie Information is shared between the client browser and a server using fields in the HTTP header.

• When the user requests a page in the future, if a matching cookie is found, the browser sends a Cookie field to the server in a request header. The header will contain the information stored in that cookie.

Custom Search Tool

• User calls the site by using an URL that request a CGI script

• The script checks whether it is the user’s first time at the site by checking whether cookie exist in the http header.

• If no cookies, all choices unselected

• If cookies exist, previous choices selected

• If user perform a search, the search results along with a Set-Cookie field in the header to reset the new cookie


Last Name

First Name

Custom Search

Sinn

Richard

Custom Search

Client Side: JavaScript// Function to create or update a cookie.

function SetCookie (name, value) {

var argv = SetCookie.arguments;

var argc = SetCookie.arguments.length;

var expires = (argc > 2) ? argv[2] : null;

var path = (argc > 3) ? argv[3] : null;

var domain = (argc > 4) ? argv[4] : null;

var secure = (argc > 5) ? argv[5] : false;

document.cookie = name + "=" + escape (value) +

((expires == null) ? "" : ("; expires=" + expires.toGMTString())) +

((path == null) ? "" : ("; path=" + path)) +

((domain == null) ? "" : ("; domain=" + domain)) +

((secure == true) ? "; secure" : ""); }

// Function to delete a cookie. (Sets expiration date to current date/time)

function DeleteCookie (name) {

var exp = new Date();

exp.setTime (exp.getTime() - 1); // This cookie is history

var cval = GetCookie (name);

document.cookie = name + "=" + cval + "; expires=" + exp.toGMTString();

}

<!-- HIDE FROM OTHER BROWSERS

// Cookie Functions

// "Internal" function to return the decoded value of a cookie

function getCookieVal (offset) {

var endstr = document.cookie.indexOf (";", offset);

if (endstr == -1)

endstr = document.cookie.length;

return unescape(document.cookie.substring(offset, endstr));

}

// Function to return the value of the cookie specified by "name".

function GetCookie (name) {

var arg = name + "=";

var alen = arg.length;

var clen = document.cookie.length;

var i = 0;

while (i < clen) {

var j = i + alen;

if (document.cookie.substring(i, j) == arg)

return getCookieVal (j);

i = document.cookie.indexOf(" ", i) + 1;

if (i == 0) break; }

return null;}


• The Navigator Object

function checkBrowser(){

if ((navigator.appVersion.substring(0,6) != “2.0b6a”) && (navigator.appName != “Netscape”))

alert(“Please use Netscape with version 2.0b6a”);

}

• The History Object

The history list of a browser is accessible in JavaScript. For example:

history.back() goes to the previous page

history.go(-3) goes back to the page visited three pages ago (like clicking the back button three times)

• The form Object and document Object

Client Side: Java and JavaScript

• We enjoy limited interaction between applets and the browser environment, using JavaScript today.

• With JavaScript capability to dynamically generate HTML code, a form in one frame could easily reload a Java applet in another frame, with new parameters.

<HEAD>

<TITLE>Example 14.3</TITLE>

</HEAD>

<BODY BGCOLOR="#FFFFFF">

<H1>Growing Text Java Applet Tester</H1>

<FORM METHOD=POST>

Text to display: <INPUT TYPE=text NAME="text" SIZE=40><BR>

Delay between updates: <INPUT TYPE=text NAME="delay"><BR>

Font to use: <INPUT TYPE=text NAME="font" SIZE=40><BR>

<INPUT TYPE=checkbox NAME="bold"> Bold

<INPUT TYPE=checkbox NAME="blur"> Blur<BR>

<INPUT TYPE=button VALUE="Test Applet"

onClick="parent['applet'].location='applet.htm';"> </FORM>

</BODY>

</HTML>

Client Side: Java and JavaScript

<BODY>

<SCRIPT LANGUAGE="JavaScript">



</SCRIPT>

</BODY>

Text to display

Delay between display

Font to use

[] Bold [] Blur

Test Applet

Csci 8180 By Prof Tsai

250

Courier

Csci 8180 By Prof Tsai

Client Side: Java and Javascript

• JavaScript should work with CGI-BIN and Java.

• Applet object should be added to enhanced interaction with Java.

• database object should be added to allow quick access to local DB systems.

• Using netscape product like LiveWire, CGI-BIN can be written with JavaScript.

• Make use of the current plug in to allow JavaScript to work with a range of file formats (Acrobat Amber Reader; ASAP WebShow, Corel Vector Graphics, EarthTime, VRML viewer, VR Scout, etc)

Client Side: Image Map

• Traditional server side image maps involves extra communication with the server.

<IMG SRC = "clientimage.gif" USEMAP = "#clientimage.map">

<MAP NAME = "clientimage.map">

<AREA SHAPE = "RECT" COORDS = "0, 0, 109, 74" HREF = "#T1">

<AREA SHAPE = "RECT" COORDS = "110, 0, 218, 74" HREF = "#T9">

</MAP>

• Spyglass Mosaic, Netscape, MS-IE do client side image map differently

• With browsers do not have build-in image map, a combination of image maps might be needed.

Go to Top Page Go to Title T9

Client Side: Code Generation

• With different platforms of databases exist out in the fields, an efficient way of extracting information for data warehouse and data mining is needed.

• As database exist in different platforms, program automation is needed to ensure consistence.

• The concept of Netobj is born. As with any object, a Network Object contains both data and operations on the data. The object itself is actually logically exists in the network. That means that a Network Object can "reside on" and "be accessed by" any system (VM, MVS, AIX, OS/2, and AS/400) connected to the network.

• The goal for a collection of Network Objects is to provide a "logical" data warehouse for a business enterprise. This warehouse will contain both data and information necessary for running a business.

Client Side: NetObj

• Code Generation

• Fourth generation language

• The following type of files are generated:- DDS Files (Data definition files)- IDL File (Definition files for rpc)- Client Code (Stub of connect)- Server Code (Stub listening)- Procedure Code (Code contain SQL statement, does operations)

• C with imbedded SQL

• Java with JDBC

DB 1

DB 2

Server Code 1

Server Code 2

Client 1 Client 2

Client Side: WinObj

• Pushing the concept of NetObj all the way to the client side, we have WinObj

• WinObj are user interface that reside on any platform on the network (Unix, Browser, VM, MVS)

• It provides consistent user interface to data and information (represented by Network Objects).

• It will link with Network Objects, and access common data and information from any system.

• It can generated Visual C++, REXX for Mainframe, C for UNIX, Java and HTML for browsers

• Remember we still have 44% of non-windows clients.

User Interface 1 User Interface 2

Client 1 from Netobj Client 2 from Netobj

Server Side Processing

• We needed an Integrated Server Solution Instead of Different Machines will all kind of different information

• Single Sign-on for Authentication to Internet/Intranet, Legacy Application, x500, LDAP, NetWare, NT and any other LAN application.

• Internet/Intranet File Serving with choices of system (NetWare, AS/400, Unix, etc).

• All Information from LAN (NetWare, NT, Lanserver) available for Internet/Intranet usage.

• All Information from Legacy Application available for Internet/Intranet usage.

• CGI-BIN has access to Database, Application Data and all the information from the LAN.

Server Side Processing

Integrated Solution From the Server Side

Servers- Files- Mail Service- Print- Directory (x.500, LDAP)

ServiceApplicationServer

Traditional ApplicationSystemMVS, VM, CICS, etc

Internet/IntranetWeb ServersHTML, JavaScriptJava Applet/Appl

Clients

HTMLGateway

NetWare “Gateway”

NT “Gateway”

Backend of Server: Database

• A Web Server gateway is needed to allow the access of relational database management system

• An easy-to-use Web page paradigm for Web application development: no complex programming is required.

• Ability to work from any Web client (browsers)

• JDBC might be one of the solution(But what happen to all the non-java platforms ?)

• Picking the common denominator, a CGI-BIN implementation should be used

WWW Database Connection

• One example to make your database information available through the internet.


• A Web server gateway can be build to access database. An application programmer writes macros, which are stored on the Web Server, letting customers query databases using HTML forms. The result of the query are displayed on the browsers.


• Authentication- Using the web server to control access to directories.- Same pwd and userid can be used to authenticate to database.

• Encryption- With SSL (Secured Sockets Layer) or SHTTP.

• Firewall- Web Database Gateway can be protected under a firewall as well.


• For most transactions you will find it useful to call an input section before making the query.

• The macro can be called using<A href="http://www.ibm.com/cgi-bin/db2www/equiplst.d2w/input">

List of hardware</A>

• Macro Section:%DEFINE DATABASE="MNS95"

%HTML_INPUT{

<H1>Hardware Query Form</H1>

<FORM METHOD="POST" ACTION="/cgi-bin/db2www/equiplst.d2w/report">

<dt>What hardware do you want to list?

<dd><input type="radio" name="hdware" value="MON" checked>Monitors

<dd><input type="radio" name="hdware" value="PNT">Pointing devices

<dd><input type="radio" name="hdware" value="PRT">Printers

<dd><input type="radio" name="hdware" value="SCN">Scanners

</dl>

<input type=submit value=Submit> </FORM> %}

%SQL{

SELECT MODNO, COST, DESCRIP FROM DB2USER.EQPTABLE

WHERE TYPE=$(hdware)

%SQL_REPORT{

<B>Here is the list you requested:</B>

<TABLE>

<TR>

<TD>$(N1)</TD>

<TD>$(N2)</TD>

<TD>$(N3)</TD>

%ROW{

<TR>

<TD>$(V1)</TD>

<TD>$(V2)</TD>

<TD>$(V3)</TD>

%}

</TABLE>

%}

%}

%HTML_REPORT{

%EXEC_SQL

%}


• Macro file format uses cross-language variable substitution (Code Generation)

• A JDBC Runtime can be provided to add advance functions

• DB2 World Wide Web Connection is an example products

• Access to other kind of database might be needed (Sybase, Access, Oracle, etc)

Hardware Query Form

What hardware do you want to list ?

[+] Monitors [] Pointing devices [] Printers [] Scanners

SubmitSubmit

Put All Existing Applications on the Web !!!

The Workstation Gateway• A lot of applications still runs under text-based workstations and emulators

• Open all the applications to the internet market

• Port to internet without re-coding

• Port to internet even without having to run conversion program

• Use existing tools to do development for the web

• No need to retrain all programmers

• Traditional applications sent out in a 5250 data stream to the workstation, which then displays the text.

• Workstation Gateway intercepts the 5250 data stream and converts it to HTML, which any WWW browser can display

Workstation Gateway

• A real example from IBM … Any PC that has a Web browser installed can run AS/400 applications !!!

Workstation Gateway

After the 5250 connection from the Web browser to AS/400 is made. A set of events occurs:

• The request is received and processed

• The user job processes the application

• The workstation I/O is converted from 5250 to HTML

• The HyperText Markup Language (HTML) code is sent to the browser

• The connection is ended

The processes involved in establishing, processing, and ending this connection are:

• The Workstation gateway server jobs

• The user job (one per client)

• The communications router tasks

• The virtual terminal task

• The Telnet task

Workstation Gateway

• Instant WWW access for all existing AS/400 applications without re-coding

• Use of existing development tools to develop for the internet

• Use Web browsers to run AS/400 applications

• Add graphics to AS/400 applications without disrupting workstation users

Client and Server Processing

• Recap …

• Client Side Processing usually being used as performance solution.

• JavaScript and Java as well as CGI-BIN should work together to produce the best results.

• Different Platforms still exist without Java or JavaScript. Code generation can be used.

• A lot of opportunities still exists for research topics.

• Client Side Processing has to work together with Server Side in order to produce the best possible business results.

Security on the Internet

• Do you know an internet a carrier has the ability to "take a peek" at your private information ?

• The following technology is available for providing limited security:

– Security ProtocolsMostly developed by commercial sites

– Methods of Securing CommunicationsRestrict access to your computer OR encrypt your communications

– Secure Applications and Netscape’s Broken SecurityLet’s look at how people break Netscape security

– Surveillance on the InternetThe government steps in …

Security Issues

Secure Transfer Protocols

• SSLSecure Sockets Layer from Netscape. It provides low-level security for application protocols such as NNTP and HTTP

• SHTTPDeveloped by NCSA and RSA. Add message-based security to HTML. Can be used with SSL.

• Both under development

Securing Communications• Firewalls

Limit the internet traffic from the rest of the world (More later)

• Public Key CryptographyAnyone may encode a message and send it to you using the public key, but only you can decode and read a message with your private key (Electronic signatures and certification)

• RSATwo prime numbers multiplied together as key

• Anonymous RemailersAnonymous remailers forward your mail anonymously to their destinations (FBI can find it anyway)

Security Issues

Secure Applications

• Application suppose to provide security functions

• July 14, 1994. The key of Netscape message is broken by 120 and two parallel supercomputers in eight days. (Netscape is using 40bit instead of 128bit encode method.)

• To deduce any key in about 25 second. Only a few seeds were used to generate random number by the system. (Increase the number of seeds.)

Surveillance on the Internet

• Clipper Chip (Skipjack algo by National Security Agency)

• Government access to the decryption key for ‘“tapping” of internet traffic

• Encryption algo restricted only to the government

• It is voluntary for now

• May enforce later ???

Security: Firewall Technology

• A firewall is a control point between the private inner network and the untrusted outside network

• The basic building-block are

– Packet filtering router (normally a piece of hardware)

– Circuit Gateways (Two software modules)

– Proxy software (Aka application gateways)

• Common configurations are “Dual Homed Gateway”, “Screened Host” and “Screened Subnet”.

Security:Packet filtering routers

Router

OK

Internet(Outside of Firewall)

No Denied

Flow of traffice

Physical Network

Send me to141.131.22

Private networks(inside firewall

141.131.44.1

Send me to141.131.44.1

141.131.22

Security:Packet filtering routers

• Router acts as network traffic cop

• Read the packet header with source network address, destination network address, and the port of the source and destination address

• Enhanced to selectively pass on packets according to rules

• Permit or denied a set of network addresses and ports

• Some protocol requires that a call be made from outside to inside the firewall to set up connection (I.e. the outside party must callback)

• FTP and X Windows are examples

• With FTP, no way to know remote port in advance

• Complex to setup, test and maintain

Security: Circuit Gateways

Flow of traffice

Physical Network



PortP1

PortP2

PortP3

PortP4

Applicationand CircuitGatewaySoftware

Internet Server

Client Inside Firewall

No, denied; talk tothe circuit gateway


Gateway

Relay

Security: Circuit Gateways

• Computers sit between two networks (Internet and private net)

• Software module on the client computer on the inner net

• Software module (relay) on the gateway machine

• Connection on P1 to P2, P3 to P4

• Packets management by copying data packets on P2 and P3

• “Dual-homed gateway”

• May be a bottom neck if not management correctly

• Still FTP will not work as there is no network callback support

Security: Application Gateways



PortP1

PortP4

TelnetApplication

T

Internet Server(weeble.other.com)Client Inside Firewall

(daisy.any.com)Flow of traffice

Physical Network

PortP2Port

P2PortP3



TelnetProxy

TP

Firewall Components

PacketFilteringRouterR

TelnetServer

TS

firewall.any.com

Security: Application Gateways

• Aka Proxies

• Application protocol can be understand

• Special version of server software and understands a subset of client programs commands

• Client has to telnet to the gateway

• Data packets copied between P2 and P3

• Router added to ensure only packet exchange between TP and Internet

• Packet only send from outside to P3

• Router only accept incoming packet from P3

• FTP and others will work

• “Screened host firewall”

Security: Screened subnet

PacketFiltering Router

R1

Gateway

Web Server

Screened Subnet

PacketFiltering Router

R2PrivateInner

Subnet

Web Client

OtherClients

Security:Screened subnet firewall

• “Screened subnet firewall”

• Good choice to offer public service (web service)

• R1 blocks internet traffic from accessing info on private inner subnet

• R2 gives an extra level of protection to the private inner subnet by blocking all network traffice from the Web server

• One of the best choice today

WWW DBCS Processing

• Multiple DBCS problems exists when we deal with client/server technology today

• Different fix pack level

• Lack of design

• Lack of education

• Lack of testing

• Important problem facing the software industry

Database

Server Application Code

Server Base OS

Network Code

Client Base OS

Client Application Code

DBCS: International Standard

• W3C - an industry consortium

• Promote standards for the evolution of the web

• Interoperability between WWW products by producing specifications and reference software

• Members: Apple, AT&T, IBM, Microsoft, Netscape, Novell, etc

DBCS: HTTP Content

HTTP Content Negotiation

• Simple request-response architecture with content negotiation

Example:Client GET /japanese.html HTTP/1.1

Accept-Language:ja, en;q=0.5

Accept-Charset: x-sjis, x-eu c-jp;q=0.5

Accept: */*

Server 200 OK

Content-Type: text/html; charset=x-sjis

Content-Language: ja

Content-Length: 1042

… data ...

• If Accept-Language is missing, server assumes any language is acceptable (Problem ?)

• If Accept-Charset is missing, server assumes any coded character set is acceptable

• Content-Language can contain more than one language

DBCS: Bidi Problem

HTML Bidi Language

• Problem:

E.g. <B> … RLE … </B> … PDF

where RLE is ‫ (U+202B) and

PDF is ‬ (U+202C)

• Solution:- Use the DIR attribute instead of RLE/LRE

and PDF

- E.g. <P DIR=RTL> … <Q DIR=LTR>

… </Q> … </P>

• The DIR attribute is used to indicate the directionality of the text order

– Normally used to indicate direction in the context of bidi text

– LTR (left-to-right) orRTL (right-to-left)

– Equivalent to U+202A Left-to-Right (LRE)U+202B Right-to-Left (RLE)

– Eng tagU+202C Pop directional formatting (PDF)

DBCS: Coded Character Set

Coded Character Set Problem from document source

• Problem:E.g. <HEAD>

<META HTTP-EQUIV=“Content-Type” CONTENT=“text/html CHARSET=ISO-8859-4”>

<TITLE> … </TITLE>

Chicken-and-egg problem: may not work unless

the META tag is encoded in US-ASCII

• Problem:E.g. <A HREF=…CHARSET=“UTF-8”> .. </A>

May fail if the linked document’s CHARSET is changed

HTML Forms Problem• The server indicates the

CHARSETs it can accept via the ACCEPT-CHARSET attribute of the INPUT and TEXTAREA elements

E.g. <INPUT Accept-Charset=“x-sjis, x-euc-jp” .>

• Interpreted as either x-jis or x-euc-jp, not both !

• User agent is responsible in letting the user know which coded character sets are acceptable

Integration and Build

• Complex Problem when doing integration for 1000+ components with 7 different platforms

• 150+ PY working on development and build tools

• OO Build Framework can be used

• Source Code Configuration managment tools are used

• HTML/CGI-BIN/JAVA can be build by the framework as well

Web Files

CGI-BIN Program

Server Code

Client Code

1000+ OtherComponents

DifferentRelease

(Version)

New Code

…………..

Release Driver 1 Driver N

Development Environment

Integration and Build

Build Framework A Web Build Tool in the Build Framework

• Macros:E.g.:

define(`BULLET', <img align=top src="/afs/rchland.ibm.com/usr8/sinn/WWW/images/bullet_blue.gif"alt="">)

<HTML>

BULLET() List one

</HTML>

• Compile like a C or Java programs

• Other usage: Code generation, CGI-BIN, Different Language

Build Tools Appications(OS/2, NT, Win31, AIX, AS/400)

GUI

CMVC DB

File System (UNIX, PC)

Communication (TCP/IP NetBios)

Internet Agent

• An interesting Idea

• What are Agents ?Agents can be considered personal software assistants with authority delegated from their users

• Human software entities that can perform a variety of tasks for their human master (e.g. Desktop agents, Software Agent as component - Steven Ketchpel 1994, COACH Agent - Lisp learning agent 1994, Learning Agent - CMU 1994)

• What about on the internet ?BargainFinder agent - Andersen Consulting 1995

• WebCrawler - resource discovery tool for the WWW that provides a fast way of finding resource by maintaining an index of specific topics

• Lycos is considered by some as an agent program with “best-first-search”

Internet Agent

The Web Maintenance Problem

• How can we pick up all the dead link on a site with 10GB content and over 200,000 files + database ?

• Past Approaches

• Server Log AnalysisURL of return code (301 Moved Permanently)URL of Web pages that refer these links

• Manual Traversal

Internet Agent

Web Maintenance Spiders - WebWalker

• Look up task description from task file

• Can run at a CGI-BIN script– A Web Robot Must Show ID

– Must obey exclusion standard

– Must not hog resources

– Must report errors

• LimitationNot recommend to run on remote siteCan not run in parallel computerResource not share with multiple users

E.g Webwalker [-h] [-f taskfile] [-d

maxdepth]

Output: Broken Links:

http://www.yahoo.com/Society_and_Culture/

People/ (603 Time out)

Changed Links

http://www.yahoo.com/Regional/Countries/

(200 OK)

…

Summary

Traversed xxx xxx xxx

Tested xxx xxx xxx

Redirected xxx xxx xxx

...

Conclusion

• Still a lot of rooms in research and new product ideas

• We are at the cutting edge

• The Next Generation Starts Now !!!

THANK YOUFOR YOUR

ATTENTION !

internet computing the next generation starts from now richard p. sinn

Documents