Presentations of the

14th International Conference and Exposition on

TESTINGComputerSoftware

Test Automation

June 16-19, 1997Washington, D.C. * USA

Ti-CHNiSCHEINFORMATIONSBIBLIOTHEK

In Cooperation with

American Society for Quality Control (ASQC)Software Division

Software Technology Support Center (STSC)

^^^^m'^^^B'^^^m'^^,^^!

TUESDAY, JUNE 17,1997

8:30 a.m. OPENING REMARKS

9:00 a.m. KEYNOTE Page 1

Test Automation: Coming of Age

Dorothy Graham, CEO Grove Consulting

10:00 a.m. BREAK AND EXHIBIT VIEWING

10:30 a.m.

MANAGEMENT I TRACK Page 37

Automation Opportunities in the Test Processes- Encouraging Signs For the Near Future of Testing Automation- A Process Framework For Applying Automation to Testing- Current Tool Shortcomings and New Tool Opportunities

Dwayne L. Knirk, Sr. Member Technical Staff, Sandia National Laboratories

AUTOMATION I TRACK Page 153

Automated Testing Works in the Trenches- Data-Driven Automated Testing and How Would It Benefit You- All Programs Should Not Be Tested Equally- Automated Test Case Demonstration

Treasure Diehl, Director of Quality Assurance, J. D. Edwards & Company

ISSUES I TRACK (DOUBLE SESSION) Page 247

The Year 2000 and Testing- How Do You Manage Testing to Minimize Your Business Risk?- What Tests Must You Run to Confirm that all is Well?- Strategies, Techniques, and Suggestions to Deal with the Y2K Problem

Boris Beizer, Consultant and Author

TUESDAY, JUNE 17,1997 (Cont)

10:30 a.m. (ConU

ADVANCED TOPICS 1 TRACK Page 363

The Ability of Non-Representative Tests to Predict Software Quality- Statistical Measures of Quality are Desirable, but Often not Possible Given

Constraints- Few Observed Failures do not Guarantee Low Probability of Failure- Ways to Guantify the Accuracy of Probability-of-Failure Estimates

Jeff Voas, Vice President, Reliable Software Technologies Corp.

11:30 a.m.

MANAGEMENT I TRACK Page 57

Test it Again - How Long?-Why Have Automated Testing When it Takes so Much Time to Set Up - TheReal Cost

- Explore The Practicalities of Using a Testing Tool from a User- Getting the buy in from the sponsors- Why People Outside the Project are Getting Interested

Give Bates, Testing Manager, National Westminster Bank pic

AUTOMATION 1 TRACK Page 163

Test Automation: Sharing the Responsibility for Quality- Guidelines for Test Automation- Automated Processes that Check the Quality of Software Throughout Its

Development Cycle- Quality Engineering Tools that Allow Developers to Participate in the Test

Development ProcessRaymond Norris, Software Quality Engineer, The MathWorks, Inc.

TUESDAY, JUNE 17,1997 (Cont.)

11:30 a.m. (ConU

ISSUES I TRACK (DOUBLE SESSION CONT.) Page 247

The Year 2000 and Testing

Boris Beizer

ADVANCED TOPICS I TRACK Page 371

Estimating Reliability During Non-Representative Testing- Reliability Models Need to Allow both Representative and Non-Representative

Testing- The Order Statistic Model Is Based upon Failure Rates of Individual Faults,

Rather than the Failure Rate of the Entire Program- Improved Data Collection Is Possible by Shifting the Focus from Testing to

DebuggingSteven Zeil, Associate Professor, Old Dominion University

12:30 P.m. LUNCHEON LUNCHEON SPEAKER Page 11

THE PEPSI CHALLENGE: HOW THE SYSTEM TESTS SAVEDTHE DAY (AND THE CASE)by Warren Reid, Managing Director,WSR Consulting Group, LLC

2:00 p.m.

MANAGEMENT H TRACK Page 69

From Problem Child to Responsible Adult- How to Recognize a Problem Child- The Essential Ingredients for a Successful Transformation- Learn from Our Mistakes

Ann Young, Test Manager, Telecom Ireland Software

TUESDAY, JUNE 17,1997 (Cont)

2:00 p.m. (Cont)

AUTOMATION H TRACK Page 177

Test Automation Snake Oil- Debunking the Simplistic Arguments Favoring GUI Regression Test Automation- Why Good Test Automation Is as Difficult to Create as Good Software- Keys to Managing a Successful Test Automation Effort

James Bach, Chief Engineer, ST Labs, Inc.

ISSUES n TRACK Page 265

Practical Approaches for Testing Non-Visible Software- Which Was the Starting Scenery?- Which Methodology Did We Use?- What Were the Results We Got with this Approach?

Hermann Kull, Professor, Hochschule fuer Technik Esslingen, Germany

ADVANCED TOPICS H TRACK Page 385

Model Based Testing: A Methodology for Improving Quality- Behavioral Models Are a Powerful Tool for Understanding a Systems Purpose- Models are Very Effective at Providing a Structured, Controlled Testing Strategy- The Model Based Testing Approach Saves Time and Lowers Costs

Larry Apfelbaum, General Manager, Teradyne Software & Systems Test

3:00 p.m.

MANAGEMENT H TRACK Page 79

Managing a Successful Transition from Manual to Automated Testing- Assembling an Effective Test Team- Leveraging Automation in Testing- Strategies for Success

Andrew L. Pollner, President, ALP International Corporation

TUESDAY, JUNE 17,1997 (Cont)

3:00 p.m. (Cont

AUTOMATION H TRACK Page 185

Is Test Automation Really Straightforward and Inexpensive?- Why Only a Small Percentage of Companies are Using

Automation- Use of Automated Test Case Generators- Experience Gained in Two Projects Using Automated Testing

Armin Beer, Siemens PSE, Test Support Center, Austria

ISSUES n TRACK Page 281

Testing with Action Words; Abandoning Record and Playback- Creating Comprehensive and Manageable Testware- Using Maintainable Navigation- A Single Approach for Automation of Many Types of Tests

Hans Buwalda, Senior Management Consultant, CMG Finance BV, The Netherlands

ADVANCED TOPICS H TRACK Page 395

A Programmable Architecture for Scalable Test Automation (PASTA)- An Introduction to the PASTA Model- How the PASTA Model Facilitates Scalability and Programmability- Preliminary Results of an Implementation of PASTA

John Buchenberger, Software Engineer, Broadband Technologies

4:30 - 6:00 p.m. EXHIBIT VTEWTNG AND RECEPTION

WEDNESDAY, JUNE 18,1997

9:00 a.m. GENERAL SESSION Page 19

The Year 200 Crisis

Peter de Jager, Speaker/Consultant, de jager & Co.

10:00 a.m. EXHIBIT VTEWING

10:30 A.M. GENERAL SESSION Page 23

2001: A Legal Odyssey

Warren Reid, managing Director, WSR Consulting Group

11:30 a.m. PANEL SESSION: Testing Issues Surrounding the Year 200

Peter de Jager, Warren Reid, Boris Beizer and Nicholas Zvegintzov

12:30 p.m. LUNCHEON

2:00 p.m.

MANAGEMENT HI TRACK Page 87

Helpful Techniques When Testing Extremely Large Systems- How to Create Documentation of Current Practices and to Plan for

Improvements with a Master Testing Strategy- Disseminate Improved Practices with Training and the Creation of an All

Volunteer Testing Improvement Networking Group- Develop Many Levels of Test Drivers to Allow Enough Isolation of the

Platforms as They Are Being Integrated TogetherClaire L. Lohr, President, Lohr Systems

WEDNESDAY, JUNE18,1997 (Cont)

2:00 p.m. (Cont)

AUTOMATION m TRACK Page 189

Test Automation at AIU: How We Did More with Less- What Is Automation All About?- Why Do Most Efforts Fail?- How to Overcome the Obstacles to Automation, and Lessons Learned

Mary Ann Dolan, Director, Quality Assurance, American International Underwriters

ISSUES DI TRACK Page 295

Testing a Five Terabyte Database Application to Determine Performance,Availability, and Reliability

- How to Approach the Task of Testing a Five Terabyte Database- Selection of Benchmarks and Realistic Scenarios Needed to Characterize

a Large Scale Database System- Designing a Database for Optimum Performance, Availability, and Reliability

Brian Hannon, Principal Systems Engineer, Enterprise Systems Engineering, DigitalEquipment Corporation

ADVANCED TOPICS HI TRACK Page 399

The Use of Run-Time Profiling for Year 2000 Testing- How Path Profiling Can Be Used to Determine Whether a Program Has a "Year

2000 Problem"- How Path Profiling Can Be Used to Identify Certain Kinds of "Date Seeds"- How to Apply These Techniques to Other Program-Testing Problems

Thomas Reps, Professor, Computer Sciences Department,University of Wisconsin

WEDNESDAY, JUNE18,1997 (CONT)

2:00 P.m. (Cont)

AUTOMATION m TRACK Page 189

Test Automation at AIU: How We Did More with Less- What Is Automation All About?- Why Do Most Efforts Fail?- How to Overcome the Obstacles to Automation, and Lessons Learned

Mary Ann Dolan, Director, Quality Assurance, American International Underwriters

ISSUES m TRACK Page 295

Testing a Five Terabyte Database Application to Determine Performance,Availability, and Reliability

- How to Approach the Task of Testing a Five Terabyte Database- Selection of Benchmarks and Realistic Scenarios Needed to Characterize

a Large Scale Database System- Designing a Database for Optimum Performance, Availability, and Reliability

Brian Hannon, Principal Systems Engineer, Enterprise Systems Engineering, DigitalEquipment Corporation

ADVANCED TOPICS HI TRACK Page 399

The Use of Run-Time Profiling for Year 2000 Testing- How Path Profiling Can Be Used to Determine Whether a Program Has a "Year

2000 Problem"- How Path Profiling Can Be Used to Identify Certain Kinds of "Date Seeds"- How to Apply These Techniques to Other Program-Testing Problems

Thomas Reps, Professor, Computer Sciences Department,University of Wisconsin

WEDNESDAY, JUNE 18,1997 (CONT.)

5:00 P.m. BIRDS-OF-A-FEATHER MEETINGS

THURSDAY, JUNE 19,1997

9:00 a.m. GENERAL SESSION: Page 31

Testing on the Web

Mamie Hutcheson, President, ProSys

10:30 a.m.

MANAGEMENT IV TRACK Page 107

Suitability Testing - Lessons Learned from Navy Operational Testing- Testing for Both Effectiveness and Suitability- Standard Measures Used to Pass Systems into the Field- Lessons Learned

Joe Saur, Software Engineer, PRC

AUTOMATION TV TRACK Page 233

How One Company Recognized and Implemented Automated Testing- How One Company Went from Manual Testing to Automation and Why- Criteria for Selection of Tool and the Process of Implementation- How to Determine Whether the Tool Was a Success and Lessons Learned

Jean L. Bums, Software Quality Analyst, Universal Instruments Corporation

THURSDAY, JUNE 19,1997 (CONT.)

10:30 A.M. (CONT.)

ISSUES IV TRACK Page 331

Lessons Learned: Making the Transition to Client/Server- A Testing Methodology and Effective Techniques for Client/Server Testing- The Differences Between Client/Server Testing and Other Testing- Experiences with Two Different Projects

Mary McDrath, CQA, CSTE, Partner, Upstream, Inc

ADVANCED TOPICS TV TRACK Page 429

The Challenges in Automated Testing of Multithreaded Programs- The Issues in Providing Automated Testing of Parallel Programs- How to Reduce the Number of Timing-Related Test Cases for Parallel Programs- How to Automate DU-Pair Testing of Multithreaded Programs

Cheer-Sun Yang, Graduate Student, University of Delaware

11:30 a.m.

MANAGEMENT IV TRACK Page 125

Testing Capability Maturity Model- The Makeup of the Testing Capability Maturity Model, It's Basis in the SEI

CMM, and It's Key Process Areas- Support Tools to Facilitate Use of the Model- How the Assessment was Performed, and the Results Obtained. How Your

Organization Can Use the TCMM to Survey Your Testing MaturityRodger Drabick, Director, ITI Inc.

THURSDAY, JUNE 19,1997 (CONT.)

11:30 a.m. fCONT.l

AUTOMATION IV TRACK Page 241

Test Automation in the Advanced Applications Group at Cabletron Systems- Dimesions of Test Automation Tools- Discussion of Four Test Automation Tools- Case Studies at Cabletron Systems

Todd Littlefield, Software Engineer, Cabletron Systems, Inc

ISSUES IV TRACK Page 351

Replacing Bruce Devisser

Component Testing for H.323 Applications- Test Harness/Automation Designs- Test Development Strategy for Rapid Internet Time Cycles- Results of Using the Design/Strategy on a Real Product

Patrick Mead, Test Manager, Intel Corporation

ADVANCED TOPICS TV TRACK Page 443

Automated Testing of Computed Tomography Scanners- Developing an Automated Testing Tool- Writing Test Plans as Automated Test Scripts- Limitations of Automation in Testing Complex Systems

Joe Glaeser, Sr. Engineer, Picker International

THURSDAY, JUNE 19,1997 (CONT.)

12:30 p.m.

LUNCHEON and WRAP-UP SESSION:Conference Reflections - And A Few Surprises Thrown In!

-What We Did-Where We've Been-Where We're Going

Boris Beizer, Consultant, Analysis, Inc.

BIOGRAPHIES Page 457