1 © 2004 jeffrey feldstein. all rights reserved. drive testing for web-based and java applications...
TRANSCRIPT
![Page 1: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/1.jpg)
1© 2004 Jeffrey Feldstein. All rights reserved.
Drive Testing for Web-Based and Java Applications with Models
Jeffrey Feldstein
Cisco Systems
Source Code: http://www.sherpas.com/mbt
![Page 2: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/2.jpg)
222© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.com
222© 2004 Cisco Systems, Inc. All rights reserved.Presentation_ID
![Page 3: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/3.jpg)
333© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.com
• Introduce myself
• Model Based Testing Introduction
• Obstacles to Model Based Testing
• Introduction to IBM Functional Tester
• Model Based Test demonstration
• Code walk-through
• How to obtain and run the source
• Next Steps
Agenda
![Page 4: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/4.jpg)
444© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comIntroduce Myself
• Software Career: 24 years
• Software Test: 6 years
• Test Automation: 5 years
• Married: 2 1/2 years
Current position at Cisco:
Manage a group of 40 test engineers testing a next generation Network Management Operating System
![Page 5: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/5.jpg)
555© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comWhat is a Model?
An abstract representation of your application, stored in memory, that enables a test program to navigate your application.
![Page 6: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/6.jpg)
666© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comVocabulary: States & Transitions
• A State is a screen or other place where an application waits for further input
• A transition is a screen object, keyboard input or other stimulus that causes the application to move to a new state
![Page 7: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/7.jpg)
777© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comModel Based Testing Introduction
• The FSM contains application states and all possible transitions
• Test Case flow is not predetermined
• Can more closely emulate the customer’s use
• New Test Cases are generated on the fly
• Application represented as a Model
• Model is stored as a finite-state-machine (FSM) or tree
![Page 8: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/8.jpg)
888© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comModel vs. Classic Automation
• Classic automation is 100% repetitive
• Nothing changes from one run to the next
• Data-Driven Automation: Data might change but flow through the program stays the same
• Model-based testing introduces randomization of test flow
• Can find bugs you aren’t looking for
![Page 9: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/9.jpg)
999© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comModel Based Testing Introduction (con’t)
• Best introduction:
http://www.geocities.com/harry_robinson_testing/robinson.pdf
Google: “Harry-Robinson Intelligent”
• Model is stored as a finite-state-machine
• The finite-state-machine is then navigated by a choice of algorithms
• Each state usually corresponds to an application screen
• Normal automation scripts can be run for each state
• Transitions can be weighted with a probability to ensure a node is visited more (or less often)
![Page 10: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/10.jpg)
101010© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comA simple Model
• Web Page Tabs
• Each tab is a state
• User can navigate from any state to any other state
![Page 11: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/11.jpg)
111111© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.com
A simple Model: With transitions
New
s
HomeIm
ages Home
News
Images
Home
![Page 12: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/12.jpg)
121212© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.com
A simple Model: With transitions
New
s
HomeIm
ages Home
News
Images
Home1
2
3
4
![Page 13: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/13.jpg)
131313© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comModel Based Testing Advantages
• Introduces a sense of randomness to testing
• Can form the basis of a “soak” test
• Interesting software development for testers
• Navigation through the states can be predetermined, randomized or both:
Random
Shuffle
Directed (Breath or Depth first)
• Can run indefinitely, until every node is visited, or for a set amount of time
![Page 14: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/14.jpg)
141414© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comModel Based Testing Obstacles
• Requires complex data structures to store and manipulate the model
• Full-powered programming language required for implementation
• Can not easily be implemented in many of XDE Tester’s competition
• Relatively high level of software engineering knowledge is required (for a test department)
• Without careful logging and playback ability, problems can be difficult to reproduce
• Business logic to verify function needs to be modular
![Page 15: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/15.jpg)
151515© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comIBM Rational Functional Test For Java & Web
• Formally Known as XDE Tester
• AKA “robotj”
• Java is the scripting Language
• Eclipse is the IDE
• GUI Map stored as an XML file
• ScriptAssure: Manages GUI Changes
• In Production use by Cisco for 18 months
![Page 16: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/16.jpg)
161616
![Page 17: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/17.jpg)
171717© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comDirected Transitions
• The Model contains all possible transitions
• Business logic must still be verified
• In cases where the data entered determines which transition to choose, it’s the job of the business logic verification to determine if the behavior of the program was correct.
• Example:
Login
New User Dialog
Exiting User Screen
![Page 18: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/18.jpg)
181818© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comMain Java objects
• Array of states
• Transition list
• Transition properties
• Verification procedures to call
![Page 19: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/19.jpg)
191919© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comJava objects: State Details
class State { public int m_state; public transitionList m_transitions; public String m_stateName; public boolean bVisited; public int distToDest; public int transitionToDest; // Method VerifyMethod; String VerifyMethod; State(int s)
{ bVisited = false; m_state = s; m_transitions = new transitionList();
}}
![Page 20: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/20.jpg)
202020© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comJava objects: Transition Details
public class transitionList { private java.util.List list = new ArrayList();public void add(transition m)
{ list.add(m); }public transition get(int index) {
return (transition)list.get(index);}public int size() { return list.size(); }}
public class transition {public GuiTestObject m_trigger;public State m_destination;public transition(GuiTestObject o, State s){
m_trigger = o; m_destination = s;}}
![Page 21: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/21.jpg)
212121© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comSample Code Features
• Implements a Model Based test for a simple application
• Can scale to much larger applications
• Navigation:
Random
All Paths
• Timed Execution
• HTML log for test status
• XML Log for replay
![Page 22: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/22.jpg)
222222© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comConclusions
• A Model of an application contains states and all possible transitions into and out of those state
• Model Based Testing will find more bugs because of the random nature of the navigation
• Model Based Testing, like classic automation requires good back-end or business-logic verification
• Model Based Testing requires good software engineering skills
![Page 23: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/23.jpg)
232323© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comObtaining & Running the Sample Code
1. Obtain a copy of Functional Test for Java
http://www-306.ibm.com/software/awdtools/tester/functional/
2. Click on “Trials and Betas” on the right
3. This page is the test application and contains directions for downloading and running the code”
http://www.sherpas.com/mbt
![Page 24: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/24.jpg)
242424© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comMBT - Further Reading
• IBM Developer Works
http://www-128.ibm.com/developerworks/rational/library/05/r-3175/
• Software Test & Performance Magazine (Feb 2005)
http://www.stpmag.com/issues/stp-2005-02.pdf
![Page 25: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/25.jpg)
252525© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.comLearning Java
Data Structures and Algorithms in Java
by Robert Lafore
Thinking In Java
by Bruce Eckelhttp://www.mindview.net/Books/TIJ/
![Page 26: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/26.jpg)
262626© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.com
262626sherpas.com/mbt
![Page 27: 1 © 2004 Jeffrey Feldstein. All rights reserved. Drive Testing for Web-Based and Java Applications with Models Jeffrey Feldstein Cisco Systems jbf@cisco.com](https://reader033.vdocuments.net/reader033/viewer/2022051017/56649d885503460f94a6cd8f/html5/thumbnails/27.jpg)
272727© 2004 Jeffrey Feldstein. All rights reserved.
sherpas.com
272727© 2004 Cisco Systems, Inc. All rights reserved.Presentation_ID