launch & grow your test automation. complex application client/server financial sector –...
TRANSCRIPT
Complex applicationClient/ServerFinancial sector – trading platformVery agile developmentQA
Manually testedOff the shelf automation
NeovestNeovest
Project RealitiesProject RealitiesLimited time to devote to developmentLittle experience in automationLimited budget and no testing engineers were available to work on the projectAutomation needed to maintain quality
Feature set growingContent scope was expandingSoftware release on aggressive schedule
People, Skills, InvestmentPeople, Skills, Investment If we had a larger budget,
we could hire more people
If we had more people or budget, we could quickly increase our skill sets
If we had a more mature skill set, we could save money
Time – major project killer
Our SolutionOur Solution
Controlling our budgetOpen source tools
Proprietary development
Tackling our time and people constraints
Choosing our tool sets
Automation EvolutionAutomation Evolution• Record and PlaybackRecord and Playback
• Procedural ProgrammingProcedural Programming
• Common LibrariesCommon Libraries
• FrameworkFramework
• AnalysisAnalysis
Automation EvolutionAutomation Evolution• Record and PlaybackRecord and Playback
• Procedural ProgrammingProcedural Programming
• Common LibrariesCommon Libraries
• FrameworkFramework
• AnalysisAnalysis
Automation EvolutionAutomation Evolution• Record and PlaybackRecord and Playback
• Procedural ProgrammingProcedural Programming
• Common LibrariesCommon Libraries
• FrameworkFramework
• AnalysisAnalysis
Automation EvolutionAutomation Evolution• Record and PlaybackRecord and Playback
• Procedural ProgrammingProcedural Programming
• Common LibrariesCommon Libraries
• FrameworkFramework
• AnalysisAnalysis
Automation EvolutionAutomation Evolution• Record and PlaybackRecord and Playback
• Procedural ProgrammingProcedural Programming
• Common LibrariesCommon Libraries
• FrameworkFramework
• AnalysisAnalysis
Automation EvolutionAutomation Evolution• Record and PlaybackRecord and Playback
• Procedural ProgrammingProcedural Programming
• Common LibrariesCommon Libraries
• FrameworkFramework
• AnalysisAnalysis
Record and PlaybackRecord and Playback Good
• Quick development of tests, quick to learn, immediate results• Proof of concept rock star – fastest tool
Record and PlaybackRecord and Playback Bad
Very susceptible to user interface changesNo easy result comparison mechanism
Ugly
Hard to maintain
Our Experience (Work in Progress) • No tests• User interaction log• Robo error reports
Record and PlaybackRecord and Playback
Good• Controlling the sprawl of monolithic
code • Reusable, maintainable, extensible
Bad• Susceptible to application changes• Basic reporting• Limited control for running tests
Procedural ProgrammingProcedural Programming
Our ExperienceBasic Result Reports
o Readable, easily accessedo Notification of test resultso Excel is fast, already built, easy to
import values
Best Diabolical Innovation
Great for prototypes
Procedural ProgrammingProcedural Programming
Prototypes and BeyondPrototypes and Beyond
Basic Rule of Software Prototypes
Simple Compelling Metrics
Common LibrariesCommon Libraries
Proliferation of procedures Solid program design was needed
to improve efficiency
Forward Thinking Required in Program Design
Dragon’s Horde Sloppy program design spreads quickly Control Board
Common LibrariesCommon Libraries
GoodStructure and organization
Increases efficiency and capabilities
Extends flexibility
Reduces development time
BadRequires coordinating groupVulnerable to poor planning
Common LibrariesCommon Libraries
Our ExperiencePattern MatcherSimple Optical Character ReaderReporting Upgrade
Common LibrariesCommon Libraries
Our ExperienceControlling the most dynamic elements ‘If’ statements based on version Object mapping
Version based mouse mapping tableLoose Integration with applicationTight integration with application
Common LibrariesCommon Libraries
Our ExperienceControlling the most dynamic elements ‘If’ statements based on version Object mapping
Version based mouse mapping tableLoose Integration with applicationTight integration with application
Common LibrariesCommon Libraries
FrameworksFrameworks Good
Flexible enough to handle a wide of variety tests
Extensible enough to accommodate future changes
Once created – faster development, keeping common code changes to a minimum
Bad
More complex programming task• Changes are made by a few architects
Requires careful design and planning
FrameworksFrameworks
Our Experience
Slingshot
• Load tests as suites
• Schedule and stop tests
• Direct emailed reports
FrameworksFrameworks
FrameworksFrameworksOur Experience
Shotgun
• Load balancing
• Stop selected tests for replacement or reschedule
• Partitioning of environments
Our ExperienceTest failure patternsDatabase test data pointsPerformance Analysis• Track common events• Identify Trends• Quantify “slowness”
AnalysisAnalysis
Our Automation Our Automation Development Plan TodayDevelopment Plan Today
Our Automation Development Plan Today• Typical release cycle
Test 29,000 data points in 120 tests• Typical Development
3-5 day development cycle for standard tests
Your ExperienceExtensible, FlexibleRapid DevelopmentMinimal Maintenance
Cheating EvolutionCheating Evolution
Copyright © 2012 JPMorgan Chase & Co. All rights reserved The views expressed herein do not necessarily represent the views and opinions of JPMorgan Chase & Co. This material is provided for information only and is not intended as a recommendation or an offer or solicitation for the purchase or sale of any security or other financial instrument. In no event shall JP Morgan be liable for any use by any party of, for any decision made or action taken by any party in reliance upon, or for any inaccuracies or errors in, or omissions from, the information contained herein and such information may not be relied upon by you in evaluating the merits of participating in any transaction. JPMorgan and its affiliates may have positions (long or short), effect transactions or make markets in securities or financial instruments mentioned herein, or provide advice or loans to, or participate in the underwriting or restructuring of the obligations of, issuers mentioned herein. Nothing in these materials constitutes a commitment by JP Morgan or any of its affiliates to enter into any transaction. Clients should contact their salesperson at, and execute transactions through, a JP Morgan entity qualified in their home jurisdiction unless governing law permits otherwise. JP Morgan is the marketing name for the investment banking activities of JP Morgan Chase & Co. and its subsidiaries and affiliates worldwide. J.P. Morgan Securities LLC. is a member of FINRA, NYSE and SIPC.